Secret ingredients to quality software

SSW Foursquare

Rules to Better Bug Management and Product Feedback - 8 Rules

If you still need help, visit SSW Consulting Services and book in a consultant.

  1. How do you want customers to send you feedback? Phone calls? Emails? A website?There are a number of web applications that do a great job on this:

    codeauditoruservoice
    Figure: The UserVoice website allows user to enter suggestions (used here by SSW Code Auditor)

    admin
    Figure: UserVoice has an Administrator console to track feedback

    UserVoice is the most popular platform to collect, manage, and prioritize user feedback. It has a voting and tickets system out of the box.

    Many software houses use this for their products Eg. SSW CodeAuditor, SSW LinkAuditor, etc

    Here are the google results as at 2014:

    uservoice jp
    Figure: Google result of UserVoice

    getsatisfaction
    Figure: Google result of GetSatisfaction

    googleresultuserecho
    Figure: Google result of UserEcho

  2. If you are unclear use IM to ask, but remember the golden rule is to not send tasks on Teams.

    It is recommended to keep track of active project backlogs on the company intranet, while also including the Product Owner and Tech Lead contact information, coupled with a link to the Teams channel of that project.

    When reporting bugs and giving product feedback, it is essential that you are as descriptive as possible. This will save both you and the developer time and frustration in the long run.

    Here are the 8 tips:

    Tip 1: Draft your bug with enough details
    Tip 2: Draft your suggestion with the complaint and what you expect to see
    Tip 3: Should you send this to the Product Owner or the Tech Lead?
    Tip 4: Should you email or put it in the backlog?
    Tip 5: Do you make it easy to find all the backlog in your company?
    Tip 6: Make sure when using backlog, the Product Owner will still get an email
    Tip 7: Separate PBIs
    Tip 8: Use emojis and prefixes for PBI/Issues titles, or email subjects

    report bugs and suggestions
    Figure: Making the Product Backlog the main source of tasks

    Tip 1: Draft your bug with enough details

    Make sure you always explain and give as many details as you can of how you got an error or a bad experience. Detailed and useful descriptions can make finding the solution quicker and easier. The goal is to include enough details so the developer can focus on the development work more rather than trying to figure out what the feature requirements or bugs are. 

    See rules on Do you have a clear definition of a bug?

    External source: How to produce a good bug report

    Figure: Bad Example - This email isn't going to help the developer much - it is vague, has no screen capture or other details to help reproducing the error

    Figure: Good Example - This email includes the product name and version, the category of the issue (BUG), a screen capture, and informs the user's system

    When possible, a great template to follow is the Functional Bug template from the ASP.NET open-source project. Spending time to provide as much detail as possible, by ensuring you have the 3 critical components:

    • Steps to reproduce,
    • Expected outcome, and
    • Actual outcome

    Figure: Bad example - Lack of details

    Figure: Good example - We can easily identify more the one way to improve the UX and there's a clear suggestion to action

    Better than a good textual description of a bug report is a screen recording. This should be followed for a more detailed report. Use Snagit or Camtasia to record your screen.

    Figure: Good example - Recording bug reports in a video can make the issue clearer to see

    Tip 2: Draft your suggestion with the complaint and what you expect to see

    Define all the requirements as per Do your User Stories include Acceptance Criteria?

    Better than a good textual description of a suggestion request is a screen recording. This should be followed for a more detailed report. Use Snagit or Camtasia to record your screen.

    Figure: Good example - Giving suggestion requests via video

    Tip 3: Should you send this to the Product Owner or the Tech Lead?

    It depends on the team, but often the Product Owner is busy. If you know the Tech Lead and your suggestion is obviously a good one, then you should email the Tech Leader and Cc the Product Owner. The Product Owner can always reply if they don’t like the suggestion.

    For a bug email:
      To: Tech Lead
      Cc: Product Owner
      Subject: Bug - xxx

    For a new feature email:
      To: Tech Lead
      Cc: Product Owner
      Subject: Suggestion - xxx

    Tip 4: Should you email or put it in the backlog?

    Always go for backlog if you have access to a backlog management system otherwise email relevant people. You may have a group email such as all@northwind.com.au, You would only Cc this email when a greater visibility is required.

    Tip 5: Do you make it easy to find all the backlog in your company?

    do you know how to report bugs and give suggestions
    Figure: An intranet page with links to projects’ backlog to make it easy for everyone to find. Note some projects have more than 1 backlog.

    Tip 6: Make sure when using backlog, the Product Owner will still get an email

    Create an Issue/PBI and @mention relevant people (GitHub and Azure DevOps will generate a nicely formatted email)

    See rules on Do you know when you use @ mentions in a PBI?

    Tip 7: Separate PBIs

    If they are all related to one area, then you could consider put them together, otherwise don’t bunch them up.

    See rules on Do you send tasks one email at a time?

    Tip 8: Use emojis and prefixes for PBI/Issues titles, or email subjects

    When you create a bug/suggestion to a backlog, it's a good idea to add emoji in the title. Not only does it look nicer, but people can look at the item and take in the necessary information quickly.

    This means that anyone looking at the backlog can glean its nature at a glance, rather than having to read each item to know what category it is (5 bugs, 2 features, etc). Examples:

    • 🐛 Bug - Calendar is not showing on iOS devices
    • ✨ Feature - Add 'Back to menu' item to top navigation

    Check out the rule on Do you know which emojis to use in Scrum?

    Tip: GitHub Issue Templates can help you with that.

  3. Do you have a clear definition of a bug?

    The answer to this question can make or break contracts. We think that it's such a fundamental issue it has to be captured clearly. This is how we strictly define a bug.

    bug feature

    A software issue can be classed as a bug where:

    1. The application crashes to code (excluding bugs resulting from third party products (e.g. "blue screen of death" or crashing in a third party data grid that we cannot control); or
    2. The application displays data inconsistent with the specified business rules; or
    3. The application is missing functionality specified ** in the specification; **or
    4. The page design/layout is substantially inconsistent with the agreed mock-ups and the developers can reproduce the above on the test server and the application is not yet "live" and the issue has been reported in time (generally 30 days).

    Examples of what *could* constistute a bug

    1. The application crashes to code because it doesn't check that a connection is valid before running a stored procedure (this is likely covered because it crashes to code)
      YellowScreenofDeath
      Figure: Yellow screen of death
    2. A sum total is negative instead of positive because the wrong operator (plus instead of minus) has been used to calculate the running balance (this is likely covered because data is inconsistent with the specified business rules)
      IncorrectSum
      Figure: An incorrect sum is likely to be a bug
    3. The application is missing the Monthly Sales report (this is likely covered because the application is missing functionality specified in the specification)
    4. The output HTML in the application is formatted way out of line and does not display in the specified browser (e.g. Internet Explorer 9) (this is likely covered because it substantially inconsistent with the agreed mockup)

    Examples of what is *not* a bug

    1. Any problem caused by software or an application not written by the organization supplying the software.
    2. The customer requirement was not included in the user interface/mock-ups/specifications.
    3. The client decides that they don't like the look of the current form even though it is the substantially the same as shown in the specification and wants the buttons at the bottom of the form instead of at the top.
    4. The original specification states that the total price excludes GST, but it really should have included GST. This is a change to the specification, and is not included in the contract.

    Using Work Items in Azure DevOps or GitHub

    Using a Work Tracking tool allows you to create work items such as user stories, bugs, tasks, test cases etc. Only create bugs for defects, faults, flaws, or imperfections that fulfill your definition of a bug. For everything else use other work item types.

    2016 02 08 12 20 59
    Figure: Do I create this as a bug, or a task?

    Handling additional work for fixed-price contracts

    Scrum wasn't designed for fixed price, fixed scope contracts, however, any new features or modifications (non-bug items) not in the original Sprint or Sprints are classed as additional work and are outside the scope of the contract. Any tasks which are bugs should be marked as additional items and be completed in the current sprint if possible. Most importantly, after the sprint plan has been sent, a PBI should NOT be entered as an item (additional or otherwise) in ANY sprints if they are not a bug. Instead, move all non-bug items to the Product Backlog for future review after the warranty period for the fixed price contract has passed.

    Handling additional work in a Scrum project

    Any new features or modifications (non-bug items) not in the original Product Backlog are classed as additional PBI's and placed on the Product Backlog. Any tasks which are bugs found during the current Sprint should be fixed within the current Sprint. Any tasks which are bugs found outside of the current Sprint should be added to the Product Backlog. See Do you know when to create bugs? and Do you know the 3 steps to a PBI?

    62034c tfs preview add bug
    Figure: Adding a bug to the Product Backlog in AzureDevOps

    Note: The above is our definition. Others have different definitions that we do not subscribe to: Painless Bug Tracking.

    You can also use the Wiki definition of "Software Bug" as a reference to understand this concept: Wikipedia Definition of Software Bug.

  4. Imagine this scenario... Mary notices a small typo on a page in her intranet. As a good employee, she opens up the backlog and reports the spelling error. As she sends it she says to herself "That took more time to report the error than it would have taken me to fix it".

    Small errors should be fixed by the person who found them stright away. Text changes can be easily done in SharePoint or WordPress. If you know who the culprit is, it might be a good idea to inform that person, including the things you have fixed.

  5. If you need more than 20 minutes of work to perform a task, then it's not free, and you need to get approval for it.

    Follow this template to reply:

    Figure: Good Example - Ask for billable hours approval if a support request demands more than 20 minutes

  6. Do you document "TODO" tasks?

    When you have an idea for content or notice some content is missing and cannot be written straight away, it is important to document it. It should be done by adding the words "TODO:" followed by what you want to be added there.

    GitHub

    For GitHub projects, creating an issue using "TODO" as prefix is the preferred way.

    VS Code Extension

    In VS Code, we recommend using the extension Todo Tree. You can find TODOs and highlight them in open files.

  7. A new version of your software is deployed. How do you tell users important information on older versions?

    You shouldn't need to check for updates to be notified of valuable information.

    Software is often deployed without any mechanism to insert a message into older versions.

    Messages might include notifications to update or simply helpful tips. Eg. Sometimes customers are not aware that their CRM installation is several years old.

    VisualStudioUpdateNotifications
    Figure: Visual Studio 2019 has helpful notifications to indicate which parts need updating

  8. Do you know the best way to do A/B testing?

    A/B Testing is the process of testing different versions of an application on different users to gather empirical evidence to learn which version is better.

    Using A/B Testing enables you to get features tested and when used effectively means that a bug will never be deployed to 100% of users. Generally, new features should be tested on 20% of users and rolled out to others once they are reliable.

    Video: What is A/B Testing? | Data Science in Minutes

    There are several ways this can be done...

    Feature flags are a modern way to toggle features for users. They are essentially a little bit of code that can be turned on and off at will. That means you can choose, when features are deployed and who gets them.

    Feature flags are often implemented by developers writing their own code. However, there are better solutions today:

    Azure Deployment Slots

    Azure Deployment Slots are another way of doing A/B testing, you essentially deploy 2 versions of your app and then direct traffic to different versions.

    Azure FrontDoor

    Azure FrontDoor is an offering that lets developers direct traffic to different versions of an app.

We open source. Powered by GitHub