Rules

Never assume automatic Gold Plating.

Most teams are getting the hang of User Stories that have subtasks. Unfortunately the same can’t be said about acceptance criteria.It is so important because real user stories tell a team when the task is done.

Also, Product Owners should not get heartburn because ‘obvious’ functionality was not included. All requirements should be specified in the Acceptance Criteria.

For example, Product Owners should not assume things like:

• They will get a message that says ‘no records found’ or
• The grid will support features such as pagination or sorting

They must be specified in the Acceptance Criteria if required.

There are 2 parts to getting this right: The Acceptance Criteria, then the Acceptance Tests.

Acceptance Criteria (from the Product Owner) define the exact requirements that must be met for the story to be completed. They answer the question, "How will I know when I' m done with the story?"

When you use Microsoft Test Manager to create test plans and test suites for your team project, there are several approaches that you can take.

You may only create 1 test plan that you use for all milestones and add test suites and tests as you progress. This is bad because if you use this approach, you do not have historical data for your test pass rates for previous milestones.

By creating test plans for each Sprint, you can see when a Sprint is complete, based on your testing goals. You can also prepare the test plan for the next Sprint while you complete your testing for the current Sprint.

By using this approach, you can track your testing progress for each of your test plans and see that the quality of your application is improving.

Tip: If you add both manual and automated tests to your test suites, you can view the overall quality based on both of these types of tests for your test suites and test plans.

Reference: Guidance for Creating Test Plans and Test Suites

Follow these steps to create a Test Case in TFS.VisualStudio.com:

This is how assign a tester to test configurations:

Figure: From the Plan menu select the Test Plan, click on the Test Case and then click the Assign button

Figure: Select the rows to assign (Hold down [Ctrl] to select multiple), choose a tester from the available list and click on the Save Assignments button

This how you configure which environments to use for a particular test:

Once the coding is done by the developers, the functionality must then be stepped through in the required browsers. You can do this manually or automating it using a great tool like Microsoft Test Manager.

The 1st step in getting automated tests, is to setup Acceptance Tests:

Tip: You can pass a test from the test list. Select the Test menu, then the Test Suite. Choose the Test Case to pass and then click the green button ‘Pass Test’.

The next step is to review the Statistics of the Sprint.

Developers think they are done when they finish coding and check in.

Wrong. It is much better to use Microsoft Test Manager (MTM) and step through the Acceptance Tests.

Once you are doing that, this is how you check the status of the current Sprint:

Key:

• The red is work remaining for the developers, and
• The blue is working remaining for the testers (unfinished testing)

Acceptance Tests (built by the developers) verify that the Acceptance Criteria are met.

The goal is for teams to move beyond manual testing and implement automated testing. E.g. CodedUI tests, Telerik Tests etc

Test cases answer the question, "How do I test and what are the test steps?"

In an agile team, pre-planning all your tests is not always the most efficient use of time for testers. PBIs can change direction, scope, and priority, and pre-planned tests are likely to change.

Exploratory testing provides the best way to create repeatable tests from the acceptance criteria - as you need them.

There are two ways to run an exploratory test in Microsoft Test Manager.

When you start an Exploratory test, you don't see any test steps, but you can click on the title of the requirement to see its Acceptance Criteria.

If you find a bug while testing, click the Create bug button to add a bug related to the PBI.

By default, the reproduction steps will be populated with the last 10 actions you took (you can change this and other defaults with configuration). You can cut this down to just the relevant actions by clicking Change steps.

Now you have a bug, you should create a matching test case so you can verify when the bug is fixed. This also gives you a handy regression test to help ensure the problem isn't reproduced later.

Again, the steps are prepopulated from your bug steps.

Use Microsoft's Exploratory Testing - Test & Feedback extension - to perform exploratory tests on web apps directly from the browser.

Capture screenshots, annotate them and submit bugs as you explore your web app - all directly from Chrome (or Firefox) browser. Test on any platform (Windows, Mac or Linux), on different devices. No need for predefined test cases or test steps. Track your bugs in the cloud with Azure DevOps.

Video: Ravi walks Adam through the exploratory testing extension - You can also watch on SSW TV

Video: Ravi Shanker and Adam Cogan talk about the test improvements in Azure DevOps and the Chrome Test & Feedback extension - You can also watch on SSW TV

1. Go to Visual Studio Marketplace and install "Test & Feedback".

2. Click "Add to Chrome" to add the extension to the browser on your computer.

3. Go to Chrome.
4. Start a session by clicking on the Chrome extension and then click start a session.

5. Upload the screenshot to a PBI.

More info: Azure DevOps - Explore work items with the Test & Feedback extension

You organize your Test Cases by adding them to a Test Plan (also called a Test Suite).

We have one Test Plan per sprint.

Figure: Select Plan from the main menu, and then choose the Test Plan that you would like to add Test Cases to

Figure: Add a filter to make it easy to find your Test Cases. Click the Run button, select the relevant items and click the 'Add Test Cases' button

Figure: Your Test Case has now been added to the suite

Running tests with MTM allows you to keep track of your testing progress.

Problem Steps Recorder is a useful tool that allows anyone to capture details of a bug. Best of all, It's already installed on your Windows PC!

Bug reports can come from anywhere and anyone - the more people reporting back, the better.

Once a problem has been discovered, it helps to collect as much information as possible. Although there are many useful tools you can give to your test team, Problem Details Recorder has one big advantage: It's shipped with Windows since Windows 7 so anyone can record details of their issue with no preinstalled software.

Once recording, the user can repeat whatever steps to reproduce an issue.

In the old days, reading and understanding test cases was something only developers could do. Behavioral-Driven Design (BDD) starts to solve this problem by enabling organizations to define their use cases in plain language and integrate these aspects with testing frameworks.

Using Gherkin syntax and a BDD framework like SpecFlow you can write test scenarios in plain language using a few key words (Given, When, Then). Plain language makes the test scenarios easy to understand, even for non-technical team members.

First think about the different scenarios that you want to test, then write them out in plain language using gherkin syntax.

Feature: Greeting Message
	Participant sees a greeting message

Scenario: Participant sees a greeting message
	Given I visit the website
	When I navigate to the greeting screen
	Then I see the greeting message

Figure: Good example - Gherkin syntax scenarios (Given, When, Then)

Once you have your scenarios lined up, you should begin to write the test steps for each scenario.

[Given(@"I visit the website")]
public async Task VisitTheWebsite()
{
    await HomePage.NavigateAsync();
}

[When(@"I navigate to the greeting screen")]
public async Task NavigateToWelcome()
{
    await HomePage.NavigateToGreeting();
}


[Then(@"I see the greeting message")]
public async Task ThenISeeTheGreetingMessage()
{
    var message = await HomePage.GetGreetingMessage();
    Assert.IsTrue(message == GreetingMessage);
}

Figure: Good example - Test steps to run, matching the Gherkin Syntax

Automated UI testing (aka end-to-end testing) is an awesome way to automate the process of browser based testing.

In the old days, Selenium was the gold standard, but these days it has been overtaken by Playwright which lets you write tests in many popular languages including .NET, Java, Python and Node.js

Playwright has a few advantages over Selenium:

• Actionability
• Performance
• Stability
• Switching browser contexts for parallel testing
• and more...

//Store the ID of the original window
const originalWindow = await driver.getWindowHandle();

//Check we don't have other windows open already
assert((await driver.getAllWindowHandles()).length === 1);

//Click the link which opens in a new window
await driver.findElement(By.linkText('new window')).click();

//Wait for the new window or tab
await driver.wait(
    async () => (await driver.getAllWindowHandles()).length === 2,
    10000
  );

//Loop through until we find a new window handle
const windows = await driver.getAllWindowHandles();
windows.forEach(async handle => {
  if (handle !== originalWindow) {
    await driver.switchTo().window(handle);
  }
});

//Wait for the new tab to finish loading content
await driver.wait(until.titleIs('Selenium documentation'), 10000);

const { chromium } = require('playwright');

// Create a Chromium browser instance
const browser = await chromium.launch();

// Create two isolated browser contexts
const userContext = await browser.newContext();
const adminContext = await browser.newContext();

// Create pages and interact with contexts independently

Playwright codegen

Playwright offers a super cool feature that lets developers record actions in the browser to automatically generate the code for tests.