Secret ingredients to quality software

SSW Foursquare

Rules to Better Specification Reviews - 13 Rules

Specification Reviews are the 1st step to properly engaging with a client and need to be done well.
The following are rules that will make sure you know how and how much, to spec-out upfront.

A lot of teams don’t spend much time reviewing what the software is intended to do, before starting implementation.Some agile(ish) environments seem to think the (often simplistic) user story is everything and taking the time to really spec things out, is deemed “too waterfall”. There’s a balance, obviously, but getting multiple perspectives from different stakeholders (both business and technical) during reviews, is essential to get the intended function and behaviour right.

Having senior testers involved in this step can be valuable as they bring different skills and perspectives to the review, especially as they’re generally one of the few people in the team with a focus on what might go wrong (via a critical thinking mindset), rather than focusing on success (typical of the builder’s mindset).

Any review still leaves stuff on the table. There will always be the unknown ‘unknowns’ that only reveal themselves as you actually get into the work. This makes knowledge work like software development (and testing as part of it) difficult to estimate accurately. Risk is what’s left over when you think you’ve thought of everything. It is important to acknowledge that we will always learn new things as we proceed through development and testing – no matter how thorough we try to be up front.

Tools like journey maps are a useful input into both development and testing, providing fertile ground for coming up with test ideas and identifying areas of risk to influence test coverage.

  1. A client will often ask for a proposal or ballpark for the project. It is very difficult to give them the price for a large project without first conducting a Specification Review.

    The Spec Review is a process that will demonstrate to the client whether you have the commercial sense to understand their business and have the technical and management capacity to complete the project.

    It is paid work conducted after the initial meeting to determine the overall scope, feasibility, and ballpark costs of the project (i.e. $50k or $500k).

    E.g. Mr. Northwind learns that the idea he presented at the initial meeting will cost approximately $80K and he has to determine if that is feasible to his business or if he will trim the functionality to better manage the cost.

    proposal
    Figure: A ballpark or proposal should start small and not be a big commitment

    "From this initial meeting, the ballpark is 6 months and $200k+GST"

    Figure: Bad example - big scary figure

    "From this initial meeting, we will first need to conduct a Specification Review.
    This first step is $8k - a 2-day Specification Review"

    Figure: Good example - work in small chunks of work with details about what you will do

    Spec Review length

    The Specification Review is conducted by two experienced developers at the client premises in close consultation with the client. The time allocated for a Spec Review is generally 1 - 5 days depending on initial expectations of the project. The rule of thumb is 1 - 2 days of Spec Review per estimated month of project time.

    The purpose is to understand the whole project but, if the project is greater than six months, focus primarily on the first six months.

    Talk about business requirements

    • Conduct workshops: Conduct workshops with different groups of users (e.g. management, back office, customer service) to build the "Product Backlog" which the business wants. This ensures that all users get their say. Some "nice-to-haves" might actually be quite easy to implement. Product Backlog Items can then be prioritized and fleshed out.
    • Review documentation: Reviewing any documentation the client may already have. Remember clients are mostly looking to software consultants to assist them in solving business problems.
    • Keep technology discussions short: Unless they have a specific business purpose, detailed discussions about technology with the client are unlikely to be useful. For example, most clients won't be interested in a discussion about whether to use .NET or Angular at this stage.
    • Identify an MVP: Most client can't afford everything they want, so make sure you're keeping track of the minimum we can do to deliver value.

    Do something valuable

    Most software consulting experts will be able to provide a small improvement to the current system 'on the fly' during the Spec Review. This may be something as simple as adding an index to a table and thereby increasing the performance of a web page.

    Use 'Corridor Conversations'

    ProjectManagement Suprise
    Figure: Use corridor conversations to prevent nasty surprises

    The hallway is your friend. It's a place where you can gather a lot of information informally.

    Formal meetings can have a "Us vs Them" feel - While the information collected and the conclusions of the Spec Review are presented formally at the end of the Review, it is important that the consultants convey key points to the client as they emerge through the course of the Review. The formal presentation is not the time to be presenting new information to the client.

    Addressing key potential sticking points of budget and technology informally during the course of the Spec Review relieves the potential for unwelcome surprises during the Spec Review presentation. Canvassing the issues beforehand in casual 'corridor conversations' clears the decks for an agreement, rather than increasing the risk of heated discussions if you surprise a client at a formal meeting.

    For example, ask the client "building the cube will add around 2 months of development time, shall we leave this out of the current scope, or do you want it in?"

    Estimates expressed in round numbers (or exact numbers for fixed price)

    Remember, no politician challenging for leadership ever calls a vote before he or she knows the numbers; you too will avoid presenting a solution at a meeting if you aren't convinced the client is already agreeable.

    Through the course of the Spec Review, the client will be aware that each month for senior software consultants is generally tens of thousands of dollars. Squabbling over $500 here or there in the ballpark phase is a level of detail neither side can be confident of. Clients need to be realistic about what they get for their money.

    "Now that we've spent a few days speccing this out, we believe the solution will take approximately 6 months which is $204,000+GST"

    Bad Example - Far too firm a price when you don't know any of the detail

    "Now that we've spent a few days speccing this out, our projection is the project will take a minimum of 6 man-months (around $200,000+GST) to complete but this may change depending on what is finally agreed in the Specification. The price will vary depending on resources used and the time that elapses over testing.

    Good Example - leaves some wriggle room at these initial phases

    Read When do you use approximate values for project costs?

    Technology options

    At this stage, you want to consider the most relevant technologies. For example, SSW will likely pursue recent Microsoft technologies. Some clients might want to do their own research or need some time to think about their options before agreeing to newer technologies.

    Proposal

    You should follow Rules to Better Proposals when documenting a Specification Review.

    Test Please

    The consultant must run a Test Please by another developer and your Account Manager before anything is formally presented to the client.

    The main things the Account Manager will be looking for will be:

    • Have you already had corridor conversations with the client so they're already expecting the ballpark estimate?
    • Are the estimates realistic and still incorporating any relevant buffers? e.g. Project Management, bug fixing, etc.

    Tip: Record a video of your PowerPoint presentation. Ideally a 5-minute summary is very handy for people that did not attend the meeting to decide if they should go ahead. Also if new developers join the project later on, this video is a nice handover.

    You can record your presentation using Recording Studio in PowerPoint or Camtasia's PowerPoint Add-In Toolbar.

    Example Schedule for a 1-day Specification Review

    You want to have all the required work for a spec review done within the allocated time, so it’s important you leave time to implement the required changes after you present and before you email the final version.

    • 9 am: Meet with the client and discuss requirements
    • 11 am: Start work on the backlog and the PowerPoint (.PPTX) or Word Doc (.DOCX)
    • 3 pm: Present to the client and gather feedback for changes
    • 5 pm: Implement changes
    • 6 pm: Send “As per our conversation” with Word or PowerPoint attachment

    In a 2 or 3-day spec review, you should assume you’ll need more time to implement changes, so push the presentation time forward to 2 pm.

    Timesheets

    Always track timesheets against a separate Spec Review project.

    It makes project cost reporting difficult later if you don't, since the cost we're looking for is always the "post-Spec Review cost" that clients can compare to the estimate you gave in your Spec Review.

    project costs
    Figure: You wouldn't want these numbers combined, as it may look like a budget overrun

  2. Usually, a specification process is done with the client before beginning work on a project, just like you would never build a house without getting an architect to create a plan.

    As you might appreciate, it is not realistic to understand the complexity of your system and give you a realistic estimate after a brief meeting. Our experience tells us we will need to spend a few days to obtain and document the requirements from your project’s stakeholders. This will help you turn your ideas into a more detailed roadmap.

    ProjectManagement BallPark Catch
    Figure: Remember, a batter aims to hit the ball way out of the ballpark. Don't set an indefensible boundary too early by estimating too small

    The deliverables for the Specification Review depend upon how large the application is and the time we have spent on the review. You should have the following:

    Requirements Analysis

    • An architectural roadmap recommending technical solutions
    • A breakdown of the required software application into its core components, likely to include the approximate number of main features (e.g. forms, reports, etc.)
    • An integration plan
    • A deployment strategy
    • An MVP (minimum viable product) will be identified, as well as a wish list - requiring the client to set the priorities for the project through defining what is in and out of scope for the MVP
    • A detailed list of 'issues' associated with the existing system which impact future development and maintenance
    • Hardware and licensed software recommendations
    • Mock-ups (if required)

    Summary Product Backlog

    • A list of product backlog items (PBIs) broken down based on the Requirements Analysis and the Architectural Design
    • These PBIs will then be estimated

    Ballpark Estimates

    • The estimated number of Sprints
    • Estimated cost of the project

    With the Specification Review, the client can see whether the consultant understands their business and the requirements for their software development project.

    The ballpark estimate allows them to decide whether the project is feasible for their budget and time-frame.

  3. Estimates contain 2 main classes of work:

    • Relating to the particular product (e.g. create page 'customers.aspx')
    • Relating to the project as a whole (e.g. management, administration, testing, software audit etc.).

    PBIs may only make up about 60% of the total project time. Project Managers and developers should not think that the only work being charged on a project are coding tasks.

    General Project Costs

    Management costs can change depending on how much management the client requires. You should recommend a suitable level of management. 'Management, accountability and transparency' has a cost.

    You should add general project costs as a % of the work items generally in line with the following (note that these numbers are just best guesses):

    • Testing: 20%
    • Bug Fixes: 20%
    • Software Audit (if relevant): 4 hours per Release - usually conducted by two experienced Architects
    • Fixes from the Software Audit: 5%
    • DevOps: 10%
    • Project Management: 15% - this includes items like stand up meetings, timesheets, standard updates, reviews, etc.
    • Unknowns (for risky projects): 10%. While this is arbitrary it raises awareness for everybody that 'there are things we still don't know!'

    Project Specific Costs

    Estimates for a project should be done by a developer, checked by another developer, and finally triple checked by an Account Manager. While every project is different in some way, there are common elements.

    SSW has built an estimates calculator to assist in creating estimates. See the Estimates Calculator here.

    If the client requires a fixed price quotation, a 20% premium is added to the estimates for the sprints specified in the Specification Release only (i.e. a fixed price is not given on the entire project). Requests for variations to a fixed price contract must wait until the contract is completed. If development is based on a fixed price contract, work is completed offsite only to facilitate project management and prevent unauthorized scope development.

    Note: A suggestion for Microsoft - It would be great if Azure DevOps had functionality to “Add Standard Items" to a Sprint.

  4. How to create an estimate for a Spec Review (Summary)

    This process can take up to a few days, so if you're just after a ballpark, use epics instead of PBIs (Product Backlog Items).

    Here are the 8 steps:

    1. List all of the PBIs into a Backlog in Azure DevOps (or visualstudio.com), sizing them with story points
    2. Open Excel and connect to the above Backlog
    3. In Excel, add a column called "Hours"
      Note: Once we move to estimating in time, this is no longer Scrum
    4. In Excel, copy this list and paste into another spreadsheet called the Estimates Calculator (see below for how to use this), in order to add all of the extra items (Testing, DevOps, Project management, etc.)
      Note: It would be great if Azure DevOps had functionality to “Add Standard Items" to a Sprint
    5. Run a Test Please on the numbers
    6. Add a screenshot of this spreadsheet to your specification review document under Costing Summary
    7. Present to the client
    8. When the estimate is approved by the client, start work following Rules to Better Scrum using Azure DevOps (Work Items).

    Extra - How to use "Estimates Calculator"

    Open the Estimates Calculator and do the following:

    Resource tab
    Figure: Set the types and numbers of resources who will be working on the project

    Estimates tab
    Figure: Enter your PBIs and estimates

    Microsoft Project is sophisticated waterfall planning software that has powerful features for auto-scheduling and dependency allocation (Note: Project allows you to add 2 people to a task, and then the calculations and dependencies are all worked out). However:

    • In MS Project, tasks are auto scheduled based on dependency and resource allocation (who is assigned to it). This generates an estimated completion date which is never accurate, due to annual leave, public holidays and general shuffling of people in the team
    • It gets the summing wrong (the totals don't seem to update and no way to trigger it)
    • It's hard to synchronize with timesheets (as it doesn't connect to 3rd party timesheet systems out of the box – however, it does have its own time sheeting system... that is missing billing information!)
  5. The findings of the Spec Review should be presented by the developers and the Account Manager at a meeting with the key decision makers of the project for review and acceptance, generally in the form of a PowerPoint presentation, or sometimes a word doc. It is important that all the required people are in a room together to review the plan.

    ProjectManagement DecisionMakers Small
    Figure: It's a lot easier to get a signature when you've got the right people in the room

    If you've run the Spec Review successfully the client should not be surprised by anything you present. This means discussions should focus on issues such as what particular requirements could be added into the scope for the MVP, or what releases can be completed by what date, rather than spending the meeting helping the client regain consciousness after they blacked out from seeing the price.

    The Spec Review presentation should be scheduled by the Consultant or Account Manager for the afternoon of the final day of the Spec Review.

    “I will send you a proposal when I get back to the office.”

    Figure: Bad example – a common mistake is to tell the client you will complete it later

    “Let’s schedule a meeting now for Wed 3 pm. I will send a meeting invite to all the stakeholders.”

    Figure: Good example – this is an appointment with a specific time for the next schedule The benefits are:

    • You are striking while the iron is hot
    • All parties benefit while the information is fresh in their minds
    • The client won't experience the inevitable delays when you go back to the office and get stuck on other client issues that appear more urgent

    What does the client get at the conclusion of the Spec Review?

    • Option A - Email (if they want to minimize documentation time)
    • Option B - Word document (if they need to get approval from someone higher up)
    • Option C - PowerPoint presentation (if they are the decision maker, and they don't want a doc)

      3 01 2014 10 13 04 PM
      Figure: Good example - Export your PowerPoint presentation as a video

    • Option D - Video of PowerPoint presentation with narrations exported to a video (the best option to gather more feedback, you can even gather public feedback E.g. PointBank https://sswconsulting.github.io/PointBank/)

      Note: Make sure to name your video according to including version numbers in your file. Publish your video to YouTube afterwards so you can easily share it with colleagues and clients.

    Video Examples

    Figure: Good example – FireBootCamp - SugarLearning Specification Video

    Figure: Good example – FireBootCamp - TimePro Invoicing Specification Video

    Once again, the presentation needs to pass a 'Test Please' from another senior employee before the meeting takes place.

  6. Different clients will have different levels of documentation on what they want to be built. You need to be ready to do a spec review for any one of the following possible cases:

    There are 5 levels/types of Specifications:

    1. I have an idea...

    Run from this
    or
    Verify they have a really hefty bank account!

    2. High-Level Requirements Document

    This will read like a wish list with no details and many unanswered questions. High-Level requirements are very vague and open to many interpretations.

    3. Detailed Requirements Document

    The details have been fleshed out and allow developers to write Functional and Technical Specifications.

    • We need a login page for northwind.com
    • Must match existing site look and feel
    • Username is already in the Users table in the ABC database (SQL Server)
    • Password should be at least 8 characters
    • .NET is already used for the existing site so that is what this should use of course

    LoginInterface
    Figure: Detailed Requirements have more of the details you want

    4. Functional Specification

    This will include detailed mock-ups for the UI, use cases/user stories and might be at a level to allow for fixed price quoting on the project. Functional Specifications go into more detail about the user interface and interactions in the system.

    • We need a login page for northwind.com
    • Must match existing site look and feel
    • Users table must be defined and added to the ABC database (SQL Server 2008)
    • Username consists of user first initial and first 7 characters of the last name

      • For example Joe Jones -> jjones
    • Password should be at least 8 characters
    • Site uses .NET 4 and this interface must be added to existing project
    • This is the layout for the login interface
    • A red asterisk (*) should be displayed if a value is left blank and Submit is pressed

    5. Technical Specification

    This is the blueprint for the application. There should be no unanswered questions and should allow for a fixed-price quote.

    • We need a login page for northwind.com
    • Must match existing site look and feel
    • Users table must be defined and added to the ABC database (SQL Server 2008)
    • Username consists of user first initial and first 7 characters of the last name

      • For example Joe Jones -> jjones
    • Password should be at least 8 characters
    • Site uses .NET 4 and this interface must be added to existing project
    • Define the data model explicitly
      Table
    • Must work with IE7, IE8, IE9, and FF3
    • Must display correctly at 1024x768 resolution
    • Must support ANSI characters for Username and Password
    • Will not support mobile browsers
    • Will not be tested with localization (assumes en-us local on US versions of software)
    • SQL Membership provider will be leveraged
  7. At the end of the Spec Review encourage the project to go ahead with the current resources, while it is fresh in their heads.

    Say words to the effect of:

    "The resources allocated to your project depend on availability. All efforts will be made to allocate the original resources used on the Spec Review, but that cannot be guaranteed.
    If you decide today or tomorrow, then we can use the same resources...
    Regardless, it is always better to move ahead while the project is fresh in the developers' heads."

  8. Product Backlog Items (PBIs) can be described in the form of a "User Stories" when appropriate. It ensures the developers will know the context for a PBI.

    As a [type of User]
    I want [some goal]
    So that [some reason]

    Figure: User Story - template for description

    user story azuredevops
    Figure: Example User Story in an Azure DevOps PBI

    userstory github
    Figure: Example User Story in a GitHub Issue

    I want to be able to search for customers.

    Figure: Bad Example - the user story is too vague and broad in scope

    As a Marketing Manager,
    I want to be able to search for customers by country and last name,
    So that I can find their numbers and call customers that are close to me.

    Figure: Good Example - Clear user story following the INVEST principle

  9. Specification Reviews can be a long process and not being able to talk with the relevant parties that hold the vision and the budget can cause a bump in the road.

    Stakeholders are the people with the business requirements or people with the $ purse strings. They are not the devs.

    When organizing the Spec Review be assertive and make sure that all stakeholders will be available. There is no point in holding a Spec Review if the stakeholders aren't present to make sure the business requirements are recorded.

    Make sure the stakeholders are aware of what is needed ahead of time. If all relevant stakeholders aren’t available, be sure to warn that this will cause issues down the line. Remember if no one in the Spec Review holds both the budget and vision, the product that you write up will look a lot different to what the business really needs.

  10. Getting accurate information from a client during a Spec Review ensures that the full scope of the project is captured and there are no surprises later on. Accurately refining the product avoids waste.

    Here are some tips that should help focus in on a feature.

    Tip 1: Check if the Product Owner can explain and show their process

    Look at how they perform the task manually and listen to them explain the process.

    ComplexSpreadsheet
    Figure: Look at the manual process in this complicated Excel. If the process has never been done before, do it in Excel first

    Tip 2: Draw a Mock-up or process flow diagram as you listen to the requirements

    Creating mockups or diagrams of the feature will be highly beneficial as the client can see the feature visually.

    Tip 3: Consider a Prototype

    Consider building a functional prototype (works, but throw away code).

    Tip 4: Record Specification Reviews

    Recording the Specification Review is another great way to make the information as clear as possible. Being able to look back and clearly lookup what was mentioned, saves re-asking questions.

    Tip 5: Consider a Journey Map

    Journey Maps help to highlight which person is responsible for decisions at various stages.

    E.g. On this Journey Map observe the experience for the Producer and the Loan Officer. See how they move through the steps of obtaining and maintaining a farm loan.

    The map highlights the emotional journey as described by these customers, including their points of pain and delight. In addition, you’ll find the technologies, systems, and touchpoint types used through the process.
    From USDA Direct Farm Loans Journey

    usda journey map
    Figure: A Journey Map for 2 stakeholders

    See the high-res version PDF.

    Tip 6: Does the client know why you need to understand?

    “I am not a code monkey. I solve business problems with code. I need to understand that problem to do that well.”

    From Bryan Finster

    1650533417256
    Source: Bryan Finster on LinkedIn

  11. Do you limit the project scope?

    A client will often ask for more features than are really necessary for the MVP. They may also think that implementing a feature is easier than it is.

    There are a few things you can do to control the scope of the project and deliver the client the most value.

    Tip 1: Flag problematic tasks as soon as possible

    If a feature is going to take a long time or isn't going to deliver much business value, make sure to flag it to the client and make sure it really is a priority. Explain that you can always deliver the feature at a later date after they go live. By following these steps, only the features in the initial project are developed, saving the client time and money.

    Tip 2: Estimate as you go

    When you are doing a Spec Review, make sure you estimate tasks as you go. That way you can flag problematic tasks as soon as possible. Once you estimate a task, get another developer to check it incase you missed anything.

    Tip 3: Flesh out requirements

    If an estimate looks high, the requirements might not be fully fleshed out. Double check if the requirements are clear enough, otherwise follow-up with the client to discuss further.

  12. The most important part of being in a Spec Review is adding value to the conversations. How do you make sure that your presence is adding value to the discussion and the product?

    Here are some tips:

    Tip 1: Know when to speak up

    Knowing when to speak up can be one of the most difficult tasks for some people. Often when talking in a group, finding a spot to add in a point is difficult. When you have a question or something of value to add, try using the phrase, "Before we move on..." or use the Teams hand 👋 feature. If you don't speak up, important information might be missed or features may be left unrefined.

    Tip 2: Think of other developers

    Sometimes you might not have all the expertise to architect a clients' solution. If you need outside expertise, remember that you can ask quick questions to other developers. You can let the client know that this isn't your specialty and that you'll ask a specialist and get back to them shortly.

    Tip 3: Don't be afraid to double check a requirement

    Make sure you understand what they want, do not be afraid to repeat and double check a requirement. It's better for you to understand it now then to go back and ask them again later.

    Tip 4: Recommend ready-made solutions

    Talk about already made solutions. There are often already made solutions to these very same problems. Be sure to Google beforehand. Present the Pros and Cons of these against a bespoke solution. Sometimes the ready-made solution solves their problem without any further work required.

    Tip 5: Refine the problematic requirements

    Talk about the most problematic parts. These are the areas that should have the most details, so make sure everybody is fully aligned about what is required.

    Tip 6: Ask great questions

    Questions are the key in understanding client needs. Asking a few simple questions goes a long way. Some examples are:

    • What is the goal of the product?
    • Where will the product be hosted?
    • Are there any performance requirements?
    • Are there any any reliability requirements? E.g. Has to respond in x seconds
    • What are the security requirements?
  13. Do you know the value of Event Storming?

    Often when building systems it isn't super clear what all the nuts and bolts should be. There might be several major stakeholders each with slightly different ideas.

    Event Storming is a way to centralize, collate and document everyone's viewpoints.

    To start Event Storming get all stakeholders into a room or meeting and hand out sticky notes + pens. Then get everyone to write out all events they think are necessary and put the sticky notes on the wall. Alternatively, an online platform like Miro could be used to do it virtually.

    Scenario - Invoicing

    Bob, Linda and Sam all sit down together to figure out their new invoicing system. Some of the events they might all come up with include:

    • Create Invoice
    • Delete Invoice
    • Update Invoice
    • Pay Invoice
    • Print Invoice
    • and many more...
We open source. Powered by GitHub