Requirements Auditing: Is the Specification Fit For its Purpose? (part II)
Tagging the specification
Use the table of contents of the template (Figure 1) as the guide to running your audit. In essence you are matching the actual contents of the requirements specification that you are auditing with the generic content defined by the template. You are looking for subject matter that is missing, duplicated, ambiguous, superfluous or irrelevant. You can start with any one of the sections in the template and use it as the driver for finding matches in the specification for that type of information. I am going to tell you the order that we have learnt (by doing many specification audits) that normally proves to be most logical and productive.
Start with section 1 – the purpose of the project – and remind yourself what you are looking for by reading the guidance in section 1 of the template. Choose one of your coloured post-it notes for tagging section 1 information – suppose you choose red. Now look through the specification that you are auditing and put a red tag on all pages of the specification that refer to the purpose of the project. Incidentally I have suggested using coloured post-it notes because it is helps to distinguish between different types of information. If you haven’t got coloured notes then use standard yellow ones and write the relevant section number on each one. In addition to tagging the appropriate pages with a red tag it also helps to highlight the information with a red coloured pencil. This is because you will often have information about several sections mixed together on the same page and this approach will make it easier for you when you are analysing the results of your audit.
Now turn your attention to sections 2 and 3, choose another coloured tag, and review your specification from the point of view of how well it defines the stakeholders. I have audited specifications that make no mention of any type of stakeholder (or just mention the stakeholders by name of role without defining their input to requirements) and have learnt that these are indicators that there will be assumptions and ambiguities in the specification. Once again the template, along with the other references at the end of this paper, contains a great deal of guidance about the type of stakeholder information that should be in the specification.
The next section we review is section 7 – the scope of the work, because this specifies the boundaries of the work that the project is investigating. The width of the boundary varies depending on the subject matter of the project. It might be the work done by part of a business, or it might be all the work that will be done by a new business, or it might be the boundaries of the part of the world that will be affected by a new consumer product. As explained in the template, the scope of the work should be precisely defined by adjacent systems and defined inputs and outputs. Also, the scope should be partitioned according to business events (or some other agreed and traceable partitioning theme) and each event should be traceable to one or more business use cases.
Next we review and tag the specification for: section 4 mandated constraints, section 6 relevant facts and assumptions, section 8 the scope of the product.
Throughout the audit you should be looking for terms that are vague, contradictory or ambiguous. This corresponds to section 5, naming conventions and definitions. If the specification contains a dictionary/lexicon/glossary or some section that defines the terminology, then you can use this as a starting point. In parallel with all of your auditing activities, whenever you come across a term then ask the question – is this term unambiguously defined and is it being consistently used? A convenient way to keep track of this is to write each term on an index card – or you can use a spreadsheet if you prefer, and note on the card whereabouts in the specification the term is defined. If you come across more than one definition of the same term then note the location on the card. To keep track, I use a specific coloured pencil – green – to mark in the specification the terms that I have noted on cards.
Now you have built some familiarity with the organisation and content of the specification and you are ready to look for and tag the atomic functional and non-functional requirements; sections 9-17 in the template cover these. Also, the requirements shell that you will find in the template’s introductory material identifies all the attributes of an atomic requirement.
Use the requirements shell to assess the completeness of each atomic requirement. If any of the attributes is missing, then make a note on the specification and tag the requirement with one of your coloured notes.
Looking for connections
Each atomic requirement should possess attributes that define its details. But a problem in many of the specifications that I audit is that there is no formal way of assessing how these requirements work together in groups, or how they affect each other.
To provide functional traceability each of the atomic requirements in the specification there should have a connection to one or more business use cases and, if the project has progressed to that stage, one or more product use cases. Depending on how the project has been partitioned, the requirements in your specification might be grouped using a different theme like features or components. The partitioning theme should support the way that you are working and should reflect the way that the requirements need to be grouped in order to communicate “chunks” of requirements to your relevant stakeholder groups. On many of the projects that I work on the requirements are tagged to use cases as a way of discovering and maintaining functional groups. Then they are also tagged to features, components or whatever other grouping the project needs to satisfy its communication needs.
Analysing the results
After comparing sections 1-17 of the template with the specification, you now you have a specification that is bristling with coloured tags and marked with coloured pencils. To prepare your audit report of the specification you need to identify:
- what is missing?
- what is redundant?
- what is inconsistent?
Missing requirements knowledge
If any of the sections (or subsections) of the template could not be matched to the some part or parts of the specification then you potentially have some missing requirements knowledge. I say potentially because you need to consider the intended detail (as discussed earlier) of the specification. If, for example, the intention is that the specification is a first cut summary of the requirements then maybe the exclusion of atomic requirements (or some of their attributes) is intentional.
If the specification is intended as input to design and implementation, then it should contain a detailed definition of the atomic functional, non-functional and constraint requirements. And each definition should include all the attributes of an atomic requirement. It should also be possible to trace each atomic requirement to the business use cases and product use cases to which it relates.
In the first pass of your audit you used coloured tags and coloured pencils to identify which parts of the specification correspond to which parts of the requirements knowledge described in the template. If there are any parts of the specification that are not tagged, then those parts are potentially redundant. When you spot these untagged parts of the specification then you might discover that they are relevant but you missed them when you were doing your matching. However there might be some things in the specification that are not relevant to the requirements and are only going to confuse the people who read it.
Another reason for a section being untagged is that it might duplicate something else in the specification. If this is the case then mark it as a duplicate.
Review the parts of the specification dealing with stakeholders (sections 2 &3) the goals (section 1) and the scope (sections 7 and 8) and look for inconsistencies:
- Does the work (investigation) scope concur with the stated goals?
- Are the stakeholders identified, not just by role but by name and reason for involvement?
- Are the goals measurable or open to interpretation?
- If the specification contains a product scope, then does that scope conform to the goals?
- Are all the interfaces on the context diagram defined in the definitions of terms? (Ref section 5).
- Are all the terms used in individual requirements defined in the specification? Earlier I suggested using either index cards or a spreadsheet as a way of keeping track of which terms occur where and whether or not they are defined.
Communicating the results
Once you have done your audit, then it is best to talk to the people who produced the specification and to ask questions about your findings. They are in a position to shed light on misunderstandings and to identify how they will address any problems that you have highlighted.
Then you can summarise the findings of your audit according to the sections on the requirements template.
Section Number:Section Name:
- Missing requirements information
- Redundant requirements information
- Inconsistent requirements information
- Other comments and questions
- Business risk of non-repair
When to do an audit
The best time to do a requirements audit is whenever you have a handover point in your project. This might be because you are handing the specification over to another group in your organisation or to an external supplier. If you are giving the specification to people in another location, regardless of whether they are inside or outside your organisation, then it is vital to spend time doing the audit. You will be surprised at how many questions arise and how many assumptions you will uncover.