How to write a software test plan

Learning what’s important and where to start

By Richard C Paterson

Many organisations do test planning, but many don’t realise all the value in test planning. Testers often produce test plans for no better reason than they always have or the process says they should. Done properly, a test plan can be a very useful weapon in your testing arsenal.

To write a test plan, or not to write a test plan: that is the question that gets brought up regularly at the Software Testing Clinic. If the answer to that question is ‘yes’, lots of new questions appear. A few questions you might want to ask yourself or your team are:

• What form should a test plan take?
• What information should a test plan include?
• Who is it for?

These questions are all valid and deserve considered answers. These questions are asked a lot and should continue to be asked, to ensure the test plan still provides sensible answers to various problems which the business needs addressed. Problems like: visibility, accountability, sharing of knowledge, or change management.

Do you need a test plan?

Documents, as a means of communication, exist firstly because the parties to the knowledge are separated in time and/or space, and so cannot efficiently share information synchronously.

If you and those parties are co-located, and don’t require any lasting proof or record of the conversation, then it’s questionable what value a document provides. As the Agile Manifesto puts it, “individuals and interactions” are more valuable than “comprehensive documentation”. That’s not to say that documents don’t have their place, but it’s important to choose carefully when and what to document. It’s a key balance to strike, and one you should continually check to make sure you’re meeting the needs of all parties in an efficient manner.

I’m not going to tell you to write a test plan or not write one. Only you can know whether, in your context, a test plan document is required. All I really want you to do is ask yourself honestly “Do I need to write a test plan?”

If you think you do, here might be a few examples of questions you might want to ask and some good possible answers:

Question: Who is asking for the document?

Answer: “The customer is asking for it; it’s a contractual deliverable”

Question: Who is going to read it?

Answer: “My project manager, who needs to verify that I’m going to test things at least to their satisfaction”

Question: What will they gain from it?

Answer: “Along with other information, it’ll give her sufficient confidence to release”

Question: What could improve if I stop writing one?

Answer: “I’ll have more time to add real value to my project because I haven’t wasted my time writing a document no-one will read”

Question: What could become worse if I stop writing one?

Answer: “Everyone connected with the project will be in the dark about what I’m testing”

Question: Who will notice if I stop writing a test plan?

Answer: “The process owner will, because production of a test plan is part of our audited process”

Using the test plan as early as possible in a release for answers to these questions can be a form of testing. You might ask: “Are there performance criteria I can evaluate and use for testing?” or “Will the product be internet-facing?” or “What disaster avoidance/recovery scenarios do the product need to support?”. By asking these questions, you lead the stakeholders to think about things like performance, security and resilience much earlier in the release than they otherwise might.

It may be that a test plan is a contractual deliverable, in which case working with your customer to clarify what it is they want to know, and what mechanism they want you to use to provide that information will be important.

Verb vs Noun

“Plans are useless, but planning is indispensable” - Dwight D. Eisenhower

While test plans are static in nature, test planning is a dynamic, discursive process of learning and negotiating. A document that is not read has little value. It costs time and effort to produce something which might not be useful to the project or stakeholders. It’s not until you use a test plan that it adds value. A test plan that no-one reads, which doesn’t inform anyone about the testing to be done, is a waste of your valuable time which could be better spent.

Balance writing an extensive test plan with planning activities related to the actual testing during the period which the test plan covers. If writing an extensive test plan is necessary, make sure to cover risks of what might not be covered or tested. Test plans can be used to inform stakeholders what you plan on testing as well as what you might not be able to test if there isn’t enough time.

Who are your stakeholders?

Documents are for communicating information between people. So you need to understand who those people are, in the context of test information. Who are your stakeholders?

If you’re not sure, determine all the questions you want to ask about the testing you want to do, then figure out who the best people are to answer those questions, and ask them. They should be invested in testing the product or application. Explain why you’re asking and the value that you can add by gaining answers to your questions.

Other people who might use a test plan are those who could use the information you provide as a result of doing the planned testing. Find these stakeholders, then ask them what information they might want, and then plan to provide it. Some examples of stakeholders:

Software Testers who want to know what they might be testing over the course of the project. The test plan can provide them with specific information about environments, product versions, or source data to use. The testers can, from their experience, help you to improve the plan, adding missing information and test approaches you hadn’t considered.

Project Managers want to know what you’re planning to test, and how, so they can be confident in making a decision to release.

Customer Support can tell you about the customer environment, how they use the system, and the kind of problems they encounter. This information can inform what testing activities you might plan to cover their concerns.

Product Owners can tell you how the product is meant to be used, and hopefully how customers use it differently. This information could create possible user profiles which can aid in testing for those users.

Sales People can tell you which products are the most popular, and the ways in which customers use them.

If in doubt as to whether someone is a stakeholder or not, include people rather than exclude them. You never know who is going to have a snippet of information that might change the direction your testing might take, or the specifics of how you test it. Over time, people will come to understand the mechanism of gathering information for a test plan and understand how they can contribute.

How to write a good test plan - form, structure and content

Test Plans can take any number of forms. Some examples are:

Word documents are often the default form that test plans take, due to the prevalence of Word and people’s familiarity with it. Test plans can range from a single page document that summarises the main points of the testing to be performed, to lengthy, IEEE829 / IEEE29119 -compliant documents that enumerate every detail.

Mind maps are an excellent way to convey testing information in a structured graphical format. At a glance, users can easily follow the structure and drill down to the level of detail they require. Google “mindmap test plans” for some examples.

SharePoint / Wikis are both excellent alternatives to Word documents, including powerful change management and editing functionality. They enable great flexibility in how information is structured and enable quick updates and multi-user editing.

Web-based planning tools like Jira can be used for more than just planning the development work to be done. Further, integrations with test management tools like TestRail can provide the team with a joined-up picture of all planned and actual testing.

Whiteboards / Kanban boards are another good way of showing graphically the scope of testing. Physical boards are a good and highly visible way of communicating what testing you will do.

A good place to start with a test plan design could be a One Page Test Plan. This can give you an easy way to begin creating a test plan that is lightweight and informational.

In terms of content, test plans are typically created to document the basic answers to the Five Ws and How of testing. The content of your test plans can change for various reasons, either from release to release or sprint to sprint. Update your test plan to encompass what you learn from release to release, and even from product to product.

Leverage your test plans as a way to improve your testing process. Use your test plan as a repository for organisational knowledge. Made a mistake in a release? Figure out how you would catch it next time, and add it to the plan template. Typically forget to plan a certain test type? Add notes about certain types of test coverage. Add notes about customer pain points and issues which could give you more information while testing. The test plan could be a way to create continuous improvements in your test planning and strategy.

Over time, update the template to support and improve the planning that you do. Make the test plan work for you, in all aspects of its form, structure and content. If it’s not working for you, change it.

How to review a test plan

Once your test plan is ready, review it. There are many forms a review can take, but I recommend a face-to-face review meeting. These meetings can lead to some discoveries about what might be necessary to add or cover. It also creates visibility for you, or your team, around what kind of testing you plan on providing for a product, app, or release.

Invite any stakeholders to the review. Consider having a minimum number of attendees, and ensure that the relevant roles are represented. A suggested quorum might consist of 4 people/roles; the test plan author (usually a tester), another tester, the project manager, and a technical or customer support representative. However, for larger releases, more stakeholders could be involved. It all depends on who has a stake in a release.

Walk through every aspect of the test plan. Discuss each part of the test plan and listen to any feedback your stakeholders have or information they might want to share.

Once the plan is approved by the necessary quorum of stakeholders, it shouldn’t remain static. As the testing for a project or application starts, use the test plan as a living document to keep and track the team’s efforts towards accomplishing goals set out in the test plan.

“No plan survives contact with the enemy” - Helmuth von Moltke the Elder

“Only certainty is uncertainty” - Heraclitus

How to update a test plan

Shift happens, so your plan should be open to change. The test planning approach should enable you to deal with changes, to clarify what you will do differently, what new information you need or obtain, and what new information you must provide, and to whom.

Choose a suitable frequency to re-review, or use any event that feels big enough as a trigger to re-review. Re-reviews don’t need to be enormous. Keep them in-line with the size of the changes. They don’t have to be meetings. Ask individuals to review the changes at their desk and email any comments. Make it easy for people to contribute to keep your test plan up-to-date and working for you.

Test plans - working for you

Without any kind of documentation around your testing, questions could come up from time to time which might be hard to answer. If you can show your test plan, which was reviewed, updated, and re-reviewed, along with your test results, you have a much more concrete basis on which to discuss your testing activities and create visibility into what you are doing for the quality of the product.

Having a test plan could help you think about all the preparations you need to make, which is especially important if you don’t control things you might need during a testing cycle. If you need servers, data, or tool access from someone else, planning for all this in advance means you’ve a much better chance of being ready when those things become available. It’s vital that you are ready to go as soon as something testable is available.

Having this information also supports a retrospective review or post-mortems, enabling you to make better decisions and have better discussions about how to improve testing efforts.

A worthwhile test plan

Using a test plan as a mechanism to seek answers, to drive information exchange and consensus, and to prepare yourself, can make it a worthwhile thing. Make it valuable for you and your stakeholders. Make your test plan work for you, not against you. And if you can’t, don’t be afraid to get rid of it.

Author Bio

Richard Paterson is Head of Testing and Application Security at SAS R&D Scotland. He considers himself a designer, leader and maker in addition to being a tester.

Richard has been a tester for nearly twenty years, initially working in Defence and more recently in Law Enforcement / Intelligence and Data Management. Richard has built up a multi-talented test organisation with SAS R&D Scotland over the last ten years. His approach to test leadership is to support and enable his testing function, constantly looking for and championing new ideas and approaches.

A relatively new contributor to the software testing community, Richard helps support the Software Testing Clinic in Glasgow, occasionally blogs and tweets, and will be giving his maiden conference talk at UKSTAR in March 2018.

You can contact Richard on Twitter @rocketbootkid, on LinkedIn and on his blog www.rcpaterson.co.uk/blog.