Small projects are rather rare in our company, and the strategies and testing tactics used for long-term projects are not very suitable for them. In this article, we show you how to design a testing strategy for short-term projects and put it into practice.
How it all begins
Sometimes, we are approached by customers who need to quickly develop a small software product. For example, they may need a new website or smartphone app. There is not a lot of work required for such a project, and everything should be ready in a few months.
The customer will usually have an idea of their future users and basis for the project. The old site can act as such a basis. For instance, suppose that a site has served faithfully for a long time, but it is now outdated. It has no responsible layout, which makes it inconvenient for mobile users. According to the customer’s statistics, most of the user traffic comes from mobile devices. The desktop version of the site is used mostly by administrators who fill it with content.
Therefore, the customer in this case may want to transfer mobile users to a native application that contains the basic user experience. The functions required only by content managers would be available on the desktop version of the site.
As another example, consider a working website that suits the customer’s needs. In this case, the customer may want to develop a mobile application for the site, in which several small new features would need to be added.
This is what the development team gets at the start of the project. On short-term projects, the team is small and consists of a project manager, two or three programmers, a UX / UI designer, and a tester.
Where does testing begin?
Testing on a short-term project starts with ‘shift left’, or the principle of shifting testing closer to the start from its traditional place at the end of the production chain. That chain consists of the requirements definition, analysis, design, coding, testing and deployment.
The ‘shift left’ principle means that the team starts the testing not at the end of the development process, but from its early stages, by testing the concept, requirements and documentation.
Image source: Devopedia.org. Shift left.
Below, you will find a set of questions that will help the tester at the start of a short-term project.
Questionnaire: Concept testing
1. Target audience and terms of use of the software
| What we find out | Example | Where we get the information from |
|---|---|---|
| What kind of software product are we working with? Short description of the product? What are its main functions? |
English grammar exercises app; 5 exercises per day. | Interview the customer. |
| Who are we developing the product for? Who is the target audience of the application? |
For schools, and specifically students in grades 5–9. | |
| In what conditions will the future software be used? Device types? Network connection type? |
From mobile devices with stable Wi-Fi in the classroom. |
2. Material base of testing: what we are testing on
| What we find out | Example | Where we get the information from |
|---|---|---|
| Platforms | iOS Android Windows |
Determined from the customer’s answers to previous questions + statistics on the target audience |
| Operating systems (specific) | iOS versions iOS 13 iOS 12 Android OS versions Pie 10 Oreo |
|
| Devices on which the software should run (list of devices) | iPhone .. Samsung Galaxy .. Huawei .. |
3. Integration with what is already there
| What we find out | Example | Where we get the information from |
|---|---|---|
| Systems with which the application should integrate | Account on the desktop version of the site | Customer team, technicians, system documentation and system testing |
| Methods and tools to verify that nothing has been lost | API |
After interviewing the customer and studying the basis of the project, we would get a detailed idea of the concept of the software that needs to be developed. We would then record this in the task tracker and the project wiki.
We would then move on to writing and testing requirements.
Requirements testing
After the concept of the project is formed, we begin testing the requirements. Where do these requirements come from? Here, they come from the customer and are recorded by the project manager; then, the team is immediately put to work.
Requirements testing operates in parallel with programmers writing code and a UX designer developing a prototype. The task of the tester at this stage is to create the most complete and consistent description of the product and its work. In other words, the tester must describe the expected behaviour.
By describing the expected behaviour, the tester lays the basis for the checklists, thus revealing contradictions and empty spaces. Then, they find answers by asking programmers and the customer questions. This way, the team can catch some bugs even before the software gets checked, and sometimes even before the programmer writes the code.
The tester takes the requirements as they are. Let us suppose these are in the user stories format.
1. User story
A user story consists of a title that represents the main use case and additional scenarios with more specific points and details.
| Scheme: As a <role> I want <feature> so that <benefit / goal> As a <role> I want <functionality> to <benefit / achieve the goal> |
| Example: As a user, I want to have access to exercises from 9 am to 9 pm. |
Additional scenarios can be described in the acceptance criteria. We will show how they can be specified later.
Acceptance Criteria
| Acceptance Criterion 1 |
| As a user, I can view the exercises of the day from 9 am to 9 pm. |
| Acceptance Criterion 2 |
| As a user, I can create answers to the exercises of the day from 9 am to 9 pm. |
| Acceptance Criterion 3 |
| As a user, I can edit the answers to the exercises of the day from 9 am to 9 pm. |
| Acceptance Criterion 4 |
| As a user, I can submit responses to the exercises of the day for review from 9 am to 9 pm. |
2. Glossary of the project
Since user stories are written in natural human language rather than technical language, it is important to ensure the following:
a) The concepts mentioned in the story are understood in the same way by all participants in the development process. For example, by ‘exercise’, everyone should mean the same thing;
b) If synonyms are used in the description of one phenomenon—for example, ‘exercises’, ‘homework’ and ‘assignments’—then we are talking about the same phenomenon.
Here, we will be helped by compiling a glossary of the project. Using content analysis, we can look at the texts of user stories and determine which synonyms are used to refer to one phenomenon. We can then write these out in the glossary plate. After that, we can check our understanding of words against those of the customer and the programmer who has already taken the task.
Example glossary
| Concept and all words with which we denote this concept | Definition of concept | Example |
|---|---|---|
| Exercises, set of exercises, assignment, homework | A set of exercises that the student must do in a day (from ... to ...) | 5 grammar exercises by 23:59 in a given time zone |
3. Defaults and additions
After we determine the definitions of concepts, it is time to move on to restoring defaults and abbreviations. In ordinary communication, people rarely use such heavy sentences, but in testing, this approach comes in handy.
| Original text with defaults | Defaults restored |
|---|---|
| ‘Change in the admin panel’ | ‘Change in the admin panel-tests-course-semester-test-folder with settings-xml’ |
| ‘Attribute A’ | ‘Attribute A of class B’ |
After restoring the defaults, we ask questions for more information:
| Concept and all words that we call a concept | Definition of concept | Example | Questions |
|---|---|---|---|
| Exercises, set of exercises, assignment, homework | Set of activities for the student to complete in the day | Daily grammar exercises to be completed today | By what time should the student submit the assignments? How many exercises are in the daily set? |
4. Acceptance criteria: basis for the checklist
By testing requirements in this way, we benefit in two ways:
a) We concretise the acceptance criteria.
b) We lay the foundation for the checklist.
Next, we move on to creating the basis for the checklist. At this stage, we throw out phrases like ‘the system should ...’ and ‘also expected ...’. Then, we ‘squeeze out’ the text from ‘water’ to acquire a list of point-by-point requirements.
| Original text | Text without ‘water’ |
|---|---|
| The system should enable students to do the exercises every day from 9 am to 9 pm | - Exercises are available from 9 am to 9 pm - Exercises are not available from 9:01 pm to 8:59 am |
This way, we can handle both positive and negative requirements in the same order as they appear in the user story. Then, we sort it as follows: negative requirements are placed at the end of the acceptance criteria list, and positive ones are placed at the beginning.
After the acceptance criteria have been fleshed out, we move on to the test design.
Test design and prototype testing
The user story header becomes the header for the review group. The acceptance criteria become the heading of a separate checklist or mind map where we write out and rank the checks.
What is the result?
| Original user story | Checklist |
|---|---|
| User story As a <role> I want <feature> so that <benefit> As a user, I want to be able to access the exercises from 9 am to 9 pm. User roles can do <> to get ... |
User role can do <> to get ... The user has access to exercises from 9 am to 9 pm. |
| Acceptance Criterion 1 Given <initial context>, when <event occurs> Then <ensure some outcomes> |
Checklist ... 1. Assignments can be viewed from 9 am to 9 pm. 2. Assigned OIA can be performed from 9 am to 9 pm. 3. Answers to assignments can be sent for review from 9 am to 9 pm. 4. Answers to assignments can be edited from 9 am to 9 pm. 5. Tasks are not available for viewing and execution from 9 pm to 9 am. |
Creation of test cases on short-term projects is usually impractical. The specificity of test cases is such that they lead to a detailed description of the steps, the expected system response and the final result. This is useful when the product is large and complex, development takes a long time and new people regularly join the team. Accordingly, test cases will be useful when connecting newcomers to the development, and for old employees when testing complex functionality with non-obvious steps.
In short-term projects, the product is small; the team is, too; and everything changes very quickly. It would take too much time to write and maintain test cases. They would constantly have to be rewritten, and time is so short. However, checklists and mind maps, which are used for testing, are the best choices of documentation for such a limited time frame.
Mind map example. Source: Author of the article
To create the most effective tests, we use test design techniques when developing them. For example, we could apply partitioning into equivalence classes, boundary value analysis and decision tables.
In parallel with how the tester writes checklists, the UX / UI designer creates a prototype. The tester checks the prototype interface, corrects checklists and discusses what is found with the team.
This is followed by a demo of the prototype for the customer. Based on the results of the demo, the team collects feedback and records the information received from tasks in the tracker and on the project’s wiki pages. If necessary, the prototype is sent for revision, after which re-testing, demoing and further development of the software product are carried out.
Specifics of testing in a short-term project
In short-term development, there is no ‘sprint planning—sprint—demo—retro’ cycle that we are all used to, but there are deadlines. However, there are user stories describing the functionality of different priorities, which are then implemented in this order. Testing runs in parallel with writing code.
To start testing a product, we need to deploy and configure a test environment, build and install a build and prepare the necessary test data. Only after all these steps have been passed will the tester be able to start testing.
We start with positive cases. We want to make sure that if the user does the right thing, they will get the expected result.
An example of a positive case
<User can do jobs from 9:00 am to 8:59 pm>
Then, we check the negative cases ‘that the user should never get out of it’.
Example of a negative case
<Jobs are not available for viewing and execution from 9:00 pm to 8:59 am>
Before release, the tester checks both the functionality described in individual user stories and the entire product. Specifically, the tester checks that the installation, update and removal of the product are proceeding correctly. The tester then gives feedback to programmers, fixes bugs in the tracker and refines the checklists.
Tracker bugs along with checklists help to maintain the product. The customer knows what defects are in the product and can then choose which ones to fix first.
Life after release
At this point, our product becomes available to end users. The first reviews and bugs come from them. This is normal, since exhaustive testing at the initial stage is impossible, and there will still be some errors.
The specificity of custom development is such that technical support is provided by the customer, who then passes the bugs to the team. As a rule, bugs from users do not have detailed descriptions and require extensive investigation.
The task of the tester at this stage of development is to reproduce the bugs that came from the users, involving programmers if necessary, and to test the fixes created by programmers.
By rechecking a fix, the tester adds checks to the checklists and conducts a detailed test of the functionality around the bug. Sometimes this is how the tester finds related bugs that the team can then fix before users can find them.
The updated checklists, along with other information that the tester entered into the tracker, will save time for the team that comes to support the project in the future and will allow them to quickly understand the project.
Leaving a memo to yourself is also a good idea. Small, short-term projects often become the basis of larger ones. After some time spent using the software, an order may come to write additional functionality.
Conclusion
Testing in a small project starts simultaneously with development and begins with determining the target audience and conditions for using the software. Based on the data received, the team chooses the devices, platforms and operating systems to test the product and considers the integration with existing systems.
This is followed by the requirements testing stage, during which the tester creates the most complete and consistent description of the product and its work. Here, the tester works with the user story and acceptance criteria, from which the tester draws up checklists and mind maps with a list of specific checks.
The next stages will be testing a prototype from a UX / UI designer, individual functionality from programmers and finished versions. Using a task tracker at all stages of a project is important not only for a long-term project, but also for a short-term one, as it helps to support the product, which is especially valuable when transferring a project to a customer.