Generate Agile Acceptance Criteria with ChatGPT to Supercharge Story Creation

ChatGPT is taking the world by storm, but can it help in software delivery? In this blog, we explain why you need agile acceptance criteria and how a large language model like ChatGPT can help streamline the story-building process.

Software developers, scrum masters and product owners often look for ways to streamline the agile story creation process. One particular area of frustration is the time-consuming and challenging task of creating acceptance criteria for these stories. Large language model (LLM) artificial intelligence (AI) solutions, like ChatGPT, can help. But why are LLMs well-suited to assist with generating agile acceptance criteria?

To answer that question, we first need to understand what agile acceptance criteria is and how it plays into development methodologies.

Why Do You Need Acceptance Criteria in Agile?

Agile development methodologies have become a mainstay of development in recent years, as they offer a flexible and collaborative approach to software development. One key component of agile development is the creation of user stories: short, simple descriptions of a feature or functionality a user wants. User stories guide the development process and ensure the end product meets the user’s needs. However, with user stories, you also must provide acceptance criteria.

Acceptance criteria are a set of specific conditions or requirements that must be met for a user story to be considered complete. They are essential to the agile development process, ensuring the final product meets the user’s needs and expectations.

Agile teams usually create acceptance criteria during the planning stage of an agile development project based on the user’s needs and goals. The acceptance criteria define the boundaries of the user story and ensure the team understands expectations.

Typically, teams write acceptance criteria in a clear, concise format designed to validate the completeness of a story. Criteria often include specific requirements for functionality, performance, security, usability and other aspects of the product.

For example, if the user story is “As a user, I want to be able to search for products on the website,” acceptance criteria might include the following:

• The search functionality must return results in under three seconds
• The search results must display in a clear and easy-to-understand format
• Users with disabilities must be able to access the search functionality
• The search results must sort by relevance to the user’s query.

Note that acceptance criteria are not the same as acceptance tests. Acceptance tests derive from acceptance criteria.

Creating acceptance criteria for user stories can be a difficult and time-consuming task for software development teams because they often require a deep understanding of the user’s needs and goals. This is where an LLM, like ChatGPT, can help speed the process and improve overall story quality.

Using ChatGPT to Create Agile Acceptance Criteria

ChatGPT is an artificial intelligence language model developed by OpenAI that uses advanced natural language processing (NLP) techniques to generate human-like responses to text prompts. Data scientists trained it on a massive dataset of text, allowing ChatGPT to learn the nuances and patterns of human language and generate natural-sounding responses.

Among its many applications, ChatGPT can generate acceptance criteria based on the user story and any relevant context or constraints the user provides. Here are some ways that ChatGPT can supercharge your agile story creation:

1. Speed up the acceptance criteria creation process: With ChatGPT, you can generate acceptance criteria in a matter of seconds, giving you time to focus on other aspects of your software development process
2. Improve the quality of your acceptance criteria: ChatGPT’s language model was trained on a massive human language dataset, making its responses natural sounding. The tool’s results can help you create more effective acceptance criteria that better reflect your intended outcomes
3. Enhance collaboration among team members: Multiple team members can simultaneously use ChatGPT as an acceptance criteria generation tool, allowing everyone to contribute to the criteria creation process and ensuring everyone is on the same page.

While ChatGPT as an acceptance criteria generation tool is a powerful instrument for agile story creation, it’s important to remember that it is only one tool in your toolkit. You’ll still need to work closely with your team members, stakeholders and end-users to ensure your acceptance criteria align with your overall goals and objectives.

Using ChatGPT as an Agile Acceptance Criteria Tool

Now that we know how to use an LLM like ChatGPT for agile story creation, let’s see how it can work in practice. We have taken a minimum viable product (MVP) approach at Centric. We created an API and integrated it into Jira stories via an Azure Static Web page. Our API includes Azure’s text-to-speech AI as well. Many prefer hearing text to reading it, so we offer both options.

Our MVP approach does not currently “copy” the result directly into the story. We’re still working on fine tuning some output before we copy over, so it’s a manual process designed to keep the human in the loop. We don’t mean for our approach to replace team discussions but rather to accelerate the development of baseline acceptance criteria for agile stories and give teams a starting point for discussions. Here’s a high-level example of an agile story:

“As a banking customer, I want to access my savings account to view and make transactions.” We pasted our user story into an integrated ChatGPT window and asked it to generate acceptance criteria, as seen in the image below:

Based on our query, ChatGPT generated these acceptance criteria:

1. As a banking customer, I should be able to access my savings account information after providing valid credentials
2. The savings account information displayed should include the account balance, transaction history and details of any pending transactions
3. The transaction history should be presented in a clear and understandable format, including the date, amount, transaction type and the other party involved (if applicable)
4. The interface must be user-friendly, providing easy navigation to see account details
5. The user must be able to make transactions such as deposits, withdrawals, transfers and payments from the savings account interface
6. If there is a need to provide additional verification for transactions, the system should prompt the user appropriately
7. Any errors or technical issues must be handled and resolved efficiently, ensuring the user is properly alerted in case of problems while accessing the savings account
8. All transactions must be secure, providing proper encryption and security protocols to ensure user data is protected
9. The savings account interface should provide clear instructions and guidance to users on the transactions they can perform and the associated fees, if any
10. The system should be able to handle a large volume of transactions concurrently without any delay or issues.

Based on our goals, some of these may be too general, such as number five. Some others are out of scope, such as number nine. We can ask ChatGPT to break down number five into additional acceptance criteria:

• The savings account interface should allow the user to make deposits of any amount
• The savings account interface should allow the user to make withdrawals of any amount
• The savings account interface should allow the user to make transfers to other accounts
• The savings account interface should allow the user to make payments to third-party vendors or services
• The savings account interface should accurately reflect all transactions the user makes in the user’s account balance.

In a matter of seconds, we have generated a great starting point for discussing user stories. While the final acceptance criteria will need modification based on stakeholder feedback, we have drastically sped up the acceptance criteria creation process. For instance, the bank in our example may place regulatory limits on certain types of savings account withdrawals and require that we omit or adjust our acceptance criteria to reflect their policy.

We must remember while this tool offers great creative potential, it is not a replacement for the human users at the heart of user story creation. In the above example, we used generic banking stories and information. We liked the results, so naturally, we were curious about how ChatGPT would handle situations involving some of our clients’ user stories (with their approval) using their domain language, acronyms, and more. Based on our testing, we have successfully accelerated acceptance criteria development.

Conclusion

You can easily customize ChatGPT as an acceptance criteria generation tool to suit your development team or projects specific needs. For example, teams can input specific keywords or constraints to generate more targeted and relevant acceptance criteria, such as specific system names, industry terms and more.

However, remember that ChatGPT’s output will need some fine-tuning and will certainly need human input. As you can see from some of the stories, the generated acceptance criteria extend beyond the story’s scope, but that can also be food for thought.

ChatGPT’s ability to generate acceptance criteria quickly and accurately makes it an ideal tool for agile teams looking to streamline their agile story creation process and achieve better outcomes. Not bad, as ChatGPT charged us $0.006 for this example. That’s right, less than one cent.