Accelerate your CI/CD Pipeline with BDD and Acceptance Testing

Acceptance testing is essential for verifying that software meets user requirements by ensuring it functions as expected. It validates the system’s behavior against predefined criteria, providing confidence in the final product. However, traditional acceptance testing methods often struggle with communication gaps between technical teams and non-technical stakeholders, leading to costly rework and project delays.

A common scenario highlights this issue: a business analyst (BA) gathers requirements from a client and then communicates them to the development team. Despite best efforts, different interpretations can cause discrepancies between what the client envisions and what the developer delivers. These misunderstandings in requirements are a major challenge for successful software delivery.

By improving collaboration through methods like Behavior-Driven Development (BDD), teams can bridge the gap between stakeholders, minimize misunderstandings, and ensure alignment from the start.

Behavior-Driven Development (BDD) is a software development methodology that bridges the gap between business requirements and technical implementation. It encourages collaboration among developers, testers, and business stakeholders. They define together how the system should behave in a shared language.

In this article, we’ll explore how to combine BDD with acceptance testing. You’ll see how BDD transforms acceptance tests into living documentation. We’ll also discuss how to automate these tests using Semaphore CI. By the end of this article, you’ll understand why BDD is the best approach for creating effective and maintainable acceptance tests.

What is Acceptance Testing?

Acceptance testing ensures a system meets user requirements and behaves as expected in real-world scenarios. It confirms the software delivers the functionality stakeholders agreed upon. It is a crucial step and can either be the final phase or come just before end-to-end testing.

Traditionally, acceptance testing was performed manually. Business analysts or QA teams would write test cases based on requirements and execute them step-by-step to confirm expected behavior. While manual testing works for small projects, it becomes time-consuming, error-prone, and unsustainable for modern software development.

Today acceptance testing relies on automation to overcome these challenges. By automating acceptance tests, teams can validate behavior continuously throughout the development lifecycle. This not only reduces manual effort but also ensures that changes do not break existing functionality.

Here’s an example of an acceptance test:

A requirement states, “The system must allow a user to reset their password.” A simple acceptance test would verify this behavior:

1. The user navigates to the “Forgot Password” page.
2. The user submits their email.
3. The system sends a reset link to the user’s email.
4. The user resets the password successfully.

The goal of acceptance testing is to confirm that the system satisfies these steps from the user’s perspective.

However, manually defining and maintaining such test scenarios often leads to inconsistencies and misunderstandings. This is where BDD comes into play. By structuring acceptance tests as executable specifications, BDD eliminates ambiguity and improves team collaboration.

In the next section, we’ll explore how BDD works and why it’s a good idea to combine it with acceptance testing.

How BDD Works

Now, let’s dive deeper into how BDD works and why it’s transformative for acceptance testing.

Structure of BDD

The core idea of BDD is to write scenarios that define how the system should behave. These scenarios use natural, human-readable language that both technical and non-technical team members can understand. A fundamental aspect of BDD is the Gherkin syntax, which structures acceptance criteria into a readable format. It uses the Given-When-Then pattern. The tests express the user’s point of view.

• Given: Describes the preconditions or context.
• When: Specifies the action or event.
• Then: States the expected outcome.

For example, let’s revisit the password reset requirement:

Feature: Password Reset
  Scenario: Successful password reset request
    Given the user is on the "Forgot Password" page
    When the user submits their valid email address "user@example.com"
    Then the system sends a password reset link to "user@example.com"

You can also enhance the clarity and expressiveness of scenarios by using additional keywords like And, But, and Examples.

• And: This keyword allows you to chain multiple actions or expectations in a natural way. For example, “Given the user is logged in, and the user has admin privileges.”
• But: This is a way to introduce an exception or a contrasting condition. For example, “When the user clicks the button, but the internet connection is lost.”
• Examples: The Examples keyword is useful for data-driven testing. It allows you to run the same scenario with multiple data. For example, “Given I am on the login page when I enter the following username and password: Examples Username | Password | Expected Result.”

This is how BDD transforms business requirements into executable scenarios that serve as documentation and tests.

The Benefits of BDD

BDD brings several advantages to software development:

• Shared Understanding: Scenarios are written in plain language, enabling collaboration between stakeholders, developers, and testers.
• Executable Specifications: Scenarios are not just text — they can be automated and executed to validate behavior.
• Reduced misunderstandings: By involving stakeholders in defining behaviors, BDD minimizes miscommunication and ensures clarity in requirements.
• Lower costs: Preventing misunderstandings and errors early in the development cycle saves time and resources.
• Safer releases: Scenarios act as living documentation, ensuring new changes don’t disrupt existing functionality.

When to Use BDD?

BDD scenarios are not necessarily mandatory for every product specification. They are most useful for complex specifications where there is a risk of misunderstanding or when more thorough testing is needed. For simple tasks, like color or text changes, using BDD scenarios would be inefficient and unnecessary.

BDD and User Stories

In practice, teams can effectively integrate BDD with user stories in Jira. For each user story, they create a corresponding BDD scenario, often in the Given-When-Then format, which directly translates the acceptance criteria into automated tests.

Jira User Story:

As a user, I want to be able to reset my password using my email address so that I can regain access to my account if I forget my password.

Acceptance Criteria (in the user story):

The user should be able to request a password reset by entering their registered email. The user should receive a password reset email with a link to reset their password. The link should expire after 30 minutes.

Corresponding BDD Scenario:

Feature: Password Reset

Scenario: User requests a password reset
Given the user is on the login page
When the user enters their registered email and clicks "Reset Password"
Then the user should receive a password reset email
And the email should contain a link to reset the password
And the reset link should expire in 30 minutes

BDD Challenges

While BDD offers significant benefits, teams often face challenges when implementing it effectively.

1. Maintaining Tests

• Challenge: Keeping tests updated as the system evolves.
• Why it Happens: Poorly written tests may become outdated or too complex.
• Solution:
• Balance detail and simplicity.
• Regularly review and refactor tests.
• Focus on aligning tests with evolving requirements.

1. Team Collaboration

• Challenge: BDD relies heavily on collaboration between developers, testers, and stakeholders.
• Why it Matters : Without strong teamwork, tests may not accurately reflect the system’s intended behavior.
• Solution:
• Foster open communication in Agile environments.
• Build cross-functional teams with shared goals and understanding.

1. Adopting the BDD Mindset

• Challenge: Shifting to BDD can feel daunting, especially for teams used to traditional testing methods.
• Why it Happens: BDD has a steep learning curve, leading to resistance.
• Solution:
• Emphasize benefits like improved collaboration and alignment with business goals.
• Share success stories and celebrate small wins.

1. BDD Anti-Patterns

• Feature-Coupled Step Definitions: These occur when step definitions are tightly coupled to specific scenarios or features, making them hard to reuse. For example:

Given the user is logged in
When the user clicks on "Add to Cart"
Then the item is added to their shopping cart

If this step relates to a single feature like “Add to Cart,” it might not be reusable for similar actions elsewhere, such as “Add to Wishlist.”

• Solution:
• Use generic, reusable steps:

When the user performs an action on "Add to Cart"
Then the system updates the corresponding data

• Conjunction Steps: These steps combine multiple actions into one, making tests harder to understand and debug. For instance:

When the user fills out the form and clicks submit

It’s hard to read and unclear whether the issue lies with filling out the form or with clicking submit.

Solution: Break into smaller, clear steps:

When the user fills out the form
And the user clicks submit

1. Trivial Functionality Tests: These are scenarios that test functionality already covered by unit tests, adding unnecessary complexity.

• Why it Happens: Some teams might feel pressure to achieve high test coverage and include trivial cases in scenarios. Others may treat BDD as a replacement for all types of testing.
• Solution:
• Keep scenarios focused on what the system does, not how it works internally.
• Use BDD to describe high-level behaviors or system-level interactions that matter to stakeholders.

Popular BDD Tools

Several tools support BDD by allowing teams to write and execute tests based on behavioral specifications:

• Cucumber: Supports various programming languages and offers strong Gherkin support, making it highly popular. It can even sync with Jira to automatically link the scenarios to the relevant user stories.
• Behave: A Python-based BDD framework ideal for teams working with Python projects.
• Reqnroll: A popular BDD framework for .NET developers.

The Links Between BDD and Acceptance Testing

Executable Specifications

BDD turns acceptance tests into executable specifications, aligning technical implementation with business goals. Using Gherkin, teams write testable scenarios that serve as a single source of truth for all stakeholders. Tip: Structure scenarios with the Given-When-Then format for clarity and consistency.

Behavior as Acceptance Criteria

BDD emphasizes translating vague requirements into clear, testable scenarios. For example:

Traditional RequirementsBDD Acceptance Criteria“Allow password reset”Scenario: Password Reset Request

Collaboration Across Teams

BDD fosters a shared understanding between developers, testers, and stakeholders. Its simple Given/When/Then syntax allows everyone to contribute to defining behaviors.

Linking BDD to acceptance testing enhances clarity, collaboration, and confidence in meeting customer expectations.

Automating BDD Acceptance Tests with CI/CD

Integrating BDD tests into your CI/CD pipeline ensures continuous validation of features. Automated tests catch regressions early, meeting user expectations before deployment.

Here’s a simplified CI/CD pipeline flow when using BDD-based acceptance tests:

With tools like Semaphore CI, you can configure this workflow to run on every commit, branch, or merge request.

Automating Acceptance Tests with Semaphore CI

In this example, we’ll write a BDD test for a Python project using the Behave framework. We’ll automate the BDD tests using Semaphore CI’s pipeline.

First, create a virtual environment to safely install Python packages:

python3 -m venv myenv
source myenv/bin/activate

Install Behave:

pip3 install behave

Create a file named password_reset.feature in a features directory. This file contains the BDD scenarios using Gherkin syntax.

Feature: Password Reset

  In order to regain access to my account
  As a registered user
  I want to be able to reset my password via email
  Scenario: User submits a valid email for password reset
    Given the user is on the "Forgot Password" page
    When the user submits a valid email "user@example.com"
    Then the system sends a password reset link to "user@example.com"
  Scenario: User submits an invalid email for password reset
    Given the user is on the "Forgot Password" page
    When the user submits an invalid email "invalid_email"
    Then the system shows an error message "Please enter a valid email address"

Create a file named password_reset_system.py to implement the system functionality:

class PasswordResetSystem:
    def __init__(self):
        self.valid_emails = []  # Store valid emails for testing purposes
    
    def add_valid_email(self, email):
        """Method to add valid email addresses."""
        if self.is_valid_email(email):
            self.valid_emails.append(email)
        else:
            raise ValueError(f"Invalid email format: {email}")

    def reset_password(self, email):
        """Simulate the password reset request."""
        if self.is_valid_email(email):
            return f"Password reset link sent to {email}"
        else:
            return "Please enter a valid email address"
    def is_valid_email(self, email):
        """Helper method to check if email is in valid format."""
        return "@" in email and "." in email  # Very basic email validation (see https://pypi.org/project/email-validator/)

Create a main.py file where you initialize the PasswordResetSystem class and interact with it.

from password_reset_system import PasswordResetSystem

def main():
    # Create an instance of the PasswordResetSystem
    system = PasswordResetSystem()
    # Example of adding a valid email
    try:
        system.add_valid_email("user@example.com")
        print("Valid email added!")
    except ValueError as e:
        print(e)
    # Simulate a password reset request with a valid email
    response = system.reset_password("user@example.com")
    print(response)  # Output: Password reset link sent to user@example.com
    # Simulate a password reset request with an invalid email
    response = system.reset_password("invalid_email")
    print(response)  # Output: Please enter a valid email address
if __name__ == "__main__":
    main()

When you run main.py, you will see output similar to this:

$ python3 main.py 

Valid email added!
Password reset link sent to user@example.com
Please enter a valid email address

Now, create a steps.py file in a steps directory to define the steps in your scenarios. Behave will look for step definitions that match the steps in the feature file.

from behave import given, when, then
from password_reset_system import PasswordResetSystem

# Initialize the system instance
system = PasswordResetSystem()
@given('the user is on the "Forgot Password" page')
def step_given_user_on_forgot_password_page(context):
    context.page = "Forgot Password"
    context.system = system  # Assign the system object to context for use in other steps
@when('the user submits a valid email "{email}"')
def step_when_user_submits_valid_email(context, email):
    context.response = context.system.reset_password(email)
@when('the user submits an invalid email "{email}"')
def step_when_user_submits_invalid_email(context, email):
    context.response = context.system.reset_password(email)
@then('the system sends a password reset link to "{email}"')
def step_then_system_sends_reset_link(context, email):
    assert context.response == f"Password reset link sent to {email}", f"Expected reset link for {email}, got {context.response}"
@then('the system shows an error message "{message}"')
def step_then_system_shows_error_message(context, message):
    assert context.response == message, f"Expected error message '{message}', got {context.response}"

Key points:

• This file defines the Python functions that implement the actions from the Gherkin steps. For example, when the user submits a valid or invalid email, the PasswordResetSystem class simulates the system’s behavior and returns the appropriate response.
• We use assert to perform assertions. For example, we assert that the response from the reset_password method matches the expected output.

To run the tests, execute this command:

$ python -m behave

You should see output similar to this:

$ python -m behave
Feature: Password Reset # features/password_reset.feature:1
  In order to regain access to my account
  As a registered user
  I want to be able to reset my password via email
  Scenario: User submits a valid email for password reset             # features/password_reset.feature:7
    Given the user is on the "Forgot Password" page                   # steps/steps.py:8 0.000s
    When the user submits a valid email "user@example.com"            # steps/steps.py:14 0.000s
    Then the system sends a password reset link to "user@example.com" # steps/steps.py:22 0.000s

  Scenario: User submits an invalid email for password reset                    # features/password_reset.feature:12
    Given the user is on the "Forgot Password" page                             # steps/steps.py:8 0.000s
    When the user submits an invalid email "invalid_email"                      # steps/steps.py:18 0.000s
    Then the system shows an error message "Please enter a valid email address" # steps/steps.py:26 0.000s
1 feature passed, 0 failed, 0 skipped
2 scenarios passed, 0 failed, 0 skipped
6 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.000s

Generate a requirements.txt file, which includes all the Python dependencies you’ve installed in your environment:

$ pip freeze > requirements.txt

Run the deactivate command to deactivate the virtual environment when you’re done.

Let’s move on to the CI part.

Semaphore expects a .semaphore folder at the root of your project, as well as a semaphore.yml file containing the CI configuration for the project:

version: v1.0
name: Python
agent:
  machine:
    type: e2-standard-2
    os_image: ubuntu2204

blocks:
  - name: "Install Dependencies"
    task:
      prologue:
        commands:
          - sudo apt-get update && sudo apt-get install -y python3-dev
      jobs:
        - name: pip
          commands:
            - checkout
            - cache restore
            - pip download --cache-dir .pip_cache -r requirements.txt
            - cache store
  - name: "Run Tests"
    task:
      prologue:
        commands:
            - checkout
            - cache restore
            - pip install -r requirements.txt --cache-dir .pip_cache
      jobs:
        - name: Run BDD Tests
          commands:
            - python -m behave

Key points:

• The e2-standard-2 machine type has 2 CPUs and 4GB of RAM, suitable for standard workloads.
• The commands in the prologue execute before the block’s jobs run.
• The “Install dependencies” block updates the system and installs Python dev tools. It also caches Python dependencies to speed up future runs.
• The “Run Tests” block restores dependencies from the cache and installs them. Then it runs the BDD tests using python -m behave.

To execute the tests in the pipeline, you need a Semaphore account. Visit the signup page and choose either GitHub or Bitbucket for registration. In this example, I’ll use GitHub.

Next, create a new project by clicking the “+ Create new” button.

Add a GitHub repository. For example, I’ll use the demo project I created for this tutorial.

Semaphore will automatically initialize the project. As you already have the semaphore.yml file in place, you can use it directly in the pipeline workflow. Each push to the project triggers the pipeline.

You can expect results like this:

The “Run BDD Tests” job displays the test results. You should see the same output as you did earlier in your Terminal:

You can find the complete source code for this demo in my GitHub repository.

Conclusion

In this article, we examined BDD’s functionality, benefits, challenges, and its integration with acceptance testing to enhance collaboration and develop effective testing strategies. Utilizing BDD with acceptance testing boosts team confidence by ensuring features address user needs.

You also learned how to implement a BDD test and automate it within the Semaphore CI pipeline.

Start small — identify critical business scenarios, write them in the Given-When-Then format, and automate them as acceptance tests. Once you integrate them into your CI/CD pipeline, these tests will ensure quick feedback and more reliable software delivery.

Originally published at https://semaphoreci.com on February 11, 2025.