Skip to main content

Behat

https://github.com/Behat/Behat

https://docs.behat.org/en/latest/user_guide.html

A php framework for auto-testing your business expectations.

Vortex comes with pre-configured Behat profiles for Drupal projects.

Usage​

Since Behat requires a running Drupal instance, it can only be run within container environment.

docker compose exec -T cli php -d memory_limit=-1 vendor/bin/behat

or

ahoy test-bdd

Running all tests in the file

docker compose exec -T cli vendor/bin/behat tests/behat/features/my.feature

or

ahoy test-bdd tests/behat/features/my.feature

Running tagged tests with @group_name tag

docker compose exec -T cli vendor/bin/behat --tags=group_name

or

ahoy test-bdd --tags=group_name

Configuration​

See configuration reference.

All global configuration takes place in the behat.yml file.

By default, Behat will run all tests defined in tests/behat/features directory.

Adding or removing test targets:

default:
  suites:
    default:
      paths:
        - '%paths.base%/web/modules/custom/mymodule/tests/behat/features'

Contexts and extensions​

The configuration uses the following contexts and extensions:

  • Drupal Extension: The Drupal Extension is an integration layer between Behat, Mink Extension, and Drupal. It provides step definitions for common testing scenarios specific to Drupal sites.
  • Behat Screenshot Extension: Behat extension and step definitions to create HTML and image screenshots on demand or when tests fail.
  • Behat Progress Fail Output Extension: Behat output formatter to show progress as TAP and fails inline.
  • Behat Steps: Collection of Behat steps for Drupal

Reports and screenshots​

Behat is configured to generate test run reports in JUnit format stored in .logs/test_results/behat.

Screenshots for failed tests or purposely made screenshots are stored in .logs/screenshots directory.

Both test results and screenshots are stored as artifacts in CI.

Profiles​

Behat runs with default profile defined in the configuration file: this runs all tests not tagged with @skipped tag.

There are also p1 and p2 profiles defined that run tests not tagged with @skipped and tagged with @p1 and @p2 tags respectively. These profiles are used in CI to run tests in parallel if parallel runners number is greater than 1. Behat will run tests using @pX tags on tests, where X is the runner index starting from 0.

In the example below, if there is one runner, Behat will run all scenarios, but if there are 2 runners - Behat will run only scenarios with @p0 tag on the first runner and scenarios with @p1 tag on the second runner.

@homepage @smoke
Feature: Homepage

  Ensure that homepage is displayed as expected.

  @api @p0
  Scenario: Anonymous user visits homepage
    Given I go to the homepage
    And I should be in the "<front>" path
    Then I save screenshot

  @api @javascript @p1
  Scenario: Anonymous user visits homepage
    Given I go to the homepage
    And I should be in the "<front>" path
    Then I save screenshot

The profile can be overridden using $VORTEX_CI_BEHAT_PROFILE environment variable set in the CI configuration.

Ignoring fail in CI​

This tool runs in CI by default and fails the build if there are any violations.

Set VORTEX_CI_BEHAT_IGNORE_FAILURE environment variable to 1 to ignore failures. The tool will still run and report violations, if any.

Writing tests​

When writing tests, it is recommended to use the following structure:

@tag_describing_feature_group
Feature: Short feature description

  As a site owner
  I want to make sure that my feature does what it is supposed to do
  So that I can be sure that my site is working as expected

  @api
  Scenario: Anonymous user uses a feature
    Given I go to the homepage
    And I should be in the "<front>" path
    Then I save screenshot

  @api @javascript
  Scenario: Anonymous user uses a feature with AJAX
    Given I go to the homepage
    And I click ".button" element
    Then I save screenshot

Use @api tag for tests that do not require JavaScript and @api @javascript for tests that require JavaScript.

tip

JavaScript tests are slow. Try using as few JavaScript tests as possible to keep the test run time low.

FeatureContext.php and example tests​

The FeatureContext.php file is a Behat custom context file that is loaded by default. This is where custom step definitions and hooks should be placed.

It is recommended to use traits to organize step definitions and hooks and include those traits in the FeatureContext.php file. There are already several traits included from the Behat Steps package (only basic ones are included; there are more available in the package).

There are also example tests in the tests/behat/features directory that can be used as a starting point for writing tests. These tests define @smoke tests for the homepage and login page as well as tests for contributed modules.

Connecting to the browser​

For @javascript tests, Behat uses Selenium to connect to the browser. The browser runs within a container and can be observed from the host machine by using a browser thanks to the VNC server running in the container and noVNC.

Run

ahoy info

and click on the link next to Selenium VNC URL on host to open the browser.

Or use the following command to get the port number:

docker compose port chrome 7900

and open http://localhost:<port>/?autoconnect=1&password=secret in the browser.

tip

When running tests, you can add And I wait for 60 seconds step to suspend the test execution for a minute and interact with the headless browser in a container through a browser on your host machine.

behat-novnc.gif