An indepth dive into using Behat/Mink/Selenium for BDD testing.
* http://behat.org
* http://mink.behat.org/
* http://docs.seleniumhq.org/
In this talk I'll cover:
* why and when to use Behat (and when not)
* Installation and configuration of Behat and Mink
* Building Behat Contexts
* Avoiding data deadlocks and "test user account" syndrome
* Introduction to Selenium and testing JavaScript
* Best practises for writing tests (what to avoid, what to aspire for, writing stories like you mean it, how to get your product owners to write them)
* Common gotchas
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
I put on my mink and wizard behat (tutorial)
1. I put on my
mink and
wizard behat
Questing in the world of
front end testing
2. 9:30 Setup
9:45 Introduction to Front End Testing
10:15 We write some tests
10:45 Coffee break
11:00 I talk about some more advanced stuff
11:30 We write some more tests
12:15 We attempt a grid
12:30 Q&A
12:45 End
Schedule
100% chance of incorrectness
3. $ sudo vim /etc/hosts
192.168.56.101 opencfp.dev
# copy T9AG1x and T9AG1x_*
$ cd T9AG1x
$ vagrant up
$ vagrant ssh
$ cd /var/www/opencfp
$ sh run.sh
Setup for Practical
Vagrant
$ sudo vim /etc/hosts
10.41.6.62 opencfp.dev
# copy just T9AG1x/opencfp
$ cd opencfp
$ php vendor/behat/behat/bin/behat
No Vagrant
17. $ sudo vim /etc/hosts
192.168.56.101 opencfp.dev
# copy T9AG1x and T9AG1x_*
$ cd T9AG1x
$ vagrant up
$ vagrant ssh
$ cd /var/www/opencfp
$ sh run.sh
Setup for Practical
Vagrant
$ sudo vim /etc/hosts
10.41.6.62 opencfp.dev
# copy just T9AG1x/opencfp
$ cd opencfp
$ php vendor/behat/behat/bin/behat
No Vagrant
19. Feature: Party harmony
As a leader, I want to ensure harmony and mutual trust, so that
we work as a team
Scenario: Teach members to respect others’ property
Given that the Wizard has 10 cookies
And the Bard eats 1 cookie
Then the Bard mysteriously catches fire
Cucumber Syntax
Readable testing language
20. class FeatureContext … {
/**
* @Given that the wizard has :num cookies
*/
public function wizardHasCookies($num) {
// $this->wizard is a pre-existing condition... like syphilis
$this->wizard->setNumberOfCookies($num);
}
}
and converts it into
FeatureContext.php
21. Feature: Party harmony
As a leader, I want to ensure harmony and mutual trust, so that
we work as a team
Scenario: Teach members to respect others’ property
Given that the Wizard has 10 cookies
And the Bard eats 1 cookie
Then the Bard mysteriously catches fire
Cucumber Syntax
What’s missing?
22. Scenario:
Given that the wizard has 10 cookies
And the Bard eats 1 cookie
Fire spell fizzled (OutOfManaException)
1 scenario (1 failed)
2 steps (1 passed, 1 failed)
0m0.03s (14.19Mb)
Remember {P} C {Q}
Set your starting states
23. Feature: Party harmony
As a leader, I want to ensure harmony and mutual trust, so that
we work as a team
Background:
The Wizard’s fire spell is fully charged
And the Bard is currently not on fire
Scenario: Teach members to respect others’ property
Given that the Wizard has 10 cookies
And the Bard eats 1 cookie
Then the Bard mysteriously catches fire
Remember {P} C {Q}
Set your starting states
24. ???
As a leader, I want to ensure harmony and
mutual trust, so that we work as a team
26. Front end testing is code coverage for your user stories
User stories
Coverage
Features are your contract with the stakeholders
Contract
Scenarios are the use cases that outline the user story
Scenarios
27. Legend has it...
… that someone once convinced their PO to
write all their front end tests.
28. class MinkContext … {
/**
* Clicks link with specified id|title|alt|text.
*
* @When /^(?:|I )follow "(?P<link>(?:[^"]|")*)"$/
*/
public function clickLink($link) {
$link = $this->fixStepArgument($link);
$this->getSession()->getPage()->clickLink($link);
}
}
Mink provides...
MinkContext.php
30. $ composer require behat/behat="~3.0,>=3.0.5"
Getting started
$ composer require behat/mink-extension="~2.0"
Behat (cucumber syntax)
Mink (browser emulator)
Web drivers
$ composer require behat/mink-goutte-driver="~1.0"
$ composer require behat/mink-selenium2-driver="~1.2"
PR
AC
TIC
AL
31. $ ./vendor/bin/behat --init
+d features - place your *.feature files here
+d features/bootstrap - place your context classes here
+f features/bootstrap/FeatureContext.php - place your definitions,
transformations and hooks here
Initialize
Create a new test suite
PR
AC
TIC
AL
33. $ ./vendor/bin/behat -dl
Given /^(?:|I )am on "(?P<page>[^"]+)"$/
When /^(?:|I )reload the page$/
When /^(?:|I )move backward one page$/
When /^(?:|I )move forward one page$/
When /^(?:|I )press "(?P<button>(?:[^"]|")*)"$/
When /^(?:|I )follow "(?P<link>(?:[^"]|")*)"$/
When /^(?:|I )fill in "(?P<field>(?:[^"]|")*)" with "(?P<value>(?:
[^"]|")*)"$/
Context
What does Mink bring to the table?
PR
AC
TIC
AL
35. Feature: Authentication and authorisation
As a security conscious developer I wish to ensure that only
valid users can access our website.
Scenario: Attempt to login with invalid details
Given I am on "/login"
When I fill in "email" with "some@guy.com"
And I fill in "password" with "invalid"
And I press "Login"
Then I should see "Invalid Email or Password"
Our first feature
auth.feature
PR
AC
TIC
AL
36. $ ./vendor/bin/behat --config behat.yml features/auth.feature
Scenario: Attempt to login with an invalid account
Given I am on "/login"
When I fill in "email" with "some@guy.com"
And I fill in "password" with "invalid"
And I press "Login"
Then I should see "Invalid Email or Password"
1 scenarios (1 passed)
5 steps (5 passed)
Victory
output
PR
AC
TIC
AL
37. Feature: Authentication and authorisation
As a security conscious developer I wish to ensure that only
valid users can access our website.
Scenario: Attempt to login with invalid details
Given I login as "some@guy.com" with password "invalid"
Then I should see "Invalid Email or Password"
Simplify
auth.feature
PR
AC
TIC
AL
38. Scenario: Attempt to login with an invalid account
Given I login as "bob@smith.com" with password "invalid"
Then I should see "Invalid Email or Password"
1 scenario (1 undefined)
/**
* @Given I login as :arg1 with password :arg2
*/
public function iLoginAsWithPassword($arg1, $arg2) {
throw new PendingException();
}
Simplify
output
PR
AC
TIC
AL
39. class FeatureContext … {
/**
* @Given I login as :username with password :password
*/
public function iLoginAsWithPassword($username, $password) {
$this->visit("/login");
$this->fillField("email", $username);
$this->fillField("password", $password);
$this->pressButton("Login");
}
}
Simplify
FeatureContext.php
PR
AC
TIC
AL
40. Scenario: Attempt to login with an invalid account
Given I login as "bob@smith.com" with password "invalid"
Then I should see "Invalid Email or Password"
1 scenarios (1 passed)
2 steps (2 passed)
Simplify
output
PR
AC
TIC
AL
41. // Manipulate the current web session
$session = $this->getSession();
$session->visit($url);
$session->setBasicAuth($user, $password = '');
$session->setRequestHeader($name, $value);
$session->setCookie($name, $value = null);
$session->getCookie($name);
$session->getCurrentUrl();
$session->reload();
$session->back();
Session
BehatMinkSession
SID
E
N
O
TE
42. Page
// Navigate and manipulate the current page in a selector style
$page = $this->getSession()->getPage();
$page->find($selectorType, $selector); // 'css', '.class-name'
$page->findById($id);
$page->hasLink($locator);
$page->clickLink($locator);
$page->fillField($locator, $value);
$page->hasSelect($locator);
$page->selectFieldOption($locator, $value, $multiple = false);
$page->hasTable($locator);
BehatMinkElementDocumentElement
SID
E
N
O
TE
43. Driver
// Access the web driver directly via xpaths
$driver = $this->getSession()->getDriver();
$xpath = '//html/body/table/thead/tr/th[first()]'
$driver->blur($xpath);
$driver->focus($xpath);
$driver->mouseOver($xpath);
$driver->isVisible($xpath);
$driver->dragTo($sourceXpath, $destinationXpath);
// Modifier could be 'ctrl', 'alt', 'shift' or 'meta'
$driver->keyPress($xpath, $char, $modifier = null);
BehatMinkDriverDriverInterface
SID
E
N
O
TE
44. Feature: Authentication and authorisation
As a security conscious developer I wish to ensure that only
valid users can access our website.
Scenario: Attempt to register a new user
Given I am on "/signup"
When I fill in "email" with "some@guy.com"
And I fill in "password" with "valid"
And I fill in "password2" with "valid"
And I fill in "first_name" with "some"
And I fill in "last_name" with "guy"
And I press "Create my speaker profile"
Then I should see "You’ve successfully created your account"
Our first hurdle
This ones easy, you do…. oh….
PR
AC
TIC
AL
46. $ composer require robmorgan/phinx="~0.4"
Phinx to the rescue
Install
$ php vendor/bin/phinx init
Phinx by Rob Morgan - https://phinx.org. version 0.4.3
Created ./phinx.xml
Configuration
$ php vendor/bin/phinx create InitialMigration
Creating
SID
E
N
O
TE
47. #!/usr/bin/env bash
DATABASE="opencfp"
mysql -e "DROP DATABASE IF EXISTS $DATABASE" -uroot -p123
mysql -e "CREATE DATABASE $DATABASE" -uroot -p123
vendor/bin/phinx migrate
vendor/bin/behat
It’s a bit extreme
run-behat-test.sh
PR
AC
TIC
AL
48. SAVEPOINT identifier;
# Run tests
ROLLBACK TO SAVEPOINT identifier;
RELEASE SAVEPOINT identifier;
Transaction/Rollback
Roll your own solution
50. # Stop the currently running service
sudo service postfix stop
# Dumps outgoing emails to file as "day.hour.minute.second"
smtp-sink -d "%d.%H.%M.%S" localhost:2500 1000 &
vendor/bin/behat
smtp-sink
run-behat-test.sh
52. class DatabaseContext {
public function __construct($dsn, $user, $pass) {
$this->dbh = new PDO($dsn, $user, $pass);
}
/**
* @When /^there is no user called :user$/
*/
public function removeUser($user) {
$this->dbh->prepare("DELETE FROM `users` WHERE username=?")
->query([$user]);
}
}
A new context
DatabaseContext.php
SID
E
N
O
TE
57. // Make sure your server and your behat client have the same time set
// Share the secret key between the two. The code should be valid for
// 30 second periods
$code = sha1($secret_key . floor(time() / 30));
if ($request->get("code") === $code) {
// Bypass captcha
}
Easier way
Simple but safe bypass
58. Our first talk
Set the stage
Feature: Manage paper submissions
In order to ensure that speakers can submit their papers
As an speaker I need to be able to manage my own submissions
Background:
There is a user called "some@guy.com" with password "secrets"
I login as "some@guy.com" with password "secrets"
Scenario: Add a new talk to our submissions
...
PR
AC
TIC
AL
59. Our first talk
Talk submission in 3, 2, 1...
Scenario: Add a new talk to our submissions
Given I am on "talk/create"
And I fill in the following:
| title | Behat Talk |
| description | Awesome |
| type | regular |
| category | testing |
| level | mid |
And I check "desired"
And I press "Submit my talk!"
Then I should see "Success: Successfully added talk."
PR
AC
TIC
AL
61. Well that won’t work
talks.feature
Feature: Manage paper submissions
In order to ensure that speakers can submit their papers
As an speaker I need to be able to manage my own submissions
Scenario: Delete a talk
Given create a talk called "Behat Talk"
And I am on "/dashboard"
When I follow "Delete"
And I should not see "Behat Talk Changed"
The text "Behat Talk Changed" appears in the text of this page,
but it should not. (BehatMinkExceptionResponseTextException)
62. // Guzzle using web scraper
behat/mink-goutte-driver
// Java-based distributed browser workers (support JavaScript)
behat/mink-selenium2-driver
behat/mink-sahi-driver
// node.js headless browser proxy (support JavaScript)
behat/mink-zombie-driver
Drivers
Some take the scenic route
64. $ java -jar selenium-server-standalone-2.*.jar
Selenium
@javascript # Or we could use @selenium2
Feature: Manage paper submissions
In order to ensure that speakers can submit their papers
As an speaker I need to be able to manage my own submissions
Start Selenium Server
Specify javascript requirement
PR
AC
TIC
AL
65. $ ./vendor/bin/behat --tags speaker,talk
Tags
Run specific tags
@speaker
Feature: Manage paper submissions
In order to ensure that speakers can submit their papers
As an speaker I need to be able to manage my own submissions
@talk
Scenario: Create a new talk
Given I am logged in as a speaker ...
SID
E
N
O
TE
66. Feature: Submitting and managing talks
As a speaker I wish be able to submit talks so I can get a chance
to talk at a conference.
@javascript
Scenario: Delete a talk
Given create a talk called "Behat Talk"
And I am on "/dashboard"
When I fill "Delete"
And I accept alerts
And I should not see "Behat Talk"
Enable JavaScript
talks.feature
PR
AC
TIC
AL
67. Run as JavaScript
talks.feature
Feature: Submitting and managing talks
As a speaker I wish be able to submit talks so I can get a chance
to talk at a conference.
Scenario: Delete a talk
Given create a talk called "Behat Talk"
And I am on "/dashboard"
When I follow "Delete"
And I accept alerts
And I should not see "Behat Talk"
LIVE
D
EM
O
68. Scenario: Edit a talk
Given I am on "/dashboard"
And I remember "tr[id^='talk']" content as "Title"
When I follow "Edit"
And I fill in "title" with "New Title"
And I press "Update my talk!"
Then I should see "New Title"
And I should not see "memory:Title"
Transformations
Sometimes we need to remember
PR
AC
TIC
AL
69. CSS Selectors
Drives are just like browser, no one ever
supports everything properly...
SID
E
N
O
TE
70. /**
* @Transform /^memory:(.*)$/
*/
public function fromMemory($key) {
if (!isset($this->memory[$key])) {
throw new LogicException("Entry $key does not exist");
}
return $this->memory[$key];
}
Transformations
FeatureContext.php
PR
AC
TIC
AL
71. /**
* @Given /^I remember "(.*)" content as "(.*)"$/
*/
public function rememberContentOf($selector, $key) {
$e = $this->getSession()->getPage()->find("css", $selector);
if (!is_object($e)) {
throw new LogicException("Element $selector not found");
}
$value = $e->getValue() ? $e->getValue() : $e->getText();
$this->memory[$key] = $value;
}
Transformations
FeatureContext.php
PR
AC
TIC
AL
72. class FeatureContext … {
public function takeAScreenshotCalled($filename) {
$driver = get_class($this->getSession()->getDriver());
if ($driver == 'BehatMinkDriverSelenium2Driver') {
$ss = $this->getSession()->getScreenshot();
file_put_contents($filename, $ss);
}
}
}
Screenshot
FeatureContext.php
PR
AC
TIC
AL
75. class FeatureContext … {
/**
* @AfterScenarioScope
*/
public function afterScenario(AfterScenarioScope $scope) {
$scenario = $scope->getScenario()->getTitle();
$filename = make_safe_filename($scenario);
// Take a screenshot and put it on a dashboard somewhere
$this->takeAScreenshotCalled($filename);
}
}
Hooks
FeatureContext.php
PR
AC
TIC
AL
76. class FeatureContext … {
/**
* @AfterStep
*/
public function afterStep(AfterStepScope $scope) {
$code = $event->getTestResult()->getResultCode();
if ($code == TestResult::FAILED) {
// Take a screenshot
}
}
}
Hooks
FeatureContext.php
PR
AC
TIC
AL
77. class FeatureContext … {
/**
* @Given /^(?:I )wait for AJAX to finish$/
*/
public function iWaitForAjaxToFinish() {
$this->getSession()->wait(5000, "(0 === jQuery.active)");
}
}
AJAX
The waiting game
PR
AC
TIC
AL
78. class FeatureContext … {
/**
* @Given /^(?:I )press the letter :l$/
*/
public function iPressTheLetter($l) {
$s = "jQuery.event.trigger({type:'keypress', which:'$l'});";
$this->getSession()->evaluateScript($s);
}
}
Raw javascript
There might be a valid use case...
PR
AC
TIC
AL
81. $ ./vendor/bin/behat --suite admin
Suites
Run a specific suite
Feature: Managing the CFP
In order to ensure that speakers can submit their papers
As an admin
I need to be able to open the call for papers
82. $ ./vendor/bin/behat --suite speaker
Suites
Run a specific suite
@speaker
Feature: Submitting to the CFP
In order to ensure that the conference has papers
As an speaker
I need to be able to submit papers
83. $ java -jar selenium-server-standalone-2.*.jar -role hub
Selenium Grid
$ java -jar selenium-server-standalone-2.*.jar -role node -hub http:
//10.41.6.62:4444/grid/register
Start the grid
Add a node
LIVE
D
EM
O
86. Feature: Show relevant promotions to non-paying activated customers
Scenario: Show promotion pricing to referred clients from the EU
Given I create a user with:
| email-activated |
| is-EU-member |
| is-referred-client |
| NOT has-made-purchase |
Then ...
Setup
What attributes does our user have?
87. interface AttributeInterface {
// Get the required attributes to have this attribute
public function getDependencies();
// Does this user have this attribute?
public function has(MinkContext $context);
// Allocate this attribute to the user
public function allocate(MinkContext $context);
// Attempt to remove this attribute from the user
public function remove(MinkContext $context);
}
Attributes
AttributeInterface.php
88. class IsEUMember extends AttributeInterface {
public function getDependencies() { return ["email-activated"]; }
public function has(MinkContext $context) {
$context->visit("/profile");
$field = $context->getSession()
->getPage()
->findField("country");
return in_array($field->getValue(), $this->eu_countries);
}
}
Attributes
IsEUMember.php
89. // ...
public function allocate(MinkContext $context) {
$context->visit("/profile");
$context->selectOption("country", "Netherlands");
$context->pressButton("Update");
}
public function remove(MinkContext $context) {
$context->visit("/profile");
$context->selectOption("country", "UK"); // Future-proofing
$context->pressButton("Update");
}
Attributes
IsEUMember.php
90. // This class must handle dependency conflicts by examining the
// dependencies of each with/without attribute
class Request {
protected $attributes = [];
public function with(AttributeInterface $feature);
public function without(AttributeInterface $feature);
// List of attributes (including dependents) required
public function getWith();
// List of attributes (including dependents) that must be removed
public function getWithout();
}
Request
Request.php
91. class User {
public function __construct(MinkContext $context, $request) {
$this->createNewUser($context);
foreach ($request->getWith() as $attr) {
$attr->allocate($content);
}
foreach ($request->getWithout() as $attr) {
$attr->remove($content);
}
}
public function canSupport($request);
}
User
User.php
92. use BehatGherkinNodeTableNode;
class FeatureContext … {
/**
* @Given /^(?:I )create a user with:$/
*/
public function iCreateAUser($type, TableNode $table) {
$attributes = $table->getRowsHash();
$request = new Request();
// Build using $request->with(...) & $request->without(...);
$user = new User($context, $request);
}
}
TableNode
Handling a table
100. Feature: Administration of talk submissions
In order to be able to manage a conference, as an admin, I should
be able to manage talks
Background:
Given there is a speaker registered as "admin@guy.com" with a
password "secrets"
And User "admin@guy.com" has admin rights
And I login as "admin@guy.com" with password "secrets"
Scenario: Approve a submitted paper
Scenario: Decline a submitted paper
Final Task
Apply what you know
102. @javascript
Feature: Admin
In order to be able to manage a conference, as an admin, I should be able to manage talks
Background:
Given there is a speaker registered as "admin@opencfp.org" with a password "secrets"
And User "admin@opencfp.org" has admin rights
And I login as "admin@opencfp.org" with password "secrets"
Scenario: Approve a submitted paper
Given I create a talk called "New Talk"
And I am on "/admin/talks"
And I click on element ".js-talk-select"
Then I should see an ".check-select--selected" element
Scenario: Reject a submitted paper
Given I am on "/admin/talks"
And I click on element ".check-select--selected"
Then I should not see an ".check-select--selected" element
Model? Answer
103. class FeatureContext … {
/**
* @Given User :email has admin rights
*/
public function userHasAdminRights($email)
{
exec("php -f bin/opencfp admin:promote " . escapeshellarg($email));
}
/**
* @Given I click on element :selector
*/
public function iClickOnElement($selector) {
$element = $this->getSession()->getPage()->find("css", $selector);
if (!is_object($element)) {
throw new LogicException("Element $selector not found");
}
$this->getSession()->evaluateScript('$("' . $selector . '").click();');
}
}
Model? Answer
104. class FeatureContext … {
/**
* @Given I create a talk called :title
*/
public function iCreateATalkCalled($title) {
$this->visit("/dashboard");
$this->clickLink("Submit a talk");
$this->fillField("title", $title);
$this->fillField("description", "Awesome");
$this->fillField("type", "regular");
$this->fillField("category", "testing");
$this->fillField("level", "mid");
$this->checkOption("desired");
$this->pressButton("Submit my talk!");
}
}
Model? Answer
105. Thank you
Photo from Flickr by John Morey, TrojanRat, Gerry Machen, USFS Region 5,
Peregrina Tyss and Thomas Hawk