stereobooster

Posted on Jun 1, 2019

Testing accessibility with Cypress

#a11y #react #javascript #webdev

In the previous post, we created an accessible React accordion component. Let's test it. I don't see much sense of writing unit tests for this kind of components. Snapshot tests don't provide much value either. I believe end-to-end (e2e) tests are the best choice here (but for testing hooks I would prefer unit tests).

Will try to test it with Cypress. Cypress uses headless Chrome, which has a devtools protocol, which supposes to have better integration than previous similar solutions.

Install Cypress

Cypress easy once you understand how to start it. It took me... more than I expected to understand how to get started. They have huge documentation, which is hard to navigate (at least for me).

Sometimes too much documentation is as bad as too little

But I figured it out after some experimentation. Install Cypress

yarn add cypress --dev

Run it the first time

yarn cypress open

It will create a lot of files. Close Cypress window. Delete everything from cypress/integration.

Add cypress.json to the root of the project.

{
  "baseUrl": "http://localhost:3000/"
}

Now in one terminal, you can start dev server yarn start and in second one Cypress yarn cypress open and start to write tests.

Configure Cypress

But how to run tests in CI? For this you need another npm package:

yarn add --dev start-server-and-test

Change package.json

"scripts": {
  "test": "yarn test:e2e && yarn test:unit",
  "test:unit": "react-scripts test",
  "cypress-run": "cypress run",
  "test:e2e": "start-server-and-test start http://localhost:3000 cypress-run"
}

Almost there. Add one more package

yarn add cypress-plugin-tab --dev

In cypress/support/index.js

import "./commands";
import "cypress-plugin-tab";

Add to .gitignore

cypress/screenshots
cypress/videos

Now we done.

Planning tests

This part I like.

Let's create test file cypress/integration/Accordion.js:

describe("Accordion", () => {
  before(() => {
    cy.visit("/");
  });
  // your tests here
});

It will open the root page of the server (we will use dev server) before tests.

We saw WAI-ARIA Authoring Practices 1.1. in the previous post:

Space or Enter
- When focus is on the accordion header of a collapsed section, expands the section.
Tab
- Moves focus to the next focusable element.
- All focusable elements in the accordion are included in the page Tab sequence.

We simply can copy-paste it "as is" in test file:

  describe("Space or Enter", () => {
    xit("When focus is on the accordion header of a collapsed section, expands the section", () => {});
  });

  describe("Tab", () => {
    xit("Moves focus to the next focusable element.", () => {});
    xit("All focusable elements in the accordion are included in the page Tab sequence.", () => {});
  });

describe - adds one more level to the hierarchy (it is optional).
xit - a test which will be skipped, as soon as we will implement actual test we will change it to it
it - a test, it("name of the test", <body of the test>)

Isn't it beautiful? We can directly copy-paste test definitions from WAI-ARIA specification.

Writing tests

Let's write actual tests.

First of all, we need to agree on the assumptions about the tested page:

there is only one accordion component
there are three section in it: "section 1", "section 2", "section 3"
section 2 is expanded other sections are collapsed
there is a link in section 2
there is a button after the accordion

First test: "Space or Enter, When focus is on the accordion header of a collapsed section, expands the section".

Let's find the first panel in accordion and check that it is collapsed. We know from the specification that the panel should have role=region param and if it is collapsed it should have hidden param:

cy.get("body")
  .find("[role=region]")
  .first()
  .should("have.attr", "hidden");

Let's find corresponding header e.g. first. We know from the spec that it should have role=button param. Let's imitate focus event because users will use Tab to reach it.

cy.get("body")
  .find("[role=button]")
  .first()
  .focus();

Now let's type Space in focused element

cy.focused().type(" ");

Let's check that section expanded (opposite of the first action):

cy.get("body")
  .find("[role=region]")
  .first()
  .should("not.have.attr", "hidden");

I guess it is pretty straightforward (if you are familiar with any e2e testing tool, they all have similar APIs).

It was easy to write all tests according to spec plus specs for the mouse.

Flaky tests

The only flaky part is when we use React to switch focus e.g. up arrow, down arrow, end, home. Change of focus, in this case, is not immediate (compared to browsers Tab). So I was forced to add a small delay to fix the problem:

describe("Home", () => {
  it("When focus is on an accordion header, moves focus to the first accordion header.", () => {
    cy.contains("section 2").focus();
    cy.focused().type("{home}");
    cy.wait(100); // we need to wait to make sure React has enough time to switch focus
    cy.focused().contains("section 1");
  });
});

Conclusion

I like how the specification can be directly translated to e2e tests. This is one of the benefits of writing a11y components - all behavior is described and tests are planned. I want to try to write the next component BDD style (tests first).

Top comments (8)

Jodi Winters • Jun 20 '19

We're just writing tests for a really popular accessible page on the Internet. We have found an interesting one, checking for "accessible" content. If we have an element that's hidden by the zero'd dimensions trick so that it's still visible by a screen reader, cypress still think it's hidden, so a :visible or should('be.visible',...) isn't really a sufficient test. I'm wondering if you've determined a good practice for testing something like this?

stereobooster • Jun 20 '19

I thought hidden (html5 attribute) means hidden to everyone. I need to double check, because I'm not sure

Jodi Winters • Jun 20 '19

I went looking myself, found webaim.org/techniques/css/invisibl.... However, I was wrong about which trick we were using, they're 1px blocks with overflow and the content is pushed away into the overflow. I'm not near my cypress at the mo, so I'll check and if there's still an issue, put up some proper detail on it :)
I think some custom should rules could really help with a11y.

bhupendra • Dec 16 '19

There is a great package to test a11y with cypress , you can follow the guide here.

However when I am trying to integrate , I get error : cy.injectAxe() is not a function.

Craig Carlyle • Jun 7 '19

Very nice. We do something similar at my company. I wrote a Cypress command to run the aXe engine against our components as a sanity check as the final step. Here's a gist if you're interested.