two software testers discussing automation during a meeting - QA

Automated accessibility testing

Having a product ready for the market is essential in software development, and automated quality gates are the way to go!

Our colleague, Alexandru Dumbraveanu, explains the way accessibility tests can be automated, which, in the end, may lead to an integration with a CI/CD pipeline.

Why should you test for accessibility?

The European Commission mentions that web accessibility allows everyone, including people with disabilities, to perceive, understand, navigate and interact with the Internet. The World Health Organization reports that about 15% of the world's population lives with some form of disability, of whom 2-4% experience significant difficulties in functioning.

Advantages of automated accessibility testing

Automated accessibility testing can be easily integrated with other UI automated tests and also provides clear feedback on the issues detected.

Limitations of automated accessibility testing

You cannot test everything accessibility-related. At best, only 40% of every WCAG criteria can be automated. Nevertheless, some can be automated partially. For example, it is possible to check that a picture has an <alt> tag but it's not possible for a script to evaluate if the tag has an appropriate semantic meaning.

WCAG (Web Content Accessibility Guideline) is a standard that we use to measure compliance when talking about accessibility properties. There are different levels of compliance that can be reached, those are A, AA, and AAA (where A is the lowest level of compliance and AAA is the highest). The example in question only assessed the first level, A.

Technologies used:

C# (programming language)
Selenium (web crawling, headless mode)
Selenium.Axe for .NET (accessibility scanning engine & reporting)
MSTest (test runner)
FluentAssertions (assert)

Example of log when violations are found (not every detail is included):

{
Description = "Ensures buttons have discernible text",
Help = "Buttons must have discernible text",
HelpUrl = "https://dequeuniversity.com/rules/axe/3.4/button-name?application=axeAPI",
Impact = "critical",
Html = "<button class='button has-icon icon-button-fourth-dark' disabled=''>",
Tags = {"cat.name-role-value", "wcag2a", "wcag412", "section508", "section508.22.a"}
}

Method assessing accessibility level A

public AxeResult RunWcag2aTestsOnCurrentPage()
{
    return new AxeBuilder(Browser.Driver)
        .Include("[class='flex-container']")
        .Exclude("[class='demo-tile']")
        .WithTags("wcag2a")
        .DisableRules("duplicate-id", "label")
        .WithOutputFile("AxeLog.txt")
        .Analyze();
}

Test Method

[TestMethod]
public void When_UserAccessesDashboardPage_Then_AccessibilityCriteriaMustBeMet()
{
    Browser.GoToEnvironmentUrl(DashboardPath);
    var axeResults = AxeMethods.RunWcag2aTestsOnCurrentPage();
    Browser.Driver.CreateAxeHtmlReport("AxeReport.html");
    axeResults.Violations.Should().BeEmpty();
}

Steps in creating the test:

Navigate to the page which needs to be tested using Selenium.
On the page that needs to be tested, the accessibility assessment is made using the criteria/parameters defined in the RunWcag2aTestsOnCurrentPage method.
Two reports are made: one within the RunWcag2aTestsOnCurrentPage method, which returns a text file, and another one after the assessment is done, in the test method, which returns an HTML file.
An assert is done at the end, looking for any violations of the accessibility criteria. If any are found, the test fails.

Author: Alexandru Dumbraveanu, Software Tester

Reference:

https://github.com/TroyWalshProf/SeleniumAxeDotnet

What we are proud of

Find out what our colleagues think

Project

Innovating HR & Payroll solutions

Project

Providing a complete shopping experience