How We Calculate the Accessibility Score
Unfortunately, it is impossible for us to run a full accessibility audit on every plugin we feature on this site. In order to understand the accessibility scores we are providing, it is important to know what we do (and don’t do) to calculate our scores.
What we do
There are 3 parts to calculating an accessibility score:
- Creating the content to be tested
- Running the tests
- Calculating the score
Let’s look at each of the steps individually.
1. Creating the content to be tested
For each test, we create a page with plugin-specific content. The page does not include a header, a footer, or any other content. This guarantees that any accessibility issues found on the page are a result of the plugin and not some other factor.
Our goal is to create generic content that can be replicated with every plugin in a particular category. This helps keep our scores from favoring plugins with smaller feature sets.
2. Running the tests
We use the Accessibility Checker plugin from Equalize Digital to check the generic content we created in step #1.
The Accessibility Checker runs a number of tests on the page contents. For each test, it assigns one of the following statuses:
- Pass: The test was passed without issue.
- Warning: A warning is “something that may be a problem or may not be a problem, and it’s going to require someone to manually look at it.”
- Error: An error is “something that we definitely know is a problem that needs to be resolved.”
The quotes explaining the error and warning statuses was taken from the “How We Use Accessibility Checker” video.
3. Calculating the score
We start with a score of 10.
We subtract 0.1* for every warning generated. Since a warning only indicates the possibility of an issue rather than a confirmed issue, the affect on the score is minimal.
We subtract 1.5* for each error generated.
* Please note: these amounts are subject to change as we test more plugins and learn more about accessibility
What we don’t do
Manual accessibility audits are beyond the scope of what we’re doing with this websites, so we do not manually assess any results. This means that when the Accessibility Checker plugin issues a warning, we do not attempt to confirm whether or not an actual error triggered the warning.