Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Color Contrast rules for WCAG2.0 And WCGA2.1 giving incorrect result. #4579

Open
praveshpr opened this issue Sep 14, 2024 · 5 comments
Open
Labels

Comments

@praveshpr
Copy link

I am using axe-core library in node js with playwright.

Details:-

Install Node.js if you haven't already.

Install Playwright: npm install playwright

Install @axe-core/playwright: npm install @axe-core/playwright

v4.10

I have below questions.
1.What default rule Axe-core uses to scan the website for accessibility ?

Issue:-
2.When I am using axe core to scan web page I noticed that It is giving issue for Color contrast.
It is showing expected color contrast should be 3, but it found 1.9.2 and apart from this I got other issues as well.
I have concern about this[I read some where in article that Default scan tag used by axe core is WCGA2.0, is that correct? if yes then is that color contrast rules are different form WCAG2.0 & WCAG2.1? if yes then what is the different?]
And same when I scan the page using tags "wcag22aa","wcag21aa",then my page does not show any accessibility issues.

Now I wanted to understand this, what happens why it gave error with default scan and why not with wcag22aa","wcag21aa" tags?
What does it mean?

3.How accurate the scan results is with axe-core? in terms of %.

4.Can we take it as 100% accessibility free it it passes certain scan rule?

@straker
Copy link
Contributor

straker commented Sep 16, 2024

Thanks for the questions.

  1. Axe-core runs all WCAG 2.0 and 2.1, A and AA rules except those with tags experimental and deprecated. WCAG 2.2 and WCAG AAA rules are not run by default.
  2. Would it be possible to see the page where the error is happening? Without that it will be hard to tell why it's giving color contrast issues.
    As for the reason it doesn't throw issues for wcag22aa, wcag21aa tags, that is because color contrast is a wcag2aa rule. The way the tags work in axe-core is you have to pass all the tags you want enabled. For example, wcag22aa does not enable wcag21aa nor wcag2aa. You'll have to pass each of those for those rules to run.
  3. Axe-core aims for a 0 false positive rate, so when axe-core reports a violation it typically means it's a real problem.
  4. Automated testing cannot guarantee that a web page is 100% accessible. Automated test can only capture ~60% of all accessibility problems. The remaining problems would have to be found by manual investigation and testing the page. Furthermore, automated tests are not always able to 100% test any given WCAG rule, so even if the axe-core result passes the rule, it doesn't mean there are no problems.

@praveshpr
Copy link
Author

praveshpr commented Sep 19, 2024 via email

@straker
Copy link
Contributor

straker commented Sep 20, 2024

The way the tags work in axe-core is you have to pass all the tags you want enabled. For example, wcag22aa does not enable wcag21aa nor wcag2aa. You'll have to pass each of those for those rules to run.

color-contrast is a wcag2a rule, and the issues above are from the link-in-text-bock rule, which is also a wcag2a rule. When you pass tags to be run in axe-core, you have to pass all tags you wanted run. Since you did not pass wcag2a as a tag it did not run those rules.

So if you wanted to run all WCAG A and AA rules, you'd need to pass the following tags (we don't have any wcag22a rules so it's not passed):

wcag2a, wcag2aa, wcag21a, wcag21aa, wcag22aa

@praveshpr
Copy link
Author

praveshpr commented Sep 21, 2024 via email

@straker
Copy link
Contributor

straker commented Sep 23, 2024

When axe-core runs, it by default will run all axe-core rules that have the tag wcag2a, wcag2aa, wcag21a, or wcag21aa. Each axe-core rule is tagged with only a single WCAG version which is when the WCAG rule was first introduced and not necessarily in which of the versions the rule is part of.

So for example, the button-name rule, which is based on WCAG 4.1.2, was first introduced in the WCAG 2.0 specification. So even though the rule applies to the WCAG 2.0 (A), WCAG 2.1 (A), and WCAG 2.2 (A) standards, the axe-core button-name rule itself is tagged only as wcag2a (since it was first introduced in WCAG 2.0).

What that means is that in order to run the button-name rule, the wcag2a tag must be passed to axe-core when you pass it tags. So if you only pass the wcag21a tag, no wcag2a axe-core rules will be run.

To see which axe-core rules use what tags, you can look at the "Tag" column for the rule in our rule descriptions documentation.

Going back to the output of the issues you posted, they look like they are from the link-in-text-block rule (the help url all point to https://dequeuniversity.com/rules/axe/4.10/link-in-text-block?application=playwright). The link-in-text-block rule checks that text links can be distinguished from the surrounding text. The rule checks the color contrast of the link to the surrounding text to determine if the link uses a contrast that is greater than 3:1. If the links do not pass the color contrast check, and the links do not use any other visual styles to distinguish them, we report violations for the link that the contrast is not sufficient.

The axe-core link-in-text-block rule is a different rule than the axe-core color-contrast rule. link-in-text-block is tagged with wcag2a. This is why when you run the page using only wcag22aa and wcag21aa tags, the rule does not run and you don't see any violations dealing with color contrast (i.e. link contrast from the link-in-text-block rule).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants