request for test page list

[wittjeff]

On https://pour.dev you mention "138 live pages". Could I get that list for some robustness testing I'm doing prior to inclusion of your package in testaro?

[davidyarham]

Hi Jeff, sorry this one sat for so long, I only caught up with the repo today.

I'm not going to publish the full list I'm afraid, I'm keeping the corpus itself private for now. But I can tell you what it is made of, and point you at the parts that are public, which should get you most of the way for robustness testing.

It is 138 live pages, sized by DOM: 123 under about 5k nodes, 14 between 5k and 50k, and one monster (a single page spec at around 190k nodes) to see what happens at the top end. The mix is deliberate failure pages, fixed controls, government and university homes, banks, news and commerce fronts, charities, accessibility community sites, design system docs, and some sparse old school markup. Nothing behind a login, and nothing that serves a bot interstitial instead of the page.

The deliberate failure pages are all public and are the most useful ones for what you're doing, since you know what should be found:

• GDS tool audit: https://alphagov.github.io/accessibility-tool-audit/test-cases.html
• Deque Mars demo: https://dequeuniversity.com/demo/mars/
• UW Accessible University, before: https://projects.accesscomputing.uw.edu/au/before.html
• and after, as a fixed control: https://projects.accesscomputing.uw.edu/au/after.html
• Beta Block, mine, with a ground truth manifest: https://pour.dev/demo/betablock/

One thing worth knowing if you build your own list: live pages move under you. This week a retailer changed how it draws struck-through prices and that alone surfaced a false contrast failure that seven weeks of runs had never shown. Carousels are the other one, a page can give two different answers a few seconds apart. So I'd treat any fixed list as a starting point and re-check the odd results against the page.

On your four PRs, they are all dealt with now, details on each. #7 and #8 were already fixed in 1.41.0, #10 is merged with a wider text test on top, and most of #9 went in rebuilt on the current engine. That is all in engine 1.44.1, tagged here as v1.44.1 and on npm now. Thanks again for those, they were properly researched and the engine is better for them.

DY