A11y plugin improvements

[juliemr]

• pending Plugins: Add ability to hook in at the end of each test #1842 - audit after each test and merge results
• Look into adding custom audits specific to Angular applications
• Enable Tenon.io auditing for users of that service
• Run the tests against Protractor's own website
• Add an option to configure whether warnings from Chrome a11y are treated as pass or fail
• Lower priority: Add meaningful duration numbers to test output
• More sophisticated truncating of long element strings

@marcysutton did I miss anything?

[marcysutton]

• Missing accessibility/successConfig in test.js
• Header identifying Chrome Accessibility Developer Tools in results

Question: will a11y plugin failures fail CI builds? Seems like part of #1842. I'm also curious what the output will look like when all the results are merged: will the accessibility Passes/Fails output separately for each page or all together at the end? I suppose we could identify elements by the page that triggered them if the results are merged together.

Added a little more info @juliemr. Thanks!

[juliemr]

Yup, plugin failures will fail CI builds - they change the exit code to non-zero, which is all that Travis cares about.

For merging results, I think it's up to what we feel is most useful. We should probably use the test name to help developers pinpoint where failures actually occurred, but do some sort of deduplication if the same issue is present in a bunch of tests.

[marcysutton]

I think I got the "identifying header" part solved by adding text to the Pass/Fail descriptions. It's especially useful when you run both integrations at the same time. PR coming shortly.

[marcysutton]

Hey @juliemr I have another improvement to add to the accessibility plugin: make the output color configurable. The red and green colors are very hard to see for some folks.

[sjelin]

There's a limited amount we can do with that. Most of the output comes from tools like jasmine.

[dylanb]

@juliemr I would like to integrate the axe-core https://github.com/dequelabs/axe-core ruleset into this functionality - WDYT?