Evidence
Check our work.
The results matter only if you can inspect how we reached them. Start with the blind pilot, then follow each verdict back to its artifacts.
Start here
Pilot 001 · blind run · 14 September 2026What happened when the fixers
couldn’t see the tests.
Four SQL injection shapes. Results from ChatGPT, Claude Code and GitHub Copilot Autofix, with the tool versions, exclusions and limitations.
Read the report ↗The source behind the numbers
The published benchmark material is separate from the blind pilot’s withheld cases. Public results and code do not make those experiments interchangeable.
Reports and worked examples
The dated reports preserve their original URLs. Earlier findings remain available alongside the later corrections.
About Assay
Assay independently checks AI-written security patches. We do not generate the fixes we evaluate. Our current work focuses on SQL injection; automated verification on your own pull requests is still in development.
We publish the method, its limitations and failures discovered in our own checks. Vendors can respond to their results; their responses are published unedited.
Earlier reference documents
Detailed method v1 · Original whitepaper. These preserve the technical history and early proposals. For current scope, see Methodology.