Methodology
The alert is gone.
Does it still work?
We test the patch against attacks and ordinary use. Closing the warning is only the first check.
Your AI’s patch
Same code. A second opinion.
Alert gone?
Re-run the scanner on the patch.
Attack stopped?
Try the SQL injection cases.
Still works?
Compare valid inputs with a correct reference.
No shortcuts?
Check that warnings were not suppressed.
A verdict.
With receipts.
See which checks passed, what failed, and the inputs behind the result.
See pilot results ↗Illustration of the method · hosted patch scoring is not live yet
Three possible outcomes.
Verified
All four checks passed for the tested cases.
Not verified
At least one check failed. The evidence shows where.
Insufficient evidence
The run could not establish a result. Never a pass.
A pass is evidence for these cases, not a guarantee that the whole application is secure.
Where we are today
SQL injection.
Four test cases.
Our current benchmark uses seeded Python/SQLite modules: quoted strings, numbers, ORDER BY and second-order injection.
Small samples and published tests have limits. The blind pilot kept test cases from the fixer; the public benchmark is a separate exercise.
Open the patch form ↗Scope, evidence and technical details
Early demonstrations tested three constructed patches. Provider results come from the separate blind pilot. The proposed real-project Index is not live. No-alert records are reported separately from scored patches.
The backend is designed to run patches in isolated Docker containers with resource limits and a timeout. It needs a separate deployed service; Vercel hosts this website.
Results depend on test coverage, an executable environment and a trustworthy reference. They do not establish safety for other languages or vulnerability classes.
Detailed method reference · Reports and source artifacts
A fix can close the hole and quietly break a real user. Inspect the evidence and its limitations before relying on a verdict.