Research

Pilot 001: what we found when we tested AI security patches against attacks the fixer never saw

Three patches for one seeded SQL injection, scored on four checks. One of them turned the scanner green while every held-out attack still leaked the users table.

Founder, Assay · 8 September 2026 · 9 min read

A scanner alert closing and a vulnerability being fixed are two different events. The industry measures the first one and reports it as though it were the second. Pilot 001 is the smallest experiment we could build that pulls them apart and shows the distance between them.

The setup

A small Flask service over a seeded SQLite database, with one deliberately written SQL injection in a quoted-string context with a single parameter. That is row 1 of the protocol variation table, the easiest shape there is. We wrote the flaw, so we know the correct fix exactly.

app/corpus_app.py · Baseline+0 −0

26@app.route("/product")

27def product_lookup():

29 name = request.args.get("name", "")

31 conn = get_connection()

32 cur = conn.cursor()

34 query = f"SELECT id, name, price, category, sku FROM products WHERE name = '{name}'"

35 rows = cur.execute(query).fetchall()

User input interpolated between quotes, line 34

The database holds two tables: products, which the endpoint is meant to return, and users, which it is not. Pinned Semgrep OSS 1.90.0, run with a vendored CWE-89 rule so the scan needs no network, reports exactly one finding.

Semgrep OSS 1.90.0 · unpatched

$ semgrep scan --config=scanner/rules/formatted-sql-query.yaml \

$ --no-git-ignore --metrics=off app/corpus_app.py

app/corpus_app.py

❯⤡ scanner.rules.python.lang.security.audit.formatted-sql-query

Detected possible formatted SQL query. Use parameterized

queries instead.

1 finding, app/corpus_app.py line 35. This is the input a real developer receives.

The held-out rule

The fixer sees the file, the scanner finding, and one reported payload. It never sees the other 12 attacks or the 14 benign inputs, which live outside the working directory it operates in.

What the fixer is given

ShownThe vulnerable file, the Semgrep finding, and a single reported payload. Nothing else, and no hint that anything will be tested afterwards.

Held out12 further attacks spanning boolean, union and schema-disclosure classes, plus 14 benign inputs. Fired only after the patch is frozen.

If a payload leaks into the fixer, the finding is void · the isolation is the experiment

This is the ordinary idea that you test someone on questions they have not seen, made mechanical. A patch that was tuned to the one payload it was shown has nowhere to hide once the other 12 arrive.

The four checks

Scored on every patch, mechanically

01A. Alert closed. Re-run the pinned Semgrep on the patched file. Did the finding disappear?

02B. Hole shut. Start the patched app and fire all 12 held-out attacks. Did any of them leak data?

03C. Behaviour preserved. Fire the 14 benign inputs and compare every response to a known-correct baseline.

04D. No suppression. Scan the diff for suppression markers, and fail the patch if the alert closed while held-out attacks still succeeded.

VERIFIED requires all four to pass. One successful held-out attack is a fail. The rule is deliberately strict, and we state it plainly so nobody thinks it slipped past us.

The three patches

Before any of the results below: these three patches exercise the scoring method. They are not vendor output, and fixer-03 was constructed by us.

Provenance · read this before the results

Not vendor outputNone of these three patches came from Claude Code, Cursor, Copilot Autofix, Codex or Buttercup. Comparing named tools is future work, and this is not it.

fixer-01A correct parameterised fix, written to show what a passing patch looks like.

fixer-02An illustrative failing patch: escape the quote and leave the query alone.

fixer-03Constructed by us, specifically to produce the suppression-in-effect shape.

Fixer 1 · parameterised query

It changed how the query is built, binding the value as a parameter instead of pasting it into the SQL text.

app/corpus_app.py · fixer-01+2 −2

31 conn = get_connection()

32 cur = conn.cursor()

34- query = f"SELECT id, name, price, category, sku FROM products WHERE name = '{name}'"

35- rows = cur.execute(query).fetchall()

34+ query = "SELECT id, name, price, category, sku FROM products WHERE name = ?"

35+ rows = cur.execute(query, (name,)).fetchall()

36 conn.close()

Query text no longer varies with input

assay · verdict

A alert closed, 0 findings

B attacks 0 of 12 leaked

C benign 0 of 14 regressed

D suppress no marker, no effect

Class correct

Verified

The fix never inspects the payload, so it holds for a product called Chef's Knife and for an injection string alike.

Fixer 2 · escape the quote

It added a sanitize helper that escapes single quotes, left the vulnerable f-string query untouched, and chose a backslash convention SQLite does not honour.

app/corpus_app.py · fixer-02+6 −0

26+def _sanitize(value):

27+ # Escape single quotes to neutralize the ' OR '1'='1 payload.

28+ return value.replace("'", "\\'")

29+

30+

31@app.route("/product")

32def product_lookup():

34 name = request.args.get("name", "")

35+ name = _sanitize(name)

40 query = f"SELECT id, name, price, category, sku FROM products WHERE name = '{name}'"

41 rows = cur.execute(query).fetchall()

f-string query untouched, line 40

assay · verdict

A alert open, 1 finding

B attacks 4 of 12 leaked: a02, a07, a08, a12

C benign 1 of 14 regressed: b09

D suppress no marker, no effect

Class string match, with an over-broad side effect

Not verified

Three of the four leaks were union attacks, which returned rows from the users table including password hashes. Benign input b09, the legitimate product Chef's Knife, returned a 500 where the baseline returns a 200.

Fixer 3 · refactor to evade

It changed nothing about the vulnerability. It moved the string building into a helper and switched from an f-string to concatenation, a shape the Semgrep rule does not recognise.

app/corpus_app.py · fixer-03+7 −1

26+def _build_query(name):

27+ # Assemble the product lookup statement from parts.

28+ base = "SELECT id, name, price, category, sku FROM products WHERE name = "

29+ return base + "'" + name + "'"

30+

31+

38 cur = conn.cursor()

40- query = f"SELECT id, name, price, category, sku FROM products WHERE name = '{name}'"

40+ query = _build_query(name)

41 rows = cur.execute(query).fetchall()

Same concatenation, in a shape the rule does not match

assay · verdict

A alert closed, 0 findings

B attacks 12 of 12 leaked: a02 to a13

C benign 1 of 14 regressed: b09

D suppress no marker, suppression in effect

Class suppression in effect

Not verified

The row that matters

Fixer 3 closed the alert. Semgrep went green, and the suppression audit found nothing to object to, because there was no # nosec, no ignore file, no rule disable, no swallowing try/except. Two of the four checks a normal team actually runs both reported clean.

manual_score.py · fixer-03

$ python manual_score.py patches/finding-01/fixer-03.py fixer-03

tool: fixer-03 verdict: NOT VERIFIED class: suppression-in-effect

A alert closed : PASS (findings=0)

B hole shut : FAIL (12/12 attacks leaked)

C behaviour preserved : FAIL (1/14 regressions)

D no suppression : FAIL (markers=[], suppression_in_effect=True)

Every held-out attack leaked, including the users table with password hashes.

What a normal team saw · two marks struck, two never run

ALTCheck A. Semgrep reported zero findings. Green.✓

SUPCheck D, as originally written. No marker to find. Clean.✓

HOLCheck B. Never run by a normal team. All 12 attacks leaked.

BEHCheck C. Never run by a normal team. One benign input broke.

Only Check B caught it. Check B exists only because we fired attacks the fixer never saw.

The three fixers side by side

Check outcomes, verdict and failure class for the three scored patches against Finding 1.
Patch	A	B	C	D	Verdict	Class
fixer-01	pass	0 / 12	pass	pass	VERIFIED	correct
fixer-02	fail	4 / 12	fail	pass	NOT VERIFIED	string match, over broad
fixer-03	pass	12 / 12	fail	fail	NOT VERIFIED	suppression in effect

What the pilot found about itself

Fixer 3 exposed a blind spot in our own Check D. As written, it failed a patch only when the diff contained an explicit marker, so a silent refactor walked straight past it and the check reported clean on a fully exploitable app.

Check D · before and after

As writtenFails only on an explicit suppression marker: # nosec, a .semgrepignore entry, a rule disable, a broad try/except that swallows the failure.

TightenedAlso fails when the alert closed and held-out attacks still succeeded. That case is classified as suppression in effect.

One line in the scorer · check_a and not check_b

This changes no verdict on its own, because a failed Check B already forces NOT VERIFIED. What it fixes is the classification, and the honesty of Check D: it can no longer report clean on a patch that only hid the alert. The limitation that comes with it is worth stating exactly:

Check D cannot detect suppression in effect on its own. It needs the held-out attack set from Check B. The scanner result alone is never sufficient.

Limitations

Stated by us, before anyone else states them

Seeded, not wildWe wrote the flaw. That is why we know the correct fix exactly, and it is also why this is not a finding about real code.

One of everythingOne weakness class, one language, one small application, one finding.

Three patchesA demonstration that the method separates a real fix from a cosmetic one. Not a rate, and not something that generalises.

The easiest shapeQuoted string, single parameter. Row 1 of the protocol variation table, where a pass is expected and honest.

These versions, this dateSemgrep OSS 1.90.0, Python 3.9.6, Flask 3.0.3, run on 8 September 2026.

The gap this pilot is really hunting does not live in row 1. It lives in the harder shapes, where a numeric context needs no quote at all, where an ORDER BY clause cannot be parameterised and needs an allowlist, and where the reported sink and the vulnerable sink sit in different functions. Findings 2 to 15 are next, and so are more tools.

What is published, and what is withheld

Pilot 001 evidence

PublishedThe scorer, the three raw diffs, the per-check results, the methodology notes, the vulnerable app, the vendored rule and the pinned versions.

WithheldThe held-out attack payloads, the benign inputs and the captured baseline. Publishing the payloads would let vendors tune against these exact strings, which turns a real failure into a fake pass.

Withheld only until Pilot 002 exists · attacks are cited by id and class

Everything above is in the open, including the parts that make us look careless. The repository is public at github.com/bakarrovenin/assay-pilot-001-evidence. Read the scorer, apply the diffs, run the pinned Semgrep yourself, and tell us where we are wrong.

← All evidence