← Field Notes
Field Note · security

The security hole an AI left in a helper function.

A filename rendered straight into the page through a toast message. It was AI-written, it just showed a message, and nobody reviewed it. Here is the commit that closed the XSS.

Builds production software, and cleans up the AI-generated code that breaks it. pdflokal, his open-source PDF toolkit, is one of the repos these notes come from.

pdflokal shows a little toast when something happens: file loaded, saved, that kind of thing. It also shows a full-screen loader with a message. Both are helpers. Both take a string and put it on the screen. One of the strings they showed was your filename.

The helpers built the message with innerHTML. Which means the browser did not treat the filename as text. It treated it as HTML. A file with an ordinary name is fine. A file named <img src=x onerror=…> is a script, and the toast that “just shows a message” would run it.

Nobody reviews a toast

That is the whole reason it survived. The file parser gets scrutiny. The export path gets scrutiny. The upload gets scrutiny. The function that renders a small grey notification at the bottom of the screen gets a glance, if that. Reviewers spend their attention where danger is supposed to live, and a toast is not supposed to be dangerous. So it was exactly where the hole was.

element.innerHTML = messageThe vulnerable line looks completely normal. Every developer has written it a thousand times. It is the most ordinary way to put text on a screen, and it is wrong only when the text is not yours.

The fix (a58f072) was to stop using innerHTML: build the node with createElement and set textContent, which the browser escapes. Two helper functions, a few lines. The XSS closed the moment the string stopped being parsed as markup.

It was not the only one

The same look found more, and all of it in the plumbing. The content security policy still allowed unsafe-eval, so any injection that slipped through could construct arbitrary functions; that got dropped (6a4fe71), but only after actually verifying, end to end, that no app or vendor code needed it. And a greedy regular expression flagged as vulnerable to catastrophic backtracking got swapped for a plain string split (4c13a39). None of these lived in the interesting code. Every one was in the boring code.

What this actually teaches

AI hides risk in the boring places, because that is where humans stop looking. It writes the toast helper with the same fluent confidence it writes the core logic, and the confidence is uniform while the review is not. We scrutinize the scary surfaces and wave the mundane ones through, and the mundane function, the one that just shows a message, is where the hole lives.

The tell is that the vulnerable line looked ordinary. Not exotic, not clever, just innerHTML with a string, the most common way there is to show text. AI reaches for the ordinary line because the ordinary line is what it saw most in training. Fluency toward the common case is a security liability precisely when the common case is unsafe, and no one is reviewing the place it landed.

So when you audit AI-built code for security, invert your instinct. Do not start with the auth or the parser; the AI probably copied those from a well-known pattern. Start with the helpers it wrote itself: the toast, the loader, the little formatter, anywhere a user-controlled string meets the DOM. The hole is in the boring code, because the boring code is where nobody looked.

The receipts, public
  • The XSS fix: showToast/showFullscreenLoading stop using innerHTML on a filename (2026-03-17)a58f072
  • Drop unsafe-eval from the CSP, after verifying nothing needed it (2026-05-28)6a4fe71
  • A greedy regex flagged as ReDoS-prone, swapped for a plain string split (2026-06-02)4c13a39

Does AI-generated code have security vulnerabilities?

Yes, and they cluster in the code nobody reviews. In our own product the XSS was not in the auth flow or the file parser; it was in showToast and showFullscreenLoading, UI helpers that rendered a filename with innerHTML. AI writes those helpers as confidently as anything else, and because they just show a message, they skip review.

What is an innerHTML XSS?

Assigning user-influenced text to an element's innerHTML makes the browser parse it as HTML, so a filename like <img src=x onerror=alert(1)> executes as code instead of showing as text. The fix is to build the node and set textContent instead, which the browser treats as text, not markup.

Where do security holes hide in AI-generated code?

In the boring places: the toast, the loader, the helper, the string that just gets displayed. Reviewers scrutinize the obvious surfaces (auth, parsing, uploads) and skip the mundane ones, and that is exactly where an unescaped value or an unsafe default survives.