How an LLM instruction header ended up in my public structured data

TL;DR: A metadata header I wrote for an extraction model was stored in the same field as the cleaned listing text, and a publish step written later copied that field into the JSON-LD structured data on public pages. Visitors never saw it, but search engines and every crawler reading structured data did, on dozens of pages of a listings site I run. I found it by opening the page source of a page an SEO crawler had flagged for an unrelated reason. The fix was a strip at every public boundary, a fallback that publishes a short summary when stripping is not safe, and a data backfill. The rule that followed: model-facing text is assembled at call time, the header builder and its stripper live in one module, and every public boundary that reads an internal field gets a mapper plus a test that injects prompt-like content and asserts it does not come out.

How did a prompt header end up in public structured data?

The listings site runs a small pipeline. It fetches a listing, cleans the HTML, and sends it to a language model that extracts structured fields. To make extraction more reliable, the pipeline prepends a short header to the text: facts it already knows from the source, such as location and offices, marked as authoritative so the model prefers them over guesses from the prose. That is ordinary context engineering: shaping what goes into the model's context window so the output is better.

The problem was where that header lived. The cleaned text, header included, was stored in the database, because the extractor needs the same input on reruns. A separate publish step, written later and for a different purpose, derived each listing's public description from that stored field verbatim. The visible page rendered a short summary, so nothing looked wrong to a human. The JSON-LD block, the machine-readable structured data that search engines read to build rich results, got the full description, header first.

No secrets were in the header, and no user data. It was still text written for a model, published to every search engine and every crawler that reads structured data. OWASP's Top 10 for LLM applications lists system prompt leakage as its own category, and this was a small, mundane instance of it. The model did nothing wrong; the leak came from the plumbing around it.

How do you find a prompt leak on a public page?

Not by looking at the page, which is exactly why it survived. I found it sideways. An SEO site audit had flagged structured-data errors across the site, which I fixed in shared layout code. After the recrawl, three pages still carried a warning about double-escaped HTML entities in the description. I opened the page source of one of them to see the actual JSON-LD, and the description started with my extraction header.

From there, the work was about scope. I scanned every listing in the sitemap and found the header in the structured data of dozens of live pages. The three flagged pages were only the ones where the leak also broke HTML escaping; the rest were valid structured data that happened to contain prompt context, which no validator will ever complain about.

That is the uncomfortable part. Tooling checks whether structured data is well formed, not whether its content was meant for the public. A leak like this is invisible to users, passes validation, and only shows up when a person reads the source.

Fix the data, fix the render, or stop storing the header?

I had three options, and each one alone fell short.

• Backfill the stored data only. Cleans today's rows. The next pipeline run writes the header again, and the next publish copies it again. It treats the symptom.
• Strip at render time only. Makes the public pages safe immediately, without touching data. But the dirty field stays, and every future consumer of it (an RSS feed, an API, a second site reading the same database, a feed prepared for LLM crawlers) has to remember to strip it too. Relying on memory is how this happened in the first place.
• Stop storing the header at all. Assemble it when the model is called and never persist it next to publishable text. This is the correct long-term shape. It also meant changing the extraction side, which runs in an older service I was in the middle of migrating, and the extractor genuinely needs the header on reruns.

I combined the first two and kept the third as the rule for all new code. The stored field stays, explicitly treated as internal, with the header intact because the extractor needs it. Every public boundary strips it. The existing rows were backfilled so the data itself is clean. That gives defense in depth: if a new publish path forgets the strip, the data underneath is already clean; if the pipeline writes the header into a publishable field again, the boundary catches it.

What shipped, and what the first fix missed

The first fix moved the code that builds the header and the code that strips it into one module. The publish step and the structured-data builder both strip at the public boundary, and a test feeds a realistic header in and asserts that no model-facing text comes out:

// illustrative shape
it("never publishes model-facing context", () => {
  const stored = buildModelHeader({ location: "Remote" }) + "Real body";
  const out = toPublicStructuredData({ description: stored });
  expect(JSON.stringify(out)).not.toContain(MODEL_HEADER_MARKER);
});

After that deploy, most of the affected pages were clean. The rest were not, and this is the part worth learning from. Those listings had been processed months earlier, when the header had a different format for each source. Some of the older variants were stored as plain text, without the markup that marks where the header ends, and fused with the body. There was no safe place to cut. Any rule I could write would either leave part of the header or eat part of the real description.

So the second fix changed the policy instead of the regex. The stripper learned the older formats it could handle reliably. When a header is still detected after stripping, the structured data publishes the short summary instead of the full description, and the publish path throws instead of writing. Never guess, never publish. A slightly thinner description is a fine price for a guarantee that prompt text does not leave.

The backfill came last. It is a script that runs as a dry run by default and only writes with an explicit flag, and it needs a production key I deliberately keep out of the local environment, so I ran it myself. The dry run was its own lesson: the header sat in several times more stored rows than the live scan had suggested, including listings that were no longer published. A scan of public pages measures exposure today; it says nothing about what the next re-publish would expose. Most rows were stripped in place, some were rebuilt from the internal source text, a few fell back to the summary, and a small group in yet another format needed a second pass with an explicit cut. The third dry run reported zero rows, and the same dry run now serves as a monitor: its expected output is zero.

What is the governance rule now?

AI governance usually means policies about how an organization uses models: what data goes in, what comes out, who reviews it. At the scale of one person running a product, it means a few rules concrete enough to be enforced by tests. I wrote this one into the global instructions my coding agents load in every project, because agents write most of my pipeline code and they need the rule at the moment they write a new publish path:

1. Assemble model-facing text at call time. Do not store it in a field that anything publishes. If it has to be persisted, the field is explicitly internal and nothing public reads it without a mapper.
2. Every public boundary that reads an internal field has an explicit mapper and a test. The test injects prompt-like content and asserts it does not come out. Public boundaries include the page, JSON-LD, meta tags, RSS, the sitemap, Open Graph images, the API, and any feed prepared for LLMs.
3. The builder and the stripper live in one module. The older variants that defeated the first fix are exactly what happens when the format and its removal evolve in different files.
4. After every pipeline change, open the source of one new page. Read the JSON-LD, the meta tags and the feed, not only the rendered view. That is how this leak was found, and no automated check would have found it sooner.

The underlying idea is data lineage: knowing where a piece of text came from and who it was written for. The header was harmless in the context it was written for. It became a leak because a field's name said nothing about its audience, and a later piece of code reasonably assumed a description was a description.

What I'd tell someone running an LLM pipeline tomorrow

• List your public surfaces and the fields each one reads. If any of those fields ever passed through a prompt, that surface needs a mapper and a test.
• Mark audience in the data model. A field that holds text for a model should be impossible to mistake for text for people.
• Prefer a fallback to a clever cut. When you cannot strip reliably, publish something safe and shorter, and make the write path fail loudly.
• Backfill with a dry run first and trust its count over the live scan. Stored data usually holds more than what is currently visible.
• Read the source, not the page. Validators check shape. Only a human or a targeted test checks intent.

One limit remains. The rule covers the boundaries I know about. A second site that reads the same database went onto the follow-up list the same day, because the whole incident was a boundary nobody had listed.