How We Made Redundant Security Analysis Nearly Free Without Trading Away Correctness

Adding AI to your security scanning introduces a big new coefficient to the marginal cost of producing and/or validating a finding. We just shipped a response cache in front of the LLM calls our SAST/SCA analysis agent makes when it decides whether a scanner's finding is actually exploitable. I want to walk through why caching that analysis is more nuanced than it sounds: the verdict has to come out exactly the same, every time, or the cache isn't worth having.

Re-analyze everything, or guess what to skip?

Any team using AI to analyze security scanner results runs into this question fast, whether you're building the system or just evaluating one: do you re-analyze every scan, in full, every time? Or do you try to guess which findings actually need a fresh look and skip the rest?

The question isn't hypothetical, because scanning isn't a one-time event. Once you're scanning continuously, you're generating a new batch of results to analyze on a regular cadence, and most of what's in any given batch is code your last analysis already looked at. A pipeline that fires on every commit reruns full analysis, even on merges that touch nothing relevant to any open finding. None of that is misconfiguration. It's just what continuous scanning looks like.

The safe bet is to always re-analyze everything, but every one of those repeats spends real time and real tokens, no matter how redundant the question actually is. The riskier bet is to skip some analyses that you think might be redundant. In this case, you're betting that nothing relevant changed, but with no real way to know you're right. And re-running it doesn't even settle the question cleanly. Two runs over identical inputs can land on different verdicts, just from ordinary non-determinism in the model's responses. Same finding, same code, two different answers depending on which run you looked at. That's not just a cost problem. It's a trust problem, and it's the one I actually wanted to fix first.

Why fingerprinting doesn't work

The first thing everyone reaches for is: if a finding has the same ID as last time, skip re-analyzing it. It sounds right, so it's worth taking seriously instead of waving it off.

But here's the problem. A finding's ID is a fingerprint of its code location, and a fingerprint answers exactly one question: is this the same finding as last time? That's identity. It has nothing to say about a completely different question: does the prior verdict still hold? That's validity, and validity is a semantic judgment about the finding along many dimensions, not a hash comparison.

Identity and validity can move independently, and I've watched it go wrong in production. Consider the case where a scanner flags an XSS finding; an AI agent determines it to be a false positive because it sees a custom sanitizer the scanner doesn't recognize. After someone removes that sanitizer, the same scanner reports the exact same finding, same ID, same fingerprint, because nothing about the flagged code location changed. But the verdict should flip the next time the AI agent looks at it. If you skip re-analysis and instead reuse the previously computed false-positive verdict, then you're operating with a false sense of confidence that you're not vulnerable and now you're deprioritizing a real threat.

The next thing everyone tries is to come up with a new way to fingerprint the finding: you stop trusting the tool's ID alone, and build your own fingerprint (e.g. from the data flow) instead. You tell yourself that your fingerprint will change when something material has actually changed about the finding. This is a painful path, and I've seen smart engineers be humbled by the journey. What they come to learn is that the data flow alone usually isn't enough to answer does the prior verdict still hold? The sanitizer from the last example wouldn't even show up in the data flow, since the scanner never recognized it as part of the flow to begin with. Other findings fail for a different reason: the setting that decides the verdict lives in a config file that never touches the data flow at all. Two identities, one finding, still no correct answer.

That config-file case isn't a one-off. A verdict can hinge on whether a web framework's own configuration enables CSRF protection automatically, a setting nowhere near the flagged line and outside anything a data-flow analysis would ever touch. Knowing to fold that file into your fingerprint requires already knowing it mattered before you had any reason to think so. That's exactly the judgment the analysis exists to make, not something a fingerprint's design can anticipate ahead of time. It's the same reason feeding an agent the right context is its own discipline, not an afterthought.

Every attempt runs into the same wall eventually. Refusing to guess at that isn't caution. It's the only correct move once you understand why the guess can't be made safely.

None of this is a tuning problem. You can't fix it by hashing more context or less, because context volume was never the axis the question lives on. Identity and validity are different questions, and no amount of turning one knob answers both.

It gets worse, not better, in a world without a scanner-issued ID to lean on at all. Imagine a system that hunts for vulnerabilities directly with an AI agent instead of triaging a scanner's list. There's no fingerprint handed to you up front: the system has to decide, every commit, whether anything worth re-checking happened. Skip a commit that should have been checked and you don't get a false positive you can catch later. You get a miss you never see. In a security tool, that's close to the worst failure mode there is.

The inversion: cache interactions with the model, not the analysis

Lay the three approaches side by side and the pattern is obvious in hindsight:

Here's what actually works, and it took inverting the problem instead of solving it harder.

An analysis's inputs can't be enumerated, so stop trying to cache at that level. But a single interaction with the model has a complete input you can hold in your hands: the call itself. The model's name, the instructions, the tool definitions, and the full conversation so far, including every piece of evidence the agent already read, all of it sits in that interaction, because there's nowhere else for it to be. The interaction doesn't approximate the analysis's inputs at that moment. It is those inputs, byte for byte.

Mechanically, this is a proxy that sits between the analysis agent and the model: a request whose bytes match one seen before gets the stored response back, and anything else goes to the model, with the answer stored for next time. None of this is a wrapper bolted onto an API call. The distinction matters more than it sounds: a wrapper forwards requests, a system knows what it's already answered.

The part I like best: the cache key is the conversation, not the commit. That dissolves the same wall as before, because the key gets assembled by actually running the agent and recording what it looked at, not by guessing upfront what it would need. Commits that never enter the conversation never enter the key. A finding whose flagged code and evidence files are untouched replays for free, even on a branch that's moving nightly underneath it.

That per-request matching also means a change mid-conversation doesn't waste the whole analysis: the turns before it still replay from cache, and only the turns after the change actually call the model.

Why a cache hit can't be a wrong answer

A response served from cache isn't an approximation: it's the model's own answer to exactly this call, replayed. Serving it is indistinguishable from calling a model that happens to be deterministic, which is a genuine upgrade over the status quo: two runs over identical inputs used to be able to land on different verdicts by ordinary sampling variance. With the cache warm, they can't. Of course, if we repeat the analysis with some steering from the user, the user's instruction is a new input and thus invalidates the cache. Absent that steering or other input changes, we actually want the same result replayed.

The failure modes here are asymmetric, and that asymmetry is the whole safety argument. Serving a hit it shouldn't is impossible, because exact-byte matching has no notion of "close enough" to get wrong. Missing a hit it could have served just costs full price, which is what every call cost before this existed. Every way this can go wrong degrades toward expensive-but-correct. It never degrades toward wrong. The worst day this cache can have is a day where it does nothing at all.

It also doesn't depend on today's particular agent design. The only premise this needs is that the model never sees anything the call didn't carry, and that's true of any API call by construction, not a property of the harness. What can be blamed on the cache stays bounded too: if a narrow question misses context it should have carried, an uncached run asks the identical narrow question and does no better. The cache never adds a blind spot, and it never removes one either.

What you'd actually own building this yourself

Neither part of this was the easy weekend project it sounds like from the outside, and I think that's the actual argument for not building it yourself.

The identity-vs-validity trap isn't theoretical caution I'm adding for effect. It's what several engineers, myself included, spent real years on at a previous company without ever finding the one true fingerprint. It's a story about the problem itself not having the shape people assume it has going in.

And getting the call-level cache to actually pay off required its own unglamorous fix: prompt construction had to be made genuinely deterministic across runs, which isn't automatic and turned out to be its own real piece of engineering, the kind of work that never shows up in a demo and shows up immediately the first week something runs in production.

If you're weighing whether to build this yourself, that's the actual bill: not the cache, but getting both invariants right before anything ships. Neither one announces itself as a problem until it's already live.

That's the narrow slice of a wider question. We've made the fuller build-vs-buy argument elsewhere if this is the part of the decision you're actually stuck on.

What it buys you

Cost now tracks what actually changed, not the size of your backlog. In a nightly scan of an active branch, most findings' relevant code didn't move, so most of them replay free. Only the findings the week's commits actually touched pay for the model inference.

Cheaper, more consistent analysis feeds the same engine that both cuts through scanner noise on the front end and produces the fix on the back end, so neither side gets cheaper by getting less careful. Making analysis cost less was never worth shipping if the price was that it could occasionally be wrong instead.