The Parts That Actually Take Time

A technical examination of what it takes to build AI-powered security remediation that engineers can trust in production, and why it is not a weekend project.

I · Introduction

Beyond a Demo

You can call the Claude API with a security finding and ask it to fix the vulnerable code. Within minutes you will have a plausible-looking patch. Show this to your manager and they will ask the obvious question: why are we paying for an AI remediation tool when I can wire this up myself in a weekend?

It is a fair question, and the answer requires some unpacking.

What the demo does not show is the other 47 Fridays: the ones where the LLM returns syntactically invalid code, or silently removes a security check while fixing another, or produces a fix that is correct for Python but catastrophically wrong for the Java dialect your team actually uses. The ones where the same vulnerability appears in 87 SARIF findings because the SAST tool traced every branch of a tainted data flow, and your script dutifully opened 87 pull requests. The ones where you discover, after three months and two incidents, that your system has been marking exploitable code as false positives because a sanitizer the model assumed existed had been deleted six sprints ago.

We built Pixee to solve these problems. Here is precisely what that took.

The problem has two independent axes. The first is scale: operating reliably across an organization's full repository portfolio rather than a single repository at a time. The second is quality: producing fixes that are correct, safe, and grounded in the specific codebase, not a generic model of it. Teams that attempt this typically make progress on one axis. A coding agent skill can produce reasonable quality fixes for the team that built it; the same solution across 16,000 repositories is a categorically different project. Pixee was built to address both.

Industry context. The Bayer team recently published a detailed account of their production LLM architecture and reached the same conclusion independently: reliable AI systems require deterministic pre-filters, layered validation, structured knowledge injection, and tight observability; none appear in a prototype. This article is our answer to the question "what does that look like specifically for security remediation?"

II · Scale

Built for Organizations, Not Repositories

Every home-grown AI security demo starts the same way: one repository, one coding agent. That falls apart at enterprise scale. One of our enterprise customers has 16,000 repositories. That same customer has a security engineering team of fewer than ten people responsible for governing remediation across all of it, serving hundreds of developers with different stacks, different workflows, and different tolerances for automated changes. Their most AI-forward teams have built impressive automation, but it only works on the repositories those teams own. The organization cannot centralize it, govern it, or extend it. What looks like a solved problem at the team level is unsolved at the organizational level. Building a centralized remediation system that operates safely across thousands of repositories (with per-repo configuration, tenant isolation, cost attribution, and backpressure) is the infrastructure problem that coding agent skills do not address.

None of this appears in a demo. All of it eventually becomes a requirement.

Concurrency control

Pixee uses an AIMD (Additive Increase / Multiplicative Decrease) controller for concurrent LLM requests, the same algorithm TCP uses for network congestion control. When responses arrive successfully, the concurrency limit increases. When errors or timeouts occur, it decreases multiplicatively. A shared LLM API can become temporarily saturated; your concurrency strategy determines whether your system degrades gracefully or falls over.

Cost attribution

Every LLM call is tagged by task type: triage, fix generation, safety judge, spelunking, feedback processing. Those tags flow through to cost dashboards. You can tell exactly what fraction of your AI spend is going to each phase. When costs spike unexpectedly, you know where to look.

Per-finding time budgets

Each finding analysis runs inside an extensible deadline scope: 600 seconds for fix generation, configurable for triage. When a budget expires, the failure report includes elapsed time and remaining budget: the data needed to tune the system, not just "it timed out."

Backpressure

An EMA-smoothed controller tracks average task completion time and can reject new workflow requests under load, returning an estimated wait time rather than accepting work it cannot complete in time. Your monitoring system can respond to a should_accept: false signal before queue depth becomes a crisis.

Per-repo configuration

A PIXEE.md file in the repository root controls Pixee's behavior for that repo: which rules to force-enable or disable, path exclusions, and vulnerability-class-specific fix guidance. Teams can tell Pixee "do not touch the legacy authentication module" or "for SQL injection findings, always use our internal SafeDB wrapper." This configuration is compiled into matchers at startup, not re-evaluated per request.

Episodic memory with quality scoring

When developers provide feedback on Pixee's fixes, that feedback is stored per-rule in an episodic memory layer. It is not blindly stored: a background quality judge evaluates each feedback record. Vague sentiment, off-topic comments, and noise are marked excluded and do not contaminate future prompts. Only signal reaches retrieval.

SCA manifest and advisory normalization

Dependency vulnerability analysis requires ingesting two independent data streams that change independently of each other. Manifest files and lockfiles exist in a dozen formats across ecosystems: npm's package-lock.json, Maven's pom.xml, Gradle's dependency tree, pip's requirements.txt, Poetry's lockfile, Cargo's Cargo.lock, Go's go.sum. Each has different semantics for transitive dependencies, version pinning, and optional versus required packages.

CVE advisories arrive from multiple sources (NVD, GitHub Advisory Database, OSV, ecosystem-specific feeds) with overlapping and sometimes inconsistent coverage. The same vulnerability may appear under different identifiers, with different affected version ranges, across sources. Normalizing these into a coherent advisory store that can be queried by ecosystem, package name, and version range is ongoing infrastructure work. The chain resolution step (mapping an advisory to the specific dependency in a customer's tree) must re-run whenever either stream changes: when a new advisory drops or when a manifest is updated. This is a background indexing problem that never goes away.

Observability

Every workflow emits a full OpenTelemetry span hierarchy: workflow → finding → analysis task → subtask. Each level carries typed counters, duration histograms, and token usage histograms. LLM gateway guardrail detection events are tracked separately: when a safety filter interferes with a request, that is a distinct counter, not absorbed into a general failure bucket.

III · Knowing what's real

Knowing What's Real: Triage Across Every Tool That Doesn't Speak the Same Language

Scale is the organizational axis. Quality is everything that follows. On the quality side, the first challenge is not fixing vulnerabilities. It is knowing which findings are worth fixing at all.

Most security scanners report more noise than signal. Studies consistently find 70–90%+ of SAST findings are false positives; in SCA the figure reaches 92%. The noise is not random: false positive rates vary by tool, by rule family, and by the patterns in a specific codebase. A finding flagged as high-severity by one scanner may be provably safe in your deployment. Determining which findings are worth fixing, accurately and consistently, is the work that happens before any LLM call is made.

The structural difficulty is that every customer's scanning landscape looks different, and almost none use just one tool. Most enterprises run two or three scanners: a tool the compliance team mandated, one the security team prefers, perhaps one inherited from an acquisition. Pixee supports more tools than any one customer runs; the roster expands with every new enterprise onboarded. Each has its own rule taxonomy, severity semantics, finding format, and fingerprinting scheme. SonarQube does not export SARIF natively: Pixee converts its proprietary JSON format. When a tool renames a rule, Pixee updates the mapping. Supporting every tool a customer might bring means building and maintaining a deep integration for each one.

For each tool, Pixee maintains a dedicated triage provider that understands that tool's quirks. These are not thin wrappers. They encode accumulated knowledge about how each tool's rules map to real vulnerability patterns, how false positive rates vary by rule family, and which rule IDs have been silently deprecated. The system also handles deduplication: when a SAST tool traces a tainted data flow through three function calls and reports three separate findings, Pixee identifies them as a single logical fix and generates one PR that resolves all three.

Key architectural fact. Triage does not ask an LLM "is this a real bug?" for every finding. Deterministic rule classification runs first. LLM analysis escalates only when the deterministic path is genuinely inconclusive, keeping costs bounded and verdicts consistent.

IV · Software composition analysis

Reachability Is Not Exploitability

SAST findings at least tell you where to look: a file, a line, a rule. Dependency vulnerabilities compound the problem. A CVE advisory names a package, a version range, and a class of attack. It says nothing about whether your code calls the vulnerable function, what it passes in, or whether that execution path can be reached from an attacker-controlled entry point.

SCA scanners default to at-risk and flag every project with a vulnerable package in its dependency graph, but that label tells you almost nothing about actual risk. The real terms that matter are Reachability and Exploitability. Reachability is determining if the dependency chain physically maps to the vulnerable call at runtime. Most SCA vendors have a reachability story, and it is genuinely useful, but exploitability is where the field thins out. Exploitability asks whether all the conditions exist for that code path to actually execute: does first-party code invoke the vulnerable API, with the input shapes the vulnerability requires, in a context that can reach it from the outside? Conflating these two is the primary reason SCA tooling produces alert fatigue at enterprise scale.

Reachability is a chain-side property: it does not require reading the customer's codebase at all. Pixee analyzes the Taint Consuming Dependency (the package immediately upstream of the vulnerable one) using call-graph analysis to determine whether it actually imports and invokes the vulnerable API. If it bundles the vulnerable package but never calls it, the chain is pruned before any LLM is involved. Build-time tools are the canonical false positive pattern here: a code generator may carry a vulnerable template engine in its transitive graph, but that engine runs only during the build phase, never in the shipped runtime. Detecting that distinction deterministically eliminates a significant fraction of transitive CVEs before context assembly begins.

Exploitability requires reading the codebase. Pixee traces imports to call sites to data flow through first-party code, collecting typed evidence: a Usage record means the application calls the vulnerable API; a Missing record means it provably does not. Critically, a verdict of NOT_EXPLOITABLE requires positive evidence of absence: the absence of a finding is not the same as proof of safety. The final verdict for a transitive dependency is conjunctive: the chain must reach the vulnerable code, and first-party code must exercise that path. Either side ruling it out is decisive. This mirrors the SAST triage architecture exactly, applied to the dependency graph rather than the source file: deterministic prefilters run first, and LLM analysis runs only when deterministic resolution is insufficient.

Reachability is not exploitability. Most CVE scanners report reachability as if it were exploitability. A library that is reachable but never called with attacker-controlled input in a specific codebase is not a live vulnerability. It is a false positive that either triggers a wasted remediation or trains the engineering team to ignore findings. The cost of distinguishing these two correctly is justified by every unnecessary PR it prevents.

V · Seeing the whole picture

Seeing the Whole Picture: Context Is the Fix

The SARIF format gives you three things: a file path, a line number, and a rule ID. That is your starting point, and it is nowhere near enough to generate a correct fix.

To fix a real vulnerability correctly, you need the full taint flow from source to sink, the project's web framework (injection fixes look different in Django vs. Flask vs. FastAPI), the HTTP route that the vulnerable endpoint handles, neighboring tests you must not break, the project's build manifest for safe dependency resolution, and the feedback your team has given on prior analyses of this rule in this codebase.

None of this arrives in the SARIF file. Pixee's context assembly pipeline discovers it before a single fix token is generated.

What distinguishes this from a coding agent that starts fresh on every finding is what happens before the LLM call. A project context discovery pipeline has already scanned the repository at startup and identified framework patterns (Spring, Django, Rails, Express, and others), HTTP route registrations per language, manifest files, and README summaries. A knowledge base retrieval layer injects rule-specific remediation guidance authored by security engineers. By the time the LLM sees the prompt, it is not reverse-engineering the project from a file path and a line number. It is working from a pre-assembled brief.

The spelunker (an LLM-backed code exploration agent) handles the residual: targeted codebase search for context that static pre-scanning could not anticipate. Most coding agents start here; Pixee ends here.

Why this matters. Claude knows what SQL injection is. It does not know that your codebase uses a custom SafeDB.query() wrapper that already handles parameterization, unless Pixee tells it. Without that context, the model introduces unnecessary complexity and potentially breaks your abstraction layer.

VI · Before it ships

Before It Ships: Five Gates Between a Finding and a Committed Fix

Between a SARIF finding and a committed fix, Pixee runs a sequential validation pipeline. Every stage can reject. No stage silently swallows failures: every rejection produces a structured result that records exactly where and why the fix was stopped, giving the engineering team the information needed to distinguish a model failure from a system boundary.

Triage is the first filter: findings that are false positives never enter the pipeline at all. The stages that follow eliminate findings that are structurally unsuitable for automated remediation before an LLM call is made, apply a safety evaluation for cases where automated fixes carry known risk, run fix generation only on the population that clears those filters, and validate the output before anything is committed. Failures at each stage are categorized separately so regressions in one area are visible independently of the others.

The result is that the LLM operates on a small, high-quality population of findings, and every output it produces is checked before it ships.

VII · What the model doesn't know

What the Model Doesn't Know: A Knowledge Base That Isn't In the Model

Claude was trained on the internet. The internet contains a great deal of security information. But it does not contain your team's validated remediation patterns for CodeQL's java/log-injection rule as applied to your specific version of Logback, with the wrapping your team uses.

Pixee maintains a knowledge base of entries, each keyed by a composite identifier: (tool, rule_id). The guidance was authored by security engineers, not generated by an LLM. It is language-specific and framework-specific, retrieved via a dimensional fallback chain: framework-specific → language-specific → generic default.

A concrete example. The same SQL injection vulnerability class has separate KB entries for CodeQL Java, Semgrep Java, Semgrep Python, CodeQL JavaScript, and Snyk JavaScript. The tool matters as much as the vulnerability class. Each has subtly different detection semantics that affect what a correct fix looks like.

When Pixee receives a CodeQL Java finding for java/log-injection, it retrieves the Java-specific remediation guidance and injects it into the fix generation prompt. The LLM's task narrows from "figure out how to fix log injection in Java" to "apply these specific remediation patterns to this specific code location." That scoping produces significantly more consistent, correct outputs.

The KB system also separates fix guidance from triage guidance: they are distinct registries. This prevents the common failure mode where triage reasoning bleeds into fix generation reasoning and produces conservative over-fixes.

Layered on top of the static KB is an episodic memory component: prior feedback from analyses of this specific rule in this specific codebase is retrieved and injected alongside the static guidance. The two are not the same argument. A KB entry captures general remediation patterns authored by security engineers: how to fix SQL injection in Java, which sanitizer to prefer in Flask, what a safe parameterized query looks like in your ORM. Episodic memory captures your team's validated corrections: the last time this finding appeared, the developer said to use SafeDB.query() specifically, not the generic parameterized approach. Both are injected into the prompt. Neither is a substitute for the other.

Knowledge base. Pixee maintains a KB of entries authored by security researchers, not generated by an LLM. Each entry is keyed to a specific tool and rule, language-specific and framework-specific, and retrieved through a dimensional fallback chain. Two separate catalogs (triage and fix) prevent remediation reasoning from bleeding into triage verdicts.

VIII · The part nobody builds

The Part Nobody Builds in a Prototype: Findings Are Durable Entities, Not Scan Rows

Any team that builds a good scanner earns a new problem: managing results over time. Contrast Security built one of the most accurate SAST scanners on the market, and still had to invest heavily in Contrast Team Server as a separate vulnerability management layer. The scanner found things accurately; keeping track of what was found, what was fixed, what was reopened, and what was already being worked on required a different system entirely.

Any team that builds their own AI scanner hits the same wall. And any team that uses a scanner but wants to do something more than a pre-merge hardening step (backlog burndown, tracking existing security debt, avoiding duplicate PRs) needs persistence. The minimum viable version of this is answering one question: do I already have an open PR for this problem? At small scale, a coding agent can probably answer that by scanning open PRs. At the scale of thousands of findings across hundreds of repositories, it cannot. You need a finding identity layer.

Pixee's architecture separates two concepts that look identical from the outside but are fundamentally different:

Identity

Is this the same code location? A fingerprint can reliably answer this. It is stable across reformats, line shifts, and minor surrounding edits.

Validity

Does my prior verdict still hold? A fingerprint cannot answer this. If the code containing the vulnerability was significantly refactored, the old verdict may be wrong, even if the fingerprint still matches. The canonical failure case: Pixee marks a finding as a false positive because it detects a sanitizer. A developer removes the sanitizer. The fingerprint still matches. A hash-only reuse system carries the false positive forward for now-exploitable code, indefinitely.

Prior results are shown labeled "prior," not presented as fresh. The TTL is the self-correction mechanism. A system that reuses prior results without a TTL has no correction path. When wrong, it stays wrong.

Pixee distinguishes findings (durable entities with persistent identity) from observations (the row a specific scan run reported). A finding is the correlation of observations across scans. The display you see at any point in time is a query-time projection over the observation history: it is never stored as a fixed result. This architecture enables capabilities that are impossible without finding identity:

• Fan-out a fix across every branch where a finding appears, not just the current scan branch
• Refresh PR suggestions when a finding's scan ID changes between branches
• Baseline a repository: separate "existing backlog" from "net-new since this commit"
• Generate one fix for multiple SAST observations that represent the same logical vulnerability (e.g., all branches of a tainted data flow)
• Act on prior results within a freshness TTL to reduce re-analysis cost without silent stale-verdict risk

Acting on prior results efficiently requires solving a harder problem: when is it actually safe to serve a cached verdict? Fingerprint similarity is not sufficient, because the verdict may depend on context that extends well beyond the code location: imported libraries, configuration files, surrounding data flow. Attempting to predict what context matters before running analysis is an unsolvable problem.

Pixee's approach is to cache the exact model call, byte for byte. If the full prompt is identical to a prior call (same code, same context, same knowledge base guidance) the cached response is safe to return. This converts the prediction problem into enumeration. The failure modes are asymmetric by design: a cache miss costs the full price of a model call; serving a hit that shouldn't be served costs correctness. Exact-byte matching makes the latter impossible. The system degrades toward expensive-but-correct, never toward incorrect verdicts.

IX · Build vs. buy

The Build vs. Buy Calculation: The Real Cost of Rolling Your Own

A capable engineering team can build a prototype of any individual layer described above. The cost is not any single component: it is the integration, the maintenance, the compounding complexity, and the opportunity cost of the engineers doing it instead of building your product.

The table above captures initial engineering cost. The harder number is the ongoing one. "High maintenance" in the triage row means running benchmark suites after every model release to catch regressions, because a model upgrade that improves average quality can silently degrade accuracy on specific vulnerability classes. "High maintenance" in the knowledge base row means a security engineer reviewing every new rule variant, not delegating that judgment to the model. This is not a one-time build. It is a feedback loop that requires staffing.

The architecture in this article is reproducible. What is not reproducible is the accumulated judgment behind it: three years of benchmark runs against real findings, weekly conversations with security engineers at production customers, the in-house expertise required to know which model behaviors are regressions versus acceptable variance, and the institutional knowledge of which rule families are genuinely dangerous versus reliably noisy. Without those feedback loops, you are adjusting prompts in the dark and hoping the next model release does not break them. A team that builds this system without that feedback infrastructure has the recipe but not the ingredients. "High maintenance: high" in the table is a proxy for those loops. A system without them has no correction path when something goes wrong, and something always goes wrong.

The rough total for an initial implementation: 48–72 months of senior engineering time, spread across security engineers (for the KB and triage knowledge), platform engineers (for the enterprise plumbing), and ML engineers (for the LLM integration). This assumes you already know what you are building; the actual number for a team learning as they go is higher.

The real question. The question is not whether your team can build this. It is whether security tooling infrastructure is your team's core competency, and whether the engineers who would build it are better deployed on your actual product.

Consider also the ongoing cost of getting findings wrong. A false positive that ships as a pull request costs developer attention. A false negative that doesn't get caught costs incident response. A fix that introduces a regression costs a revert, a root cause analysis, and trust. The safety stack exists to bound these failure modes; the cost of not having it is measured in incidents, not engineering weeks.

What you are buying is not a wrapper around Claude. You are buying three years of engineering runway: the accumulated knowledge of what breaks in production and the infrastructure built to prevent it.

— · Conclusion

The LLM Is Not the Product

Any engineer with an API key can get an LLM to generate a plausible security fix. That is not the hard part, and it is not what Pixee built.

The hard part is the tool integrations, the triage logic, the context assembly, the five validation gates, the knowledge base, the finding continuity architecture, the concurrency controller, the cost attribution system, the per-repo configuration layer, the episodic memory with quality scoring, and the full-span observability that tells you when any of it goes wrong.

Pixee is not a wrapper around Claude. It is security remediation infrastructure that uses LLMs as one of its components. The LLMs are not the product; they are a capability. The engineering work around them is what makes that capability reliable, safe, and trustworthy enough to run on your production codebase.

If you want to build that infrastructure yourself, this article is a reasonably accurate map of the terrain. If you want to ship secure code instead, Pixee already built it. That is the Pixee Advantage.