The number in this session's title is a triage improvement. The story underneath it is that the queue being triaged should never have grown that large, and the thing that fixed the root cause was not AI.
The dysfunction they describe
The diagnosis is stated with unusual candour: it was easier to obtain an exception than to fix the underlying problem, and part of the reason was that application teams did not know how to fix certain vulnerabilities (24:35).
The follow-up matters. These were not bad developers — security simply was not their area, and they did not know how to do particular things (25:58).
That combination produces a specific and self-reinforcing failure. A scanner reports findings. Developers who cannot act on them request exceptions. Exceptions are granted because the alternative is blocking a release. The security team then spends its time processing exception requests rather than reducing risk, and the scanner keeps producing findings at the same rate.
Adding a better scanner to that system makes it worse. More findings enter a pipeline whose throughput is limited by developer capability, and the exception queue grows.
What the automation actually did
Against that background, the reported improvement is real and narrow: average time to review a false positive fell from thirty days to thirteen (22:11).
Thirty days to determine that a finding was not a finding is the number that should shock people, and it is a direct measure of the queue problem. Nothing about the underlying risk changed during those thirty days. The organisation simply did not have the capacity to look.
Halving that is worth having. It is also, precisely, a faster way to process the symptom. The finding still had to be reviewed, and the reason there were so many to review is unchanged by the tooling.
The intervention that addressed the cause
The durable change described is a security champions programme: training sessions, a hands-on component, and a tiered structure (25:58, 30:09).
The design is better than most such programmes. They describe balancing formal instruction against real work — in some cases skipping the exercise entirely and using an application that actually had a problem as the live example (35:18) — and they tested comprehension at the end, with over eighty-six people across fifteen sessions (34:50).
The second tier is the part worth copying. Its function is verification: preventing a situation where a first-tier reviewer marks everything as a false positive, on a trust-but-verify basis (32:31).
That is a sharp piece of process design, because it anticipates the incentive the programme creates. Giving application teams authority to dismiss findings speeds things up and gives them a reason to dismiss aggressively. A verification tier makes the delegation safe without taking it back.
The economics that justify all of it
The cost argument underpinning the shift-left approach is stark: a vulnerability found after user acceptance testing costs at least eighteen times more to fix, and one found at launch approaches sixty-four times (13:03).
Those multipliers are widely cited and worth treating as an order of magnitude rather than a measurement. Even discounted heavily, they explain the entire strategy. If late fixes cost that much more, then developer capability — the thing that determines whether an issue is fixed early or deferred — dominates every other variable, including detection quality.
Which is why the ordering in this session is the right way round even though the title inverts it. The champions programme changed what arrives in the queue. The automation changed how fast the queue drains. Only one of those addresses why the queue existed.
What they leave unexamined
The unexplored risk is what happens to the second tier over time. Verification works while the reviewers have both the expertise and the time. The programme's own success — fewer findings, faster reviews, less friction — reduces the pressure that justified staffing it.
The session does not say how the verification layer is sustained once the crisis it was built for has passed. That is the question every organisation copying this will face in its second year, and it is not answered here.
关键数据
演讲章节
关键要点
- 01
Obtaining an exception was easier than fixing the problem, partly because application teams did not know how to fix certain vulnerabilities. 24:35
- 02
Average false-positive review time fell from thirty days to thirteen, which halves the symptom rather than reducing what enters the queue. 22:11
- 03
A vulnerability found after acceptance testing costs at least eighteen times more to fix, and at launch approaches sixty-four times. 13:03
- 04
The second champions tier exists to verify the first, preventing a reviewer from marking everything a false positive. 32:31
- 05
Training balanced formal instruction against fixing a real application live, with over eighty-six participants across fifteen sessions. 34:50
提及的实体
相关演讲
The practical counterpart to the argument made elsewhere this season that specification is what contains model entropy. Raval and Harris name the failure they are addressing precisely — a prompt-and-pray loop in which working code arrives with no record of what the model assumed, which requirements were fuzzy, what design was chosen or why, leaving nothing to review and nothing to iterate against when a defect surfaces months later. Their answer is three committed markdown artefacts: requirements written in a structured requirements syntax with acceptance criteria attached to each user story, a design document carrying technical decisions together with the reasoning behind them, and a task list whose entries cite the requirement numbers they satisfy. The traceability is the point — a reviewer questioning a decision in a pull request can follow it back through the task to the design to the requirement, all in the same repository. Notably they keep the human between each phase rather than after it, with the agent surfacing ambiguity as questions before proceeding.
Drawn from a year of engagements with more than a hundred companies, this is the most direct challenge in the season's programme to the assumption that faster code generation produces faster delivery. Mishra and Raja open with external evidence rather than their own: an industry study putting realised velocity gains in the ten to fifteen per cent range, and a controlled experiment in which developers using AI estimated themselves roughly a fifth more productive while measurement showed them a fifth slower. Their diagnosis is that both prevailing working styles fail for opposite reasons. Handing an ambiguous problem to an agent and awaiting a finished result produces a volume of code the developer must nonetheless sign for and cannot confidently review, so it stalls before production. The senior engineer's alternative — decomposing the work personally and inserting AI into narrow slots — keeps the intellectual load exactly where it was, and leaves the surrounding process untouched, so hours saved in editing are consumed by the meetings that process still requires.
Brooker builds the definition from the bottom up rather than asserting it, using a deliberately absurd arithmetic task to separate three categories: what a model computes reliably as a fixed function of its input, what merely needs to arrive in the system prompt, and what genuinely requires reaching into the world. Only the third category justifies a tool, and the distinction matters because most production disappointment comes from tools built for the first two. His working definition follows — a system given a goal that loops between inference and tool calls until it reaches one — with the observation that modern agents increasingly embed code in their definitions, not for expressiveness but because replacing inference steps with deterministic code improves reliability while lowering both latency and cost. The remainder covers what production actually demands around that loop: somewhere to run, memory that persists preferences, a gateway to internal and external tools, evaluation, and formal methods applied to policy.
The most concrete attempt this conference season to answer a question the agentic coding sessions mostly leave open: if commit counts and hours saved are the wrong measures, what replaces them? Otto's account is unusually specific about why the obvious alternative fails — summing the small time savings a platform team delivers produces figures exceeding a hundred per cent of a developer's time, and a minute returned is not code in production. Their replacement borrows from Amazon's retail supply chain, where cost to serve measures what it takes to place a package on a doorstep, and applies the same shape to software: total cost divided by units of delivery, with the unit chosen to fit the team. The supporting research is the more quotable finding — across tens of thousands of developers over five years, individual velocity reverts to the team's mean, making team velocity the strongest predictor of both individual output and perceived productivity, which is the empirical case against measuring individuals at all.
The framing statistic is organisational rather than technical: around eighty per cent of organisations expected to have platform engineering teams going into 2026, up from about forty-five per cent a couple of years earlier. The interesting part is the doubling. The problem described is teams solving the same problems separately, producing inconsistency and redundancy — dangerous not because of duplicated effort but because each independent solution has its own security properties, and the organisation's real posture is the weakest rather than the average. The most valuable content is that two financial services organisations went in diametrically opposite directions on workload identity and both are described as working, which implies the choice is determined by context rather than by a general answer. The honest note follows immediately: even with standardised patterns the result remains fragmented.
The line that explains this session comes from the customer in the last ten minutes: they are preparing for a world where metadata is how agent-based systems find the data they need and access it through the controls being built. That relocates a function — governance has spent two decades as compliance activity describing data that people locate by other means, and if agents navigate by the catalogue then the catalogue stops describing the access path and becomes it. An incomplete catalogue is a documentation problem when humans can ask a colleague; an agent has no such workaround. The most honest moment addresses the perennial failure that rules get written and ignored, with enforcement rather than publication as the argument. Generated descriptions and greyed-out classification suggestions divide the labour correctly, keeping a person accountable while removing the burden of finding candidates.