The most useful sentence in this session is about where reliability is achievable: for workflows calling well-structured interfaces and producing code that can be checked, the path to reliability is faster (17:10).
The implication is the part worth extracting. There is a slower path, and it is the one everyone else is on.
Verifiability as the sorting criterion
The framing divides agent work into two categories that look similar and behave completely differently.
In the first, the output can be checked mechanically. Code compiles or does not. An interface call returns a valid response or an error. A test passes. Reliability here is an engineering problem: measure, find the failures, fix them.
In the second, correctness is a judgement. Whether a summary captured the right points, whether a recommendation was appropriate, whether the tone was suitable for the customer. No mechanical check exists, so every improvement claim rests on sampling and human review — which is slow, expensive, and does not scale with the number of workflows.
Most enterprise pilots are in the second category and are being managed with expectations from the first. The session names the distinction and then spends most of its time on the easier side.
The testing idea that generalises
The best illustration comes from browser automation: rather than examining the markup the way a conventional test would, the system looks at what the customer would actually observe — walking through checkout and booking flows, and identifying revenue leakage (25:49).
That reframing is more portable than the specific application. A conventional interface test asserts on structure, which is a proxy for what the user sees and breaks whenever the structure changes for reasons that do not matter. Asserting on the observable outcome tests the thing you care about.
The revenue-leakage framing is the sharper part. It defines success in the business's own terms rather than in engineering ones, which means the test measures whether the flow works commercially rather than whether the elements are present. That also makes it verifiable in the good sense — money either arrives or it does not.
What the capability numbers actually change
Two figures are offered as headline capabilities and deserve different treatment.
The demonstration workflow makes roughly fifteen to twenty tool calls to understand an issue, locate the changes needed, and make them (12:29). That count is the honest measure of what agentic means — not one clever response but a chain of twenty decisions, each of which can be wrong and each of which the next depends on.
The context length of up to a million tokens, described as enabling something like a hundred thousand lines of code, four-hundred-page documents, or ninety-minute videos (13:56), is the more heavily promoted number and the more ambiguous one. Capacity to accept that much input is not the same as attending to it well, and the session does not distinguish them.
The two figures interact in a way that is not discussed. Twenty sequential tool calls each returning substantial output is how a context fills, which means large windows are less about ingesting one enormous document and more about surviving a long chain of moderate ones.
The layer they are right to emphasise
The architectural point is that models sit on top of infrastructure — tool orchestration, memory for managing context, and observability for monitoring behaviour (4:11) — and that not every model handles agentic work equally well, since the requirement is decomposing a task, planning, and calling tools reliably (4:42).
The progression they use to explain this is clarifying. Coding moves from explaining an error, to generating an artefact, to writing code, running tests and deploying a fix (2:51). Support moves from summarising issues for a human, to a first-line interface, to something that resolves the issue (3:18).
Each step transfers a decision from the person to the system. The final step in each is where the verification question becomes unavoidable — and where the earlier distinction decides whether you are on the fast path or the slow one.
关键数据
演讲章节
关键要点
- 01
Where interfaces are structured and output is verifiable, the path to reliability is faster — implying a slower path everywhere else. 17:10
- 02
Browser testing examines what a customer would observe rather than the markup, and defines success as revenue leakage rather than element presence. 25:49
- 03
The demonstrated workflow makes fifteen to twenty tool calls, which is the honest measure of what an agentic chain involves. 12:29
- 04
Context of up to a million tokens is offered as capacity, without distinguishing capacity from attention quality. 13:56
- 05
Not all models handle agentic work equally, since the requirement is decomposing a task, planning, and calling tools reliably. 4:42
提及的实体
相关演讲
The practical counterpart to the argument made elsewhere this season that specification is what contains model entropy. Raval and Harris name the failure they are addressing precisely — a prompt-and-pray loop in which working code arrives with no record of what the model assumed, which requirements were fuzzy, what design was chosen or why, leaving nothing to review and nothing to iterate against when a defect surfaces months later. Their answer is three committed markdown artefacts: requirements written in a structured requirements syntax with acceptance criteria attached to each user story, a design document carrying technical decisions together with the reasoning behind them, and a task list whose entries cite the requirement numbers they satisfy. The traceability is the point — a reviewer questioning a decision in a pull request can follow it back through the task to the design to the requirement, all in the same repository. Notably they keep the human between each phase rather than after it, with the agent surfacing ambiguity as questions before proceeding.
Drawn from a year of engagements with more than a hundred companies, this is the most direct challenge in the season's programme to the assumption that faster code generation produces faster delivery. Mishra and Raja open with external evidence rather than their own: an industry study putting realised velocity gains in the ten to fifteen per cent range, and a controlled experiment in which developers using AI estimated themselves roughly a fifth more productive while measurement showed them a fifth slower. Their diagnosis is that both prevailing working styles fail for opposite reasons. Handing an ambiguous problem to an agent and awaiting a finished result produces a volume of code the developer must nonetheless sign for and cannot confidently review, so it stalls before production. The senior engineer's alternative — decomposing the work personally and inserting AI into narrow slots — keeps the intellectual load exactly where it was, and leaves the surrounding process untouched, so hours saved in editing are consumed by the meetings that process still requires.
The most concrete attempt this conference season to answer a question the agentic coding sessions mostly leave open: if commit counts and hours saved are the wrong measures, what replaces them? Otto's account is unusually specific about why the obvious alternative fails — summing the small time savings a platform team delivers produces figures exceeding a hundred per cent of a developer's time, and a minute returned is not code in production. Their replacement borrows from Amazon's retail supply chain, where cost to serve measures what it takes to place a package on a doorstep, and applies the same shape to software: total cost divided by units of delivery, with the unit chosen to fit the team. The supporting research is the more quotable finding — across tens of thousands of developers over five years, individual velocity reverts to the team's mean, making team velocity the strongest predictor of both individual output and perceived productivity, which is the empirical case against measuring individuals at all.
Brooker builds the definition from the bottom up rather than asserting it, using a deliberately absurd arithmetic task to separate three categories: what a model computes reliably as a fixed function of its input, what merely needs to arrive in the system prompt, and what genuinely requires reaching into the world. Only the third category justifies a tool, and the distinction matters because most production disappointment comes from tools built for the first two. His working definition follows — a system given a goal that loops between inference and tool calls until it reaches one — with the observation that modern agents increasingly embed code in their definitions, not for expressiveness but because replacing inference steps with deterministic code improves reliability while lowering both latency and cost. The remainder covers what production actually demands around that loop: somewhere to run, memory that persists preferences, a gateway to internal and external tools, evaluation, and formal methods applied to policy.
The framing statistic is organisational rather than technical: around eighty per cent of organisations expected to have platform engineering teams going into 2026, up from about forty-five per cent a couple of years earlier. The interesting part is the doubling. The problem described is teams solving the same problems separately, producing inconsistency and redundancy — dangerous not because of duplicated effort but because each independent solution has its own security properties, and the organisation's real posture is the weakest rather than the average. The most valuable content is that two financial services organisations went in diametrically opposite directions on workload identity and both are described as working, which implies the choice is determined by context rather than by a general answer. The honest note follows immediately: even with standardised patterns the result remains fragmented.
The line that explains this session comes from the customer in the last ten minutes: they are preparing for a world where metadata is how agent-based systems find the data they need and access it through the controls being built. That relocates a function — governance has spent two decades as compliance activity describing data that people locate by other means, and if agents navigate by the catalogue then the catalogue stops describing the access path and becomes it. An incomplete catalogue is a documentation problem when humans can ask a colleague; an agent has no such workaround. The most honest moment addresses the perennial failure that rules get written and ignored, with enforcement rather than publication as the argument. Generated descriptions and greyed-out classification suggestions divide the labour correctly, keeping a person accountable while removing the burden of finding candidates.