Twenty-nine minutes after asking whether anyone really wants to hand the keys to a production environment to an agent (8:04), the demonstration disables tool confirmation entirely — allowing any tool without a human approval step (37:17).
That is not hypocrisy. It is a live demo, and asking the audience to approve every call would be unwatchable. But the sequence is instructive, because the gap between the stated concern and the working configuration is exactly the gap every team will encounter, and nothing in the session addresses how to close it.
What the agents are actually allowed to do
The scope makes the concern concrete. The example prompt asks the system to find failed login attempts exceeding a threshold in the last hour, analyse the source addresses and affected accounts, then block the suspicious addresses and create snapshots (10:20).
That is not a read-only investigation. Blocking network traffic and creating snapshots are write operations on production infrastructure, initiated by a chain of reasoning that began with a sentence typed in English. The examples mentioned earlier — stopping compute instances, taking forensic snapshots of storage volumes (8:04) — are in the same category.
The design response is decomposition. A coordinating agent delegates to specialists and synthesises their answers (8:58), with the security specialist finding the source addresses across regions before an operations agent acts on them (10:47).
That split is worth noticing for a reason the session does not state. The investigating agent and the acting agent are different components, which means the permission boundary can sit between them. Everything before the handover is read-only; everything after it is not. Whether anyone configures it that way is a separate matter, but the architecture allows the distinction that the confirmation prompt was standing in for.
The construction is genuinely simple
The code walkthrough is the strongest part, and its message is that this is not hard to build.
An agent is defined by a model, a set of tools and a prompt (26:37). Selecting the model takes one line (27:31). The iteration logic that previously had to be written by hand is now a call (14:48). One presenter notes he had not written Python since university and picked it up quickly (12:06).
Each tool is small and specific. One runs a query against the log store and returns results, a table, or an error (29:17). Another lists available tables so queries can be constructed (29:45). An agent then becomes a tool for another agent, so the coordinating agent holds the specialists the way it holds any other capability.
The recommendation to start with a straightforward case in order to learn what works (23:54) is sound and slightly undersells the difficulty. Defining a tool is easy. Deciding which tools an agent should have, in an environment where the wrong combination stops production infrastructure, is the part that takes judgement.
What the output looks like
The generated report is the most useful artefact shown: 859 critical findings across storage buckets, identity users, compute instances and modified security groups, with an assessment of risk, an attack analysis identifying source addresses and the compromised account, and a set of recommended actions ordered by urgency (38:49).
That format is genuinely valuable and is also the place where the trust question resurfaces in a new form.
A report of 859 findings recommending immediate credential rotation is a document nobody can verify by hand. The value is that it aggregates what a human could not read. The cost is that the aggregation is the only thing standing between the analyst and the raw logs, and no one is going to check its arithmetic.
Which is the pattern underneath the whole session. Delegating investigation to an agent is safe in the narrow sense that reading logs breaks nothing. It is unsafe in the broader sense that the conclusions become the basis for actions, and the actions here include blocking traffic and stopping instances on production.
The presenters raise the right question at the eight-minute mark. The honest answer is that their architecture makes a good answer possible and their demonstration does not give one.
关键数据
- 859
- critical findings in the generated security report across storage, identity, compute and network resources 38:49
演讲章节
关键要点
- 01
The session names the obvious concern — handing production keys to an agent — early and directly. 8:04
- 02
The demonstration then bypasses tool confirmation so any tool can run without a human approval step. 37:17
- 03
The example prompt blocks suspicious addresses and creates snapshots, which are write operations rather than investigation. 10:20
- 04
Investigation and action live in separate agents, which is where a permission boundary can be placed. 10:47
- 05
The generated report aggregates 859 critical findings into ranked recommendations that no analyst can verify by hand. 38:49
提及的实体
相关演讲
The practical counterpart to the argument made elsewhere this season that specification is what contains model entropy. Raval and Harris name the failure they are addressing precisely — a prompt-and-pray loop in which working code arrives with no record of what the model assumed, which requirements were fuzzy, what design was chosen or why, leaving nothing to review and nothing to iterate against when a defect surfaces months later. Their answer is three committed markdown artefacts: requirements written in a structured requirements syntax with acceptance criteria attached to each user story, a design document carrying technical decisions together with the reasoning behind them, and a task list whose entries cite the requirement numbers they satisfy. The traceability is the point — a reviewer questioning a decision in a pull request can follow it back through the task to the design to the requirement, all in the same repository. Notably they keep the human between each phase rather than after it, with the agent surfacing ambiguity as questions before proceeding.
Drawn from a year of engagements with more than a hundred companies, this is the most direct challenge in the season's programme to the assumption that faster code generation produces faster delivery. Mishra and Raja open with external evidence rather than their own: an industry study putting realised velocity gains in the ten to fifteen per cent range, and a controlled experiment in which developers using AI estimated themselves roughly a fifth more productive while measurement showed them a fifth slower. Their diagnosis is that both prevailing working styles fail for opposite reasons. Handing an ambiguous problem to an agent and awaiting a finished result produces a volume of code the developer must nonetheless sign for and cannot confidently review, so it stalls before production. The senior engineer's alternative — decomposing the work personally and inserting AI into narrow slots — keeps the intellectual load exactly where it was, and leaves the surrounding process untouched, so hours saved in editing are consumed by the meetings that process still requires.
Brooker builds the definition from the bottom up rather than asserting it, using a deliberately absurd arithmetic task to separate three categories: what a model computes reliably as a fixed function of its input, what merely needs to arrive in the system prompt, and what genuinely requires reaching into the world. Only the third category justifies a tool, and the distinction matters because most production disappointment comes from tools built for the first two. His working definition follows — a system given a goal that loops between inference and tool calls until it reaches one — with the observation that modern agents increasingly embed code in their definitions, not for expressiveness but because replacing inference steps with deterministic code improves reliability while lowering both latency and cost. The remainder covers what production actually demands around that loop: somewhere to run, memory that persists preferences, a gateway to internal and external tools, evaluation, and formal methods applied to policy.
The most concrete attempt this conference season to answer a question the agentic coding sessions mostly leave open: if commit counts and hours saved are the wrong measures, what replaces them? Otto's account is unusually specific about why the obvious alternative fails — summing the small time savings a platform team delivers produces figures exceeding a hundred per cent of a developer's time, and a minute returned is not code in production. Their replacement borrows from Amazon's retail supply chain, where cost to serve measures what it takes to place a package on a doorstep, and applies the same shape to software: total cost divided by units of delivery, with the unit chosen to fit the team. The supporting research is the more quotable finding — across tens of thousands of developers over five years, individual velocity reverts to the team's mean, making team velocity the strongest predictor of both individual output and perceived productivity, which is the empirical case against measuring individuals at all.
The framing statistic is organisational rather than technical: around eighty per cent of organisations expected to have platform engineering teams going into 2026, up from about forty-five per cent a couple of years earlier. The interesting part is the doubling. The problem described is teams solving the same problems separately, producing inconsistency and redundancy — dangerous not because of duplicated effort but because each independent solution has its own security properties, and the organisation's real posture is the weakest rather than the average. The most valuable content is that two financial services organisations went in diametrically opposite directions on workload identity and both are described as working, which implies the choice is determined by context rather than by a general answer. The honest note follows immediately: even with standardised patterns the result remains fragmented.
The hardest problem in this session has nothing to do with machine learning: you cannot reliably tell who owns a machine. Unlike vehicles, which carry an identification number and go through state registration, heavy equipment has no equivalent — someone can simply assert ownership. Everything the connected-product strategy promises depends on solving that, because every step after fault detection requires knowing who to contact. The estate explains why it was not solved earlier: millions of machines with 1.5 million connected, and around 160 dealers who are independent businesses with their own systems, holding the service history that makes telemetry meaningful. The prior state is described directly — multiple accumulated platforms, and dealers confused because the same question returned different answers, which destroys trust in all of them including the correct ones.