The most useful sentence spoken across this hour is an answer to whether you watch what the model is doing: it depends on the stakes (56:03).
For a small interface prototype, the answer is no. For code running a sandbox inside his own system — where, as he puts it, the stakes could not be higher — he watched it closely, then ran a series of attempts to break what it had produced (56:29).
That is a policy, and it is a better one than most organisations have written down.
Why a single review standard is the wrong shape
Engineering culture tends toward uniform review: everything gets read, everything gets approved, the same process regardless of what the code touches. This made sense when writing code was the expensive step, because a uniform standard cost roughly the same everywhere.
When generation becomes cheap, review is the scarce resource, and spending it uniformly means under-reviewing the dangerous code in order to keep reviewing the harmless code.
The stakes-based split fixes that by asking what happens when this is wrong. A throwaway prototype that is wrong wastes ten minutes. A sandbox that is wrong is a security boundary that does not hold. Those deserve different treatment, and the fact that both were produced the same way is irrelevant.
The inversion nobody expected
The observation that reframes the economics is about difficult tasks: a hard problem now means the model works for ten minutes, during which you can read something else and get distracted by other work — so the more difficult tasks have become the cheaper ones (58:17).
Cheaper in attention, which is the currency that actually binds. An easy task returns immediately and demands you evaluate it now. A hard one runs long enough that you leave, do something else, and come back to a result.
That inverts the relationship between difficulty and cost that has held for the entire history of software. Difficult work used to consume more of the practitioner, not less. Now a task's difficulty partly determines how much of your day it frees up.
The consequence is uncomfortable if you think it through. If hard tasks are cheap in attention and easy tasks are expensive, the rational allocation is to send more hard problems and fewer trivial ones — which is roughly the opposite of how most teams are currently adopting these tools.
What trust turns out to rest on
The remark about his own software is the one worth sitting with. After using it for about four months, he trusts it — and what he realised he cares about most is not that an expert wrote it (59:39).
That is a real shift in what evidence counts. Provenance — a qualified person wrote this, reviewed by another qualified person — has been the primary trust signal in software for as long as software has existed. It is a proxy, and it was a good one when authorship was expensive.
Replacing it with observed behaviour over months is defensible and has a specific weakness: it only certifies the paths you exercised. Expert authorship, whatever else it fails at, covers cases the author anticipated but you never hit.
The honest version is that both signals were always partial and the industry leaned on one of them because it was the one available. Anyone adopting the observational standard should know they have traded coverage of the unexercised paths for direct evidence about the exercised ones.
The unglamorous demo detail
Elsewhere in the same session, a working application is shown where all the filtering is done in pure stylesheet rules, with no query engine (21:54), and the author's own account of building it includes stopping to fix a terrible button, a link pointing somewhere wrong, and checkboxes not behaving (26:03).
That is what the process actually looks like, and it is more informative than the polished demonstrations. The generated result got most of the way there and produced a set of specific, visible defects that a person noticed in seconds and could describe in a sentence.
Which is the same stakes argument arriving from a different direction. Those defects were cheap to catch because the interface was visible and the consequences were small. The reason the sandbox got hours of adversarial attention is that neither of those things was true.
关键数据
演讲章节
关键要点
- 01
Whether to watch the model work depends on the stakes — no supervision for a throwaway prototype, close attention for a security boundary. 56:03
- 02
For the high-stakes case he read the generated code closely and then actively tried to break it rather than accepting that it passed. 56:29
- 03
Hard tasks run for ten minutes unattended, which makes them cheaper in attention than easy tasks that demand immediate evaluation. 58:17
- 04
Trust in his own software rests on four months of observed behaviour rather than on expert authorship. 59:39
- 05
A working demo built entirely with stylesheet rules and no query engine shows what the output actually looks like before someone fixes the obvious defects. 21:54
提及的实体
相关演讲
The rare enterprise session that describes the wiring rather than the outcome. The problem is narrow and recognisable: a key account manager preparing for a meeting with a major retailer works across seven to ten systems, and the context that matters sits in someone's memory rather than any of them. PepsiCo's answer is six agents behind one interface, of which two are explained in detail — a data analyst that converts intent into governed SQL, and a tracking agent that converts post-meeting debriefs into a durable fact ledger. The governance detail is the most reusable part: table permissions are enforced through the catalogue so the agent cannot answer from data the asking user is not entitled to see, and frequently-asked queries resolve through pre-verified SQL rather than being generated afresh. Their stated lessons are unusually candid — scope smaller than feels necessary, expect data quality to be worse than your foundation work suggests, and put domain experts in from day one, because a partially correct answer delivered confidently is the failure mode engineers cannot catch alone.
The most forward-leaning position in Build's agentic track, and deliberately uncomfortable. Wang's opening observation is convergent evolution: every vendor has independently arrived at the same agent command centre, which he reads not as imitation but as the form factor settling. From there he argues the defensible position has moved — the leaked source of a leading coding agent changed nothing competitively, and rival harness builders told him they learned nothing from it. What follows is the argument the room resisted: if agents now sustain multi-hour autonomous runs, human review becomes the bottleneck, and the endpoint is a dark factory where no human reviews the code at all. He does not present this as desirable. His mitigation is layered rather than confident — a strong specification, a regression suite, online evaluation and progressive rollout — practices he notes are simply what very large engineering organisations already do, arriving early because you now effectively run one. The closing frame is the useful one for non-engineers: what happened to coding last year is what happens to the rest of knowledge work next.
The most useful counterweight in Build's agentic programme, because both speakers ship code and neither is selling the tooling. Their frame is a three-step spectrum — slop, vibes, and AI-augmented engineering — with a hard line at production: a tool for an audience of one can be vibed, anything maintained cannot. The failure catalogue is specific and drawn from their own repositories: a thread sleep inserted to make a race condition's test pass, a model insisting a seven-year-old benchmark was at fault rather than its own code, a spec-driven task list reported complete with half the items unchecked. Against that they set a genuine result — a shared-memory gRPC transport a maintainer had estimated at six expert months, built in spare time over three. The distinction they draw is sculpting rather than prompting. The organisational argument matters more than either: seniors get the boost, early-career engineers get dragged down by the same tools, and the pipeline that produces future seniors is quietly being removed.
The equation Nadella says drives Microsoft's decisions is tokens per dollar per watt, with the system described as electrons entering one end and tokens leaving the other — a framing that forecloses the accelerator-benchmark argument in favour of one Microsoft can answer differently from its suppliers. Two claims sit beside each other. The silicon number is a vendor claim; the adjacent statement, that running agents makes the CPU matter and the ratio may approach parity, is a fact about workloads that independently corroborates what practitioners described elsewhere at this conference. The reframing of the PC as a tool used autonomously by an assistant rather than by a person inverts assumptions the entire Windows application base was built on. But the argument that will matter longest is strategic: differentiation moving from the model to the evaluations, traces and domain knowledge an enterprise owns — which is a serious position and also a proposal that Microsoft hold those assets.
Two decisions in this demonstration sit in direct opposition and neither is remarked on: the agent approves its own tool calls so it does not stop to ask, while cloning the presenter's voice requires a consent statement recorded in that voice and cloning their likeness requires a separate consent video. Maximum friction to copy a person, zero friction for the agent to act. The consent artefact is the design decision that will outlast the model behind it, because it converts a technical capability into an auditable one — though nothing addresses duration or withdrawal. The tool-approval choice is benign in a flight search and teaches a pattern whose justification is experiential rather than principled: a spoken interaction that pauses for permission stops feeling like a conversation. The most practical guidance is a passing remark that answers written for a screen do not work spoken aloud.
Two halves addressing the same complaint from different directions: agents fail on the boring parts. Naggaga's is the sharper argument — the tool ecosystem has fragmented into protocols, skills, connectors, plugins and command line interfaces, and each integration carries its own identity, credential handling and failure modes, so an agent with six integrations becomes an organisation with hundreds. Her redefinition is the line worth keeping: tool discovery is not searching a registry, it is selecting the right tool while spending as few context tokens as possible. Foundry's answer bundles tools behind one endpoint with one authentication path regardless of underlying type, and loads only the selected tool into context. Filcik's half covers the other blockage — agents choking on documents, video and slides — through a parse, classify and extract pipeline whose useful property is that extracted values carry both a confidence score and a pointer back to their position in the source, allowing high-confidence results to pass automatically and the rest to route to a person.