议题

Engineering Productivity

How organisations measure the output and experience of software teams, and how AI tooling changes what those measures mean.

9
演讲
15
嘉宾
7
机构

最新演讲

Three Times the Issues Is What a Successful Agent Rollout Looks Like
Three Times the Issues Is What a Successful Agent Rollout Looks Like

The chart organising this session is not about productivity. It shows roughly three times more issues arriving and considerably more open pull requests since January, which is the honest shape of a successful agent rollout and almost never the number presented. The team's response was to move from monthly to weekly releases, a change forced by throughput rather than chosen for confidence. Their code-survival metric — the share of a model's output that actually gets committed — is unusually well chosen because it is measured after the human decision and so cannot be inflated by generating more. The most transferable practice is running the editor with and without a change and attaching the comparison to the pull request, which turns the pull request into the specification. They also state plainly that running AI over community-submitted issues means running it on untrusted input, without detailing the mitigation.

Microsoft Build

The Review Loop That Never Terminates
The Review Loop That Never Terminates

The observation that should change how you work is about a review loop that does not terminate: ask for a review, get findings, fix them, ask again, get new findings, repeated through a whole day until the speaker concludes he has become the machine in the loop. That is better read as a property of the arrangement than as model unreliability — traditional review terminates because a human gets bored or runs out of time, and removing the human cost removes the stopping condition with it. The organising idea of the talk is that your job is no longer building software faster but helping your agent build software faster, which reframes a portfolio of small side tools as accumulated leverage rather than failed products. His review-depth policy scales with blast radius rather than applying uniformly, and his renaming of the pull request as a prompt request inverts thirty years of open-source etiquette about what counts as a generous contribution.

Microsoft Build

Nobody Is Worried About Whether It Builds
Nobody Is Worried About Whether It Builds

The most revealing phrase here is a design goal rather than a feature: teaching not just how to use the packages but how to use them effectively, building applications that have good taste, because a merely functional dashboard is not what anyone wants. That is an admission about where the constraint has moved. Nobody in the demonstration worries about whether the agent can produce a working dashboard; the worry is whether it will produce a good one. For thirty years the limiting factor was construction, and the demonstration makes the point without meaning to by skipping past the building to the finished result. What remains is judgement about what should be built, which the tooling does not provide and which cannot be verified by a test. The narrative is honest about the consequence: requests arrive faster because the cost of asking fell along with the cost of building.

Microsoft Build

Hard Tasks Became the Cheap Ones
Hard Tasks Became the Cheap Ones

The most useful sentence across this hour answers whether you watch what the model is doing: it depends on the stakes. A small interface prototype gets no supervision; code running a sandbox inside his own system got close attention and a series of attempts to break it. That is a better review policy than most organisations have written down, because when generation becomes cheap, review is the scarce resource and spending it uniformly under-reviews the dangerous code. The observation that reframes the economics is that a hard problem means the model works for ten minutes while you do something else, so difficult tasks have become the cheaper ones in attention — inverting a relationship that has held for the entire history of software. The remark about trusting his own software after four months of use, rather than because an expert wrote it, is a real shift in what evidence counts.

Microsoft Build

Write Instructions the Agent Can Skip
Write Instructions the Agent Can Skip

The transferable content here is not the demonstrations but a set of rules for writing instructions an agent will follow. Supporting files are provided alongside a main instruction file, with explicit statements about when and how each applies, so that material loads at the start but is used only for specific tasks — and so the agent can skip a step entirely when it is unnecessary. One subtask was extracted into its own file specifically so it could be skipped when the user has already named the target, which is instruction design as software design. The practical warning is that customisations should be extracted and renamed, because a bulk update of the shared instructions can silently overwrite them. Two unremarked demo moments establish the real working rhythm: start the long-running task first and fill the time with something else.

Google I/O

You Cannot Measure the Impact of AI Tooling Without a Baseline You Never Built
You Cannot Measure the Impact of AI Tooling Without a Baseline You Never Built

Cudby opens by asking what baseline exists before any AI tooling is deployed, and answers it himself: for most organisations, none. That absence is what makes impact reporting unreadable, and the session's argument is about sequencing rather than instrumentation. Adoption and engagement are leading indicators; financial return is a lagging one, and reading the lagging figure without the leading ones produces a number nobody can act on — a poor result might mean the tools do not work, or that nobody uses them, or that they are pointed at the wrong tasks. The finding drawn from this year's industry research sets the expectation for anyone planning a rollout: structured enablement determines outcomes, and switching the tools on and hoping does not work. Placed beside Amazon's own measurement work at the same conference, the notable thing is how careful the category has become about what it claims.

AWS re:Invent

The 10-15% Reality Check: Why AI Coding Gains Stay Small (re:Invent 2025)
The 10-15% Reality Check: Why AI Coding Gains Stay Small (re:Invent 2025)

Drawn from a year of engagements with more than a hundred companies, this is the most direct challenge in the season's programme to the assumption that faster code generation produces faster delivery. Mishra and Raja open with external evidence rather than their own: an industry study putting realised velocity gains in the ten to fifteen per cent range, and a controlled experiment in which developers using AI estimated themselves roughly a fifth more productive while measurement showed them a fifth slower. Their diagnosis is that both prevailing working styles fail for opposite reasons. Handing an ambiguous problem to an agent and awaiting a finished result produces a volume of code the developer must nonetheless sign for and cannot confidently review, so it stalls before production. The senior engineer's alternative — decomposing the work personally and inserting AI into narrow slots — keeps the intellectual load exactly where it was, and leaves the surrounding process untouched, so hours saved in editing are consumed by the meetings that process still requires.

AWS re:Invent

Amazon's Answer to the Productivity Metrics Problem: Cost to Serve Software
Amazon's Answer to the Productivity Metrics Problem: Cost to Serve Software

The most concrete attempt this conference season to answer a question the agentic coding sessions mostly leave open: if commit counts and hours saved are the wrong measures, what replaces them? Otto's account is unusually specific about why the obvious alternative fails — summing the small time savings a platform team delivers produces figures exceeding a hundred per cent of a developer's time, and a minute returned is not code in production. Their replacement borrows from Amazon's retail supply chain, where cost to serve measures what it takes to place a package on a doorstep, and applies the same shape to software: total cost divided by units of delivery, with the unit chosen to fit the team. The supporting research is the more quotable finding — across tens of thousands of developers over five years, individual velocity reverts to the team's mean, making team velocity the strongest predictor of both individual output and perceived productivity, which is the empirical case against measuring individuals at all.

AWS re:Invent

Bottlenecks Hide Other Bottlenecks
Bottlenecks Hide Other Bottlenecks

The most useful admission here is that intuition is what fails: the team stepped back because performance work was not going as fast as they wanted and found their instincts were leading them astray. The mechanism is that bottlenecks hide others — you remove the visible one and uncover a worse problem that was masked behind it, which is why a correct fix so often moves no numbers. Their answer is to survey broadly before drilling in, explicitly checking whether the processor back end is worse, finding it is not, and setting it aside. The diagnostic figure is instructions per cycle averaging below one against hardware capable of eight to twelve, which reframes the problem from slow code to an idle processor. Every fix that follows is about memory layout rather than logic, and returns nearly twenty per cent more throughput from configuration alone.

AWS re:Invent

如何引用本页

复制这份有来源支持的实体档案的稳定引用。