Measure a carpenter by hammer use and everything gets smacked
Two days at AMD's Advancing AI, an industry showing how cheap a token can get, and a conversation about what it takes to know it was worth it.
When I sat down with Quentin Anthony, VP of AI Engineering at Zyphra, at AMD’s Advancing AI in San Francisco, I learned what it takes to write kernels with agents.
For Quentin, working with kernels and building a frontier model is as much a physics problem as a software one, and he sits directly at that intersection. Fun fact: he’s 1 of the 16 developers who took part in METR’s randomized controlled trial on AI and developer productivity (the study we gabbed about endlessly on the show last year), that found experienced engineers ran 19% slower with AI while believing they were 20% faster.
Why the hardware choice was a software decision
Quentin’s road to AMD didn’t start at Zyphra. During his PhD he trained models on Frontier, the Oak Ridge supercomputer built on MI250X GPUs, where he had no budget and no alternatives (reminds me of one of my favorite episodes this year, about a researcher in exactly this conundrum!) He had GPUs available, so he learned the hardware and wrote the kernels himself.
Today Zyphra runs on MI300X, whose high memory capacity lets them use far less parallelism and gives their large sparse mixture-of-experts models room to work. They’ve trained an 8B and a 74B model and are scaling toward the frontier on a three-way arrangement where IBM houses hardware that is AMD end to end, GPUs and interconnects both.
Hiring for first principles
Turns out Zyphra hires more physicists and PhDs from outside computer science than computer scientists, and Quentin was direct about why. What he screens for is a shared latency, the ability to think deeply about a problem from first principles and stay with a single experiment for a month or more.
The practical version of that argument is that deep analytical training is common across many doctoral fields, while computer science PhDs are expensive and heavily recruited elsewhere. That research mindset matters most when you are operating out of distribution, inventing the method as you go.
Regular developers now need that same mindset. Steering an agent well draws on the same discipline as designing an experiment well.
Where the models break
Because Quentin works on the metal, he can explain model failure from the training side, which almost nobody writing about AI coding tools does.
Kernels are out of distribution. By contrast, things like JavaScript have enormous troves open data, so they're strong at it. The weakness is about what the model saw, not how smart it is.
His mitigation is craft rather than tooling. He hands the model two or three good AMD kernels from his own organization that use the exact principle he wants applied, and steers from there. The line he gave me is the best distillation of the thought:
“You have to yourself know how to write kernels in order to have a model write kernels with you.”
AKA: you have to know what’s good, to make good.
AMD’s own answer to that gap is a software one, and software was most of what the company shipped at the show. ROCm now moves on a six-week feature-release cadence, and the ROCm.ai work puts AMD-authored skills directly inside Claude, Cursor, and Codex, alongside Hyperloom, an open-source agentic system whose whole job is analysis, kernel optimization, and validation. AMD’s claim for it is hours instead of weeks. Whether that closes the distribution problem Quentin describes is an empirical question, but it is the right shape of attempt, which is to put the missing expertise where the model can reach it.
The failure he warns about most is the opposite instinct. Agents condition people to pull the lever and walk away expecting magic, when the job is active steering and verifying the output yourself. He reminded me of the old adage that developers spent 10% of their time writing code and 90% debugging it. Careful thinking moved to the front of the process rather than out of it. A clear objective with good examples gets a better answer and spends fewer tokens getting there.
Context rot is the other failure, and his explanation of it comes from having tried to train against it. Long-context data is genuinely hard to source, because there are only so many long textbooks in the world and few conversations that run to 100 turns, so models are thin exactly where a long session needs them to be strong. Managing that takes a deliberate strategy.
Models struggle with long-context data and extended multi-turn conversations.
When a model starts to go off the rails, detect the error carefully, summarize the conversation, and start a new chat with shorter context.
Quentin works through the most basic APIs available with extra features turned off, so he controls exactly what enters the context window.
What comes next, and why developers stop seeing it
When I asked what the industry looks like in a few years, Quentin described a heavily distributed future, and he helps design clusters, so he has a view. Expect an explosion of hardware types alongside model blocks that map onto them, with more memory living inside the model itself.
For everyday developers, the payoff is that the heavy infrastructure starts to lighten. A phone handles ordinary chat locally without reaching for a two-trillion-parameter model, and pushes the hard problems up to a local node or a cluster only when the work demands it. The cheapest available compute does what it can before passing the rest along.
The carpenter and the hammer
And, well, that brings us back to tokenmaxxing (you thought you got away from that, huh?! Well guess again, this ghost will be haunting us a while), the practice of treating AI spend as evidence of AI progress. Quentin called it an unfortunate artifact of making the metric the goal, a Goodhart’s Law problem. Then he put it like this:
“If you measure every carpenter by how much they use a hammer, everything would get smacked.”
I loved this line. We’ve made this argument many times before, so this hit home for me. It carries more weight coming back from someone who trains the models than from anyone who sells measurement software.
Whether tokenmaxxing or tokenminimizing, you’re measuring the wrong thing
For 30 days, the busiest leaderboard at Meta ranked its engineers by the tokens they burned. What shipped, and whether it held up, never made the board. An internal tool nicknamed Claudeonomics logged 60.2 trillion AI tokens in a single month, a volume worth roughly $900 million at list prices, though a buyer Meta’s size pays a steep discount on that fi…
His read on why the frenzy persists was about market structure rather than discipline. A great deal of private equity sits in the space and nobody is turning a profit yet, so the incentives point at consumption. Once model companies charge the full price of the compute, efficient routing stops being a virtue and becomes arithmetic.
His prescription is the part I keep coming back to. Give verifiable tasks to AI, and give verifiable tasks to humans too. Judge people on clear outputs over a quarter, using the review structures every organization already has, and let them find their own tools. If someone can do the whole thing with AI, they have earned the room to take on something more ambitious.
The scoreboard problem
Quentin has no measurement product to sell, which is what makes the hammer line worth taking seriously. The pattern he describes is measurable, and measuring it is the half of the equation I work on.
At LinearB we look at how software moves from a developer’s machine into production, and our 2026 AI benchmarks put a number on the gap Quentin is describing. Agentic pull requests, the ones an agent opens on its own, reach production at sharply different rates depending on the team receiving them, at 79% in elite organizations, 58% in good ones, and 37% in the rest. Across all pull requests those same tiers run between 81% and 90%. The stated reason is human rather than technical. When an agent opens a pull request nobody owns, it languishes.
Quentin’s individual failure mode shows up here scaled to an org chart. Pull the lever, walk away, and the output accumulates where nobody is accountable for it. Steering is what closes the gap, and steering is a management practice before it is a tooling choice.
Kraken went looking for exactly that. Their engineering operations team measured where review time goes across thousands of engineers, and the answer was not the code. Nik Sudan, their engineering operations lead, put it plainly:
“Size and complexity accounted for maybe 5 to 10% of the slowdown. 70 to 80% came from time zone differences, team silos, and reviewer availability.”
They cut P90 review time from 2 days 15 hours to 1 day 3 hours over four months, and they now track cost per contribution by dividing AI spend by impact-weighted output. That last number is the one Quentin was reaching for.
How LinearB helps Kraken find hidden bottlenecks across thousands of engineers | Nik Sudan
Are you confusing a skyrocketing AI token bill with actual engineering value? This week on Dev Interrupted, Kraken's Engineering Operations Lead, Nik Sudan, joins the show to break down the harsh realities of moving agentic AI projects from pilot to production without compromising code health. He unpacks why raw AI adoption is a flawed vanity metric, de…
The cost side lands in proportion too. Token spend at the 90th percentile runs about $481 per developer per month, against roughly $12,500 a month for a fully loaded developer, which is under 4% of what that person costs. Capping the spend caps your best people. The number worth managing is the cost of a pull request that shipped.
These figures are correlations from LinearB’s 2026 AI benchmarks, not evidence that AI is what moved them, and the full picture is in the guide.
Activity is not impact
I asked Quentin what a developer should take from all of this, and his answer was deliberately small: Reflect on your own actual workflow and be thoughtful about them.
That restraint is the point. The teams that come out of this period will not be the ones that burned the most tokens or bought the most tools. They’ll be the ones who could say which work reached production and which work sat in a queue nobody owned.
Cheap inference is close to a solved problem, and the show floor was two days of proof. Knowing what the inference bought you is still open, and it is the question we should be asking next.
Thanks to Sophie Snow and Katie Fielder at Axicom for the invitation to Advancing AI, to Quentin Anthony for half an hour of unusually direct answers, and to Anush Elangovan, AMD’s VP of AI software, whose two Dev Interrupted appearances shaped how I think about tokens and time.
