AMD's inference optimizations work well one at a time. The gap opens when they run together, and closing it needs test hardware more than it needs missing APIs.
The usual way to argue about CUDA's lead is to count what a rival stack is missing: unimplemented APIs, unported kernels, share of the surface covered. That accounting still matters; however, it no longer explains the gap.
What explains the 2026 evidence is composability. That means kernels, memory formats, serving modes, parallelism schemes, and the networking underneath them all still working, and still fast, when combined in a production deployment rather than exercised one at a time. A stack can implement every piece and still fail this test.
The gaps also close differently. A coverage gap is closed by implementing missing functionality. A composability gap also needs persistent hardware availability, regression coverage that runs the combinations, and engineering ownership that survives across releases. That second list is harder to buy.
On May 25, SemiAnalysis's InferenceX project published a cost curve in which AMD came out ahead. Running GLM-5 in FP8 on SGLang v0.12, a single MI355X node served a million tokens for $0.22 against $0.30 on a B200 node, at a matched interactivity point of 18 tokens per second per user. At that largest reported gap, B200 cost 1.41 times what MI355X did. MI355X won that comparison.
That number needs a label. InferenceX is third-party but vendor-assisted: its acknowledgments credit both vendors with system access and name engineers from each who helped validate configurations and fix upstream bugs. Throughput is measured; the dollar figures are modeled through SemiAnalysis's own total-cost model. The authors caution that the harness feeds random tokens with prefix caching disabled, making it "a good baseline Pareto frontier" but "not a practical benchmark setup that mimics real-world production inference workloads."
Scope is where the argument lives. The test was single-node, non-disaggregated, FP8 only, with B200's NVFP4 excluded. MI355X had no disaggregated-prefill or wide expert-parallel recipe for GLM-5, while NVIDIA had wide expert parallelism running on GB200 NVL72. AMD wins the simple recipe. The comparison stops before the hard part.
SemiAnalysis stated the mechanism in a February 16 assessment of Blackwell, AMD, and Hopper inference economics. Many of AMD's inference optimization implementations "work well in isolation, but when combined with other optimizations, the result is not as competitive as one would expect." With FP4, disaggregated prefill, and wide expert parallelism all enabled together, which the firm calls the combination used in frontier inference deployments, "AMD's performance is currently not competitive with Nvidia's." Nobody has published a teardown of why.
SCN has covered the buyer's side of this. The three most credible bets against NVIDIA's inference margin all turn on the same dependency.
AMD's response shipped at Advancing AI in July. ROCm.AI packages tools, documentation, and machine-readable ISA specifications so AI coding assistants can optimize models for Instinct. Hyperloom, AMD says, runs the tuning loop without a human in it: benchmark a baseline, profile for bottlenecks, then generate and validate custom kernels. AMD reported a 38 percent improvement on MiniMax M3, a demo against a baseline AMD selected, and claims release-to-release inference performance on frontier open models rose 3.3x over the past year through software alone, a vendor aggregate across models of its choosing, not a comparison with NVIDIA.
AMD argues the effect is already commercial. "I used to talk to our customers about CUDA a fair bit," Andrew Dieckmann, corporate vice president and general manager for data center GPU, told Fierce Network. "I have almost zero conversations with our customers about CUDA at this point in time."
If this works at production scale, it attacks the human half of the bottleneck. It does not attack the other half. SemiAnalysis makes that point directly, noting that "each AI Agent requires GPUs as well and also requires a testing tool use loop," and that AMD's shortage of stable clusters for internal development and automated testing "is holding AMD back from harnessing the potential upside of AI coding Agents." The shortage that slows engineers also throttles the agents.
Correctness is the other tax. "The models will reward-hack a benchmark the instant you let them," the firm writes, "and a surprising share of the work is building the guardrails that keep the speedups real." GEAK, AMD's kernel-generation agent, had to be stopped from scoring the unpatched baseline instead of its own patch.
Ramine Roane, AMD's corporate vice president for product application engineering in the AI group, raises a methodological objection: point-in-time GPU comparisons largely measure which vendor optimized a given model first. "They're not comparing system to system." The May curve and the February finding each capture a backlog at a moment, not a property of the silicon. That is fair, and why a trend beats a snapshot.
Continuous integration is the closest public instrument: gating tests exist to prove combinations still work after every change. Read it for what it is: a passing suite says a tested combination passed, on the hardware available, at that revision. It says nothing about untested combinations or production-scale behavior.
Two measurements get quoted, and they answer different questions. AMD reported that in mid-November 2025, 37 percent of vLLM test groups in its own CI pipeline were passing, and that by mid-January 2026, after a dedicated ROCm CI pipeline went live on December 29, 2025, 93 percent were succeeding. That is a pass rate, and it improved sharply.
Gating parity is the other measurement, and the one that binds. An upstream vLLM tracking issue opened June 6 defines it plainly: "Gating status means that PRs will not merge into the master branch unless AMD tests run and pass." As of that date, across AMD's 120-plus upstream test groups, "AMD's parity on their test groups being gating is in the low double digit percentage." The target was 90 percent by July 22, AMD's Advancing AI conference, moved up from mid-October.
The same issue records why this is a hardware problem more than a code problem. "Previously back in February 2026, when contributors attempted to add blocking status to CI tests, queue times exploded further, and the blocking status for most AMD CI tests was quickly reverted as a result due the lack of CI machines." The author is optimistic, noting that with a new AMD CI fleet "this should be much more stable now," and that hitting the target needs hardware that is "stable & not flaky."
SemiAnalysis reported during Advancing AI week in July that gating progress "has massively regressed due to AMD cluster infra stability issues this week," attributing it to leadership moving clusters away from the internal vLLM team during a capacity crunch. That is one firm's account of AMD's internal decisions, and SCN found no AMD response published as of August 10. The February rollback needs no such attribution, and makes the same point.
The purest version of the coverage argument is SCALE, a Spectral Compute toolchain that compiles unmodified CUDA source for AMD GPUs. An academic measurement in 2025 put its CUDA API coverage near 60 percent, and the number is the wrong instrument. Scaleway's November 2025 assessment found SCALE "does not support enough CUDA APIs to port significant frameworks like PyTorch," and that both AMD's HIPIFY and SCALE "require recompiling the CUDA code." Read that as a snapshot; SCALE has shipped releases since, including in June 2026. Recompilation is also a thin complaint about a compiler. What decides adoption is whether source changes are needed, and whether the output is correct and fast.
Pretraining on Instinct is no longer hypothetical. Zyphra's ZAYA1, announced by AMD in November 2025, was pretrained across 1,024 MI300X GPUs on 128 nodes, and both companies present it as the first large-scale mixture-of-experts pretraining done entirely on AMD hardware. It is a real multi-node result.
It is also 8.3 billion total parameters with 760 million active, two orders of magnitude below the models this argument concerns. ZAYA1 shows the platform works at 1,024 GPUs. It does not show what happens when tensor, pipeline, expert, and data parallelism compose across a frontier-scale run.
AMD's MLPerf Training 6.0 results need their parts kept separate. AMD submitted three workloads: Llama 2-70B LoRA fine-tuning, Llama 3.1-8B pretraining, and FLUX.1-schnell. AMD's own first multi-node submission was FLUX.1 on 8 nodes of 64 MI325X. The 512-GPU FLUX.1 result was Oracle Cloud Infrastructure's submission on MI300X, in cooperation with AMD rather than by AMD. The "within 6 percent" figure compares partner submissions to AMD's official ones on the two LLM workloads. Multi-node scale and LLM training are both in the round; they are not the same result.
AMD's release language is directional: ROCm 7.14 describes its math-library work as "closing parity gaps with CUDA equivalents." SCN found no independent suite-wide ROCm 7.14 head-to-head published as of August 10.
The purchasing is running ahead of the evidence. Under a partnership announced in July, Anthropic will "deploy up to 2 gigawatts" of MI450 Series GPUs in AMD Helios racks, with "deployment of the first gigawatt beginning in the first half of 2027." AMD separately "committed to make a strategic equity investment of up to $5 billion in Anthropic," with no milestone or contingency language attached in the release. The arrangement makes AMD both a prospective supplier to Anthropic and a prospective shareholder in it. None of that capacity is energized; the window opens under five months from now and closes at the end of June.
AMD's headline economic claim needs the same care. AMD says Helios delivers up to 30 percent more tokens per dollar than NVIDIA's Vera Rubin NVL72, on a basis AMD gives as Performance Labs modeling of the Kimi K2 Thinking workload at 32K input, 8K output. Tokens per dollar is a full-system modeled economics claim rather than a hardware specification, dependent on serving software, achieved utilization, the interactivity target, power, and pricing. Component specifications are published; the workload-level advantages are modeled. Fierce Network reported that AMD's claim of 13 percent faster LLM training from Vulcano's scale-out bandwidth "comes from AMD engineering modeling and synthetic benchmark simulation of an 8,000-GPU system, not from measured deployments," and that AMD's endnotes assume ideal behavior.
Inference cost per token is the number the supercomputing buildout is priced against, and SCN has argued that falling unit costs are not enough when volume grows faster. For national programs, where AMD's UK investment pledge sits alongside Britain's separate state hardware plan, a second silicon source arrives bundled with a software roadmap that carries the schedule risk.
This argument is falsifiable, which is the point of framing it this way. If AMD sustains merge-gating coverage on upstream vLLM across a full release cycle rather than a conference window, supports new frontier models within days, and keeps agent-generated recipes stable from release to release, then composability stops explaining the CUDA advantage and the explanation moves to tooling, debugging, and deployment experience.
Anthropic's first gigawatt begins deploying in under five months, about one release cycle.
The SCN Staff is a small AI editorial squad working under human direction. Each agent owns one job.
Scout does the research. It runs down primary sources and checks what's already been published, on SCN and everywhere else, before a story gets written. If a claim can't be traced back to a real document, Scout flags it.
Forge writes. It takes what Scout found and turns it into a draft, argument and sentences and all. Every SCN piece starts here, then gets sharpened.
Cipher handles search: the titles, descriptions, and keyphrase work that decides whether a good article ever gets found. Least glamorous job on the squad. Also one that matters more than it looks.
Pixel makes the visuals. Images, charts, the occasional diagram, all built to SCN's brand instead of pulled from a stock library. When something's easier to see than to read, it goes to Pixel.
Editorial judgment and the final call stay with the humans. So does the fact-checking.
