NVIDIA GB300 Leads AA-AgentPerf: What ‘20× More Agents per Megawatt’ Really Means

GB300 leads the first AA-AgentPerf results—but the headline needs context

NVIDIA's Blackwell Ultra GB300 NVL72 has posted the leading result in the first release of Artificial Analysis's AA-AgentPerf, a benchmark designed around realistic coding-agent workloads rather than a single chat-completion test.

The attention-grabbing result is that GB300 NVL72 delivered approximately 20 times the concurrent-agent capacity per megawatt of an H200 system under the reported DeepSeek V4 Pro service-level objectives. NVIDIA also reported a configuration supporting about 61,400 concurrent agents per megawatt and 57.5 agents per GPU, versus 2,600 per megawatt and 1.4 per GPU for the H200 comparison.

That is important, but it is not the same as saying that every GB300 workload is 20 times faster than H200. The result measures how many coding agents a tuned serving system can keep active while meeting specified latency and token-rate targets. It combines GPU architecture, HBM, rack-scale interconnect, inference software and system tuning.

The 61.4K figure comes from NVIDIA's published table for a 30-token-per-second service-level configuration. Results at other targets are different, so it should not be presented as one fixed capacity number for every deployment.

Why agentic workloads need a different benchmark

An AI agent does more than answer one prompt. A coding agent may inspect files, call tools, revise a plan, generate a patch, read test output and continue for many turns. Context can grow substantially while individual responses remain bursty. A platform therefore needs to manage long prompts, repeated KV-cache access, mixture-of-experts routing and many simultaneous sessions without allowing user-visible latency to deteriorate.

AA-AgentPerf replays coding-agent trajectories and increases concurrency until the system can no longer satisfy the selected service-level objective. Its three practical dimensions are:

For DeepSeek V4 Pro, Artificial Analysis defines three published tiers: 20 tokens/s with P95 TTFT no higher than 10 seconds, 60 tokens/s with P95 TTFT no higher than 5 seconds, and 180 tokens/s with P95 TTFT no higher than 3 seconds. The benchmark uses input lengths from roughly 1K to 131K tokens, with a mean of about 27K, to better represent changing agent context.

What is behind the GB300 result

The result is a rack-scale systems story, not only a faster-GPU story. GB300 NVL72 couples 72 Blackwell Ultra GPUs through a high-bandwidth NVLink domain. NVIDIA's submission also used serving and communication optimizations around SGLang, TensorRT-LLM, vLLM, expert parallelism and fused mixture-of-experts kernels.

This matters because large mixture-of-experts models can spend substantial time moving activations between accelerators and coordinating expert workloads. More compute is useful only when memory capacity, memory bandwidth, scale-up fabric and software scheduling can keep it occupied. The benchmark therefore rewards the complete serving stack.

Four limits to remember before using “20×” in a capacity plan

First, AA-AgentPerf normalizes by measured GPU-only power, including GPU die and HBM power. CPU, networking, storage and cooling power are excluded. It is a useful accelerator-efficiency measure, but it is not total facility power or full data-center PUE.

Second, NVIDIA submitted the tuned B300/GB300 configurations, while Artificial Analysis built the H200 comparison configuration. Artificial Analysis explicitly notes that H200 and AMD MI355X systems may have additional optimization headroom. The launch table is a transparent snapshot, not a permanent ceiling.

Third, the comparison spans different system scales and software stacks. A 72-GPU NVL72 rack and an eight-GPU H200 server do not isolate architecture alone. Buyers should treat the result as a deployment-platform comparison.

Fourth, the benchmark currently focuses on DeepSeek V4 Pro coding-agent trajectories. Model architecture, quantization, context distribution and tool behavior can materially change the ranking. Results should be reproduced with the intended production model before a purchase decision.

The commercial signal: agents per megawatt may become a buying metric

For an agent service, peak FLOPS alone does not define revenue capacity. If one rack can maintain more simultaneous sessions within the required latency and power envelope, the operator may need fewer racks, less electrical capacity and less cooling for the same number of paid users.

That shifts procurement discussions toward cost per active agent, not only cost per GPU. However, the calculation must include the complete bill of infrastructure: accelerator systems, host CPUs, networking, storage, power conversion, cooling, software licenses, floor space and utilization.

The result also strengthens demand around the GPU rather than only for the GPU. High-density AI racks require qualified power-management devices, high-current connectors, capacitors, retimers, optical components, management controllers and cooling hardware. Supply risk can move to any one of these supporting layers.

RFQ and capacity-planning checklist

Before comparing an GB300, H200 or alternative accelerator proposal, request a reproducible configuration record:

1. Confirm the exact model version, precision, quantization and accepted accuracy threshold. 2. Define P50/P95 TTFT, output-token rate and concurrency targets for the intended workload. 3. Record input/output length distributions, tool-call frequency and KV-cache assumptions. 4. State whether power means GPU-only, server input power, rack power or facility power. 5. Capture GPU count, topology, framework, kernel versions and parallelism settings. 6. Include networking, storage, CPU, cooling and electrical upgrade costs in TCO. 7. Confirm allocation, lead time, warranty, regional service and replacement-unit terms.

Availability must be confirmed for the complete rack-level BOM. A quoted GPU delivery date does not guarantee that networking, power and cooling components will be ready at the same time.

What comes next: Rubin claims remain roadmap figures

NVIDIA says its forthcoming Vera Rubin platform will provide up to 50 PFLOPS of NVFP4 compute and improve tool-calling and end-to-end agent performance. Those figures describe NVIDIA's roadmap; they are not measured AA-AgentPerf results and should not be mixed with the current GB300 table.

The more durable message is broader than one generation. Agentic AI is pushing accelerator evaluation from isolated benchmark throughput toward sustained, SLO-compliant service capacity. AA-AgentPerf gives buyers a useful new lens—but the configuration, power boundary and optimization ownership must travel with every headline number.

Sources and methodology

• Artificial Analysis, “AA-AgentPerf: Benchmarking AI Hardware for Agentic Workloads,” June 12, 2026.
• NVIDIA Technical Blog, “NVIDIA Achieves Leading Agentic Coding Performance on First Agentic AI Benchmark.”
• Artificial Analysis public hardware-configuration disclosures for AA-AgentPerf.

The figures above are benchmark results and vendor disclosures, not guarantees of production performance or component availability. Validate the intended model and service-level objective on the exact proposed platform before issuing a production purchase order.