AMD launched its amd helios architecture on July 23 as a rack-scale system for training, fine-tuning and serving large AI models. The pitch is a machine designed as one unit: host CPUs, GPU accelerators, high-bandwidth memory and networking are specified together instead of left for a customer or server maker to assemble into a workable rack.
AMD says Helios is in production and combines sixth-generation EPYC “Venice” 9006-series server CPUs, fifth-generation Instinct MI455X GPUs, Pensando networking and a UALoE fabric. The company positions the system for frontier AI and calls its design open because it uses open industry standards. That does not mean every part of the hardware or software stack is open source.
What is the AMD Helios architecture?
Helios is AMD’s attempt to sell an AI rack as an integrated computing domain rather than as a pile of GPUs with cables and wishful thinking. A workload enters through the CPU host layer, runs on MI455X accelerators, keeps model data close to those GPUs in HBM4 memory, and uses the fabric to exchange data across the rack.
AMD says one Helios configuration links 72 GPUs into a single scale-up domain with 260 TB/s of scale-up bandwidth. Scale-up is the traffic inside that 72-GPU domain, where accelerators need to coordinate on a workload. Scale-out is the network capacity used to extend work beyond one rack. AMD lists 43 TB/s of scale-out bandwidth for the rack.
For context on the physical setting, an AI rack is part of a larger facility that must keep servers powered, cooled and connected. Data centers turn those constraints into online services, and Helios is designed for the portion of that job dominated by AI compute.
What hardware and capacity does AMD specify?
- EPYC hosts with 96 high-frequency CPU cores, PCIe Gen6 links to GPUs and 1.6 TB/s of CPU memory bandwidth.
- MI455X accelerators rated by AMD at 40 petaflops of FP4 AI compute, 432 GB of HBM4 memory and 23.3 TB/s of memory bandwidth per GPU.
- Three 800G network interface cards per GPU, according to AMD’s architecture graphic.
- Rack aggregate specifications of 2.9 exaflops FP4, 1.4 exaflops FP8, 31 TB of HBM4 and 1.7 PB/s of aggregate HBM bandwidth.
Those figures are AMD specifications, not independently measured results. AMD also says Helios is designed to provide up to 15% more AI compute, 50% more HBM capacity and 50% more scale-out bandwidth than Nvidia’s Vera Rubin NVL72 rack. Its Kimi K2 Thinking comparison uses a 32K-token input and 8K-token output, and reports modeled per-GPU throughput gains of up to 15%, 12% and 10% at low, medium and high interactivity. Useful numbers, but still vendor modeling with vendor-selected conditions.
AMD says its ROCm software supports frameworks including PyTorch, TensorFlow and JAX. The company argues that agentic workloads increase pressure on compute, memory and data movement because they can involve reasoning steps, retrieval, tool calls and CPU-side orchestration.
What could complicate Helios deployment?
SemiAnalysis reported on July 24 that Helios, which it described as AMD’s first rack-scale AI system, was moving through a slow production ramp and encountering backplane-reliability issues. The analysts also alleged that more than 550 Broadcom Ethernet retimers could be required in a rack because much of the backplane needs retiming. Those are SemiAnalysis assessments, not independently confirmed in the material available here, and sit alongside AMD’s statement that Helios is in production.
This story draws on original reporting from ServeTheHome.
