Amazon keeps the pressure on Intel, AMD with 192-core Graviton5 CPU
Amazon keeps the pressure on Intel, AMD with 192-core Graviton5 CPU
Story by Tobias Mann • 15h
Amazon keeps the pressure on Intel, AMD with 192-core Graviton5 CPU
The homegrown chips now account for half of all new CPUs added to AWS over the past three yearsre:invent Amazon on Thursday unveiled Graviton5, its densest, highest performance CPU yet, cramming 192 processor cores into a single socket and promising new levels of AWS performance.…
Since they first launched in 2018, the Graviton chips have become a mainstay of AWS's compute offering. For the past three years running, Amazon claims that Graviton accounted for more than half of all new CPU capacity.
With Graviton5, capacity is in ample supply with 192 Arm Neoverse V3 cores fabbed on TSMC's 3nm process tech. Those cores are backed by a larger 192MB L3 cache, which helps to cut down on cache misses and improves performance by reducing the amount of data that needs to be fetched from slower DRAM. The chip also features an improved memory subsystem that boosts speeds to 7200 MT/s, with support for 8800 MT/s DIMMs in the works.
Combined, Amazon says its new M9g instances deliver 25 percent higher performance than its Graviton4-based M8g instances. The latter featured a pair of CPUs with 96 cores each. With Graviton5, AWS has consolidated that to a single socket. This puts it in contention with the highest core count CPUs from AMD and Intel today, which top out at 192 and 144 cores respectively.
"Linking processors introduce new latency paths, and when a core needs to access memory on the other CPU, the request has to move across that interconnect, and that adds latency, extra protocol overhead, and sometimes even queuing. In certain scenarios, it could take up to three times longer," AWS EC2 VP David Brown said in Thursday's keynote.
According to Amazon, Graviton5's higher core count effectively cuts inter-core latency by roughly a third, benefiting workloads like online gaming, high-performance databases, electronic design automation, and analytics. While Graviton5 instances will run on one socket, Amazon VP and distinguished engineer Ali Saidi tells us that the actual compute nodes still contain two sockets, which share a Nitro smartNIC.
Much like Graviton3, which was the first server CPU to add support for PCIe 5.0, Graviton5 will be the first to support PCIe 6 out of the box.
The jump from 96 cores on Graviton4 to 192 on Graviton5 is notable, as AWS hasn't historically chased core counts. As Brown told The Register back in 2023, the company preferred to standardize on CPUs that can be adapted to multiple roles by tweaking things like the ratio of memory to compute or the speed of the interconnect to each instance.
This philosophy hasn't changed with Graviton5, Nafea Bshara, co-founder of Amazon's Annapurna Labs division, told El Reg this week at re:Invent.
"The best way to get the lowest cost is to build a product that can serve everybody. Because fluctuations in usage, in the morning you might be running HPC and in the evening it's Fortnite," he said.
Having a high performance yet versatile chip allows Graviton to slot into any job Amazon needs it to, whereas more specialized parts might be left idling waiting for a job. "High utilization gets passed to customers as lower cost."
For AWS M9g instances, Graviton5 has been paired with the company's custom Nitro 6 smartNICs, which double the network bandwidth to 100 Gbps.
The real value of Nitro is the ability to offload storage, networking, and virtualization functionality, freeing up CPU resources that would otherwise be consumed by these processes for client workloads. These cards also introduce Nitro Isolation, which aims to provide mathematical proof that customer workloads are isolated from one another.
Amazon's M9g instances are currently available in preview with additional compute (C9g) and memory (R9g) optimized variants slated for release in the new year. As was the case with past Gravitons, these instances all use the same chip, but adjust the ratio of memory capacity or bandwidth per core.
Graviton5's debut comes just days after AWS unveiled its Trainium3 UltraServers AI rack systems, which utilize a scale up switch fabric similar to Nvidia's NVL72, to deliver a 4.4x performance uplift over last gen.
For the first time, Amazon has ditched the x86 cores used in previous UltraServers in favor of a full stack of Annapurna silicon including Graviton, Trainium, and Nitro.
Amazon may have been one of the first to deploy a homegrown CPU, but it's not the only one. At Microsoft Ignite last month, Redmond revealed its second-gen Arm CPU codenamed Cobalt 200.
The chip is based on Arm's Neoverse Compute Subsystems V3 and is fabbed on TSMC's 3 nm process tech. Each processor features 132 active cores, each with a large 3 MB L2 cache and a total of 192 MB of shared L3. The chips also utilize a custom memory controller that enables memory encryption by default as well as Arm's Confidential Compute Architecture to prevent customer workloads from mingling with the hypervisor.
Google, which has a long history of building custom AI accelerators, recently got in on the custom CPU game with its Axion family of instances, which it first showed off during Cloud Next last year. These chips are a fair bit smaller than either Graviton or Cobalt with up to 72 Arm Neoverse V2 cores per instance, complemented by 576 GB of memory and 100 Gbps of networking.
And while Oracle doesn't have a CPU to call its own just yet, the burgeoning cloud provider and database giant operates one of the largest fleets of Arm-based CPUs from Ampere. Back in October, Oracle unveiled new instances based around the 192-core AmpereOne M processors. ®
This article may contain affiliate links that Microsoft and/or the publisher may receive a commission from if you buy a product or service through those links. |
