Amazon’s AI Chip Revolution: How They’re Ditching Nvidia’s High Prices and Speeding Ahead

Amazon tested its new AI chips in its Austin lab. The goal is to make its AWS cloud services less reliant on expensive Nvidia chips and offer up to 50% better pricing and performance.

Six engineers tested a brand-new, secret server design on a Friday afternoon in Amazon.com’s chip lab in Austin, Texas. Amazon executive Rami Sinno said on Friday during a visit to the lab that the server was full of Amazon’s AI chips, which compete with Nvidia’s chips and are the market leader.

Amazon is making its processors so it doesn’t have to rely so much on expensive Nvidia chips (“Nvidia tax”) to run some of its AI cloud business at Amazon Web Services, which is the company’s main growth driver.

With its chips, Amazon hopes to make it easier for customers to do complicated math and handle huge amounts of data more cheaply. This is also being done by Microsoft and Alphabet, two of its competitors.

Amazon’s Annapurna Labs is part of the company’s AWS cloud business. Sinno is in charge of engineering there. He said that more and more Amazon customers were asking for cheaper alternatives to Nvidia. Amazon bought Annapurna Labs in 2015.

The business is only now beginning to work on AI chips, but Amazon’s main chip, Graviton, which does non-AI computing, has been in the works for almost ten years and is now in its fourth generation. Trainium and Inferentia are two types of AI chips that are still relatively new.

It should be half as expensive as running the same model with Nvidia, according to David Brown, Vice President of Compute and Networking at AWS, who said this on Tuesday. “There is a potential improvement in price and performance by up to 40% to 50%.”

AWS Sales Surge 17% to $25 Billion

AWS sales, which make up just under 20% of Amazon’s total income, rose 17% to $25 billion in the January-March quarter compared to the same time last year.

AWS controls about a third of the cloud computing market, while Microsoft Azure controls about a quarter.

Amazon used a quarter-million Graviton chips and 80,000 of its custom AI chips to handle the extra traffic on all of its platforms during Prime Day, the company said.