Etched is developing a specialized AI chip for single-model execution

Etched is a new company run by former tech executives and Harvard dropouts. They are making the Sohu chip for transformer AI models and want it to be faster, cheaper, and more efficient than Nvidia’s GPUs. Despite the tough AI chip market, they have raised $125.36 million in funding.

More and more industries are using generative AI, which is excellent for companies that make chips that run the models.

A huge amount of power lies with Nvidia, which controls between 70% and 95% of the market for AI chips. Because they don’t want to fall behind in generative AI, cloud providers like Microsoft and Meta are spending billions of dollars on Nvidia GPUs.

It’s straightforward to see why generative AI vendors are unhappy with how things are now. Their success depends on the big chipmakers’ actions. That’s why they, along with shady VCs, are looking for bright new companies to compete with the big names in AI chips.

Many alternative chip companies want to play, but Etched is one of the most intriguing. Gavin Uberti (ex-OctoML and ex-Xnor.ai) and Chris Zhu, two Harvard dropouts, founded Etched just two years ago. They wanted to make a chip that could only do one thing: run AI models. Robert Wachen and Mark Ross, former CTOs of Cypress Semiconductor, joined them.

That happens a lot. Many small and large tech companies have developed or are in the process of developing inferencing chips, which are specifically designed to run AI models.

Meta owns MTIA, Amazon owns Graviton and Inferentia, and so on. However, Etched’s chips stand out due to their exclusive capability to operate transformer models.

Since a group of Google researchers first proposed it in 2017, the transformer has emerged as the most popular method for building generative AI models.

Sora, OpenAI‘s model for making videos, is based on Transformers. They depend on models that make text, such as Anthropic’s Claude and Google’s Gemini. They also power art generators, such as the newest version of Stable Diffusion.

At that time, Uberti, CEO of Etched, said, “We bet that transformers would take over the world.” “In the history of AI, we’ve reached a point where specialized chips that work better than general-purpose GPUs are unavoidable. The world’s technical leaders are well aware of this.”

The chip that Etched is using, called Sohu, is an ASIC, which stands for “application-specific integrated circuit.” This implies that Etched specifically designed it to function with active transformers.

Uberti claims that Sohu, utilizing TSMC’s 4nm process, surpasses GPUs and other general-purpose AI chips in inferencing performance while consuming significantly less power.

Uberti said that Sohu outperforms even Nvidia’s next generation of Blackwell GB200 GPUs in terms of speed and cost when running word, picture, and video transformers.

One Sohu server replaces 160 H100 GPUs. For business leaders who need specialized chips, Sohu will be a cheaper, faster, and more eco-friendly choice.

 Sohu does all of this in what ways? There are several methods to achieve this, but the most straightforward and straightforward approach is to simplify the technology and software utilized for inference. Because Sohu doesn’t run non-transformer models, the Etched team was able to eliminate hardware parts unnecessary for transformers and reduce the software typically required to set up and run non-transformers.

When Etched came out, it changed the game in the race for generative AI infrastructure. Besides being costly, the GPUs and other hardware components required to run models at scale today consume excessive power.

According to Goldman Sachs, AI will cause a 160% increase in the amount of electricity used by data centers by 2030, resulting in a significant rise in greenhouse gas emissions.

According to researchers at UC Riverside, the use of AI around the world could cause data centers to use between 1.1 and 1.7 trillion gallons of fresh water by 2027, which would affect local resources. Many data centres use water to cool their servers.

Uberti either optimistically or overtly promotes Sohu as the solution to the industry’s low consumption problem.

The Uber driver said, “In short, our future customers won’t be able to afford not to switch to Sohu.” “Businesses are willing to risk Etched because speed and cost are a must for the AI products they want to make.”

But how can Etched succeed when so many others come right after it? That is, if the company meets its goal of bringing Sohu to the mass market in the next few months.

At the moment, Etched doesn’t have a direct competitor, but AI chip startup Perceive just showed off a processor that can speed up transformers with hardware. Groq has also put a lot of money into making its ASIC work better with transformers.

What if, despite the competition, transformers became less popular over time? In that case, Uberti says Etched will do what everyone else does: design a new chip. But that’s a pretty big backup when you consider how long it took to make Sohu happen. Despite these concerns, investors are still putting a huge amount of money into Etched.

Etched closes $120M series A funding round amid high expectations

Etched said today that it had closed a $120 million Series A funding round. Primary Leading the round were Positive Sum Ventures and Venture Partners.

Etched raised a total of $125.36 million in this round, which included big-name angel investors like Peter Thiel (Uberti, Zhu, and Wachen are all alumni of the Thiel Fellowship), Thomas Dohmke, CEO of GitHub, Kyle Vogt, co-founder of Cruise (and the Bot Company), and Charlie Cheever, co-founder of Quora.

These investors probably think that Etched has a decent chance of being able to grow its server-selling business. And maybe it does. Uberti says that so far, “tens of millions of dollars” worth of hardware have been reserved by customers who have not been named.

Uberti said that the upcoming release of the Sohu Developer Cloud, which will give customers a chance to try out Sohu in an online interactive playground, should lead to more sales.

Still, it looks like it’s too early to tell if this is enough to move Etched and its 35-person team toward the future that the company’s founders want. In the best of times, the AI chip market can be tough. For example, well-known AI chip startups like Mythic and Graphcore came very close to failing, and funding for AI chip startups dropped sharply in 2023.

“Video generation, audio-to-audio modalities, robotics, and other future AI use cases will only be possible with a faster chip like Sohu,” Uberti says in a strong sales pitch. How far AI technology goes will depend on how well the infrastructure can handle growth.