Nvidia blew past all expectations on Wednesday, reporting soaring profits driven by its graphics processing units (GPUs) that excel at powering artificial intelligence workloads. But other classes of AI chips are beginning to gain momentum.
Every major cloud provider is now designing its own application-specific integrated circuits (ASICs), from Google’s TPU to Amazon’s Trainium to OpenAI’s plans with Broadcom. These chips are smaller, cheaper, easier to use, and may reduce those companies’ dependence on Nvidia’s GPUs. Daniel Newman of Futurum Group told CNBC he expects ASIC chips “to grow faster than the GPU market over the next several years.”
Alongside GPUs and ASICs, there are field-programmable gate arrays (FPGAs), which can be reconfigured after manufacturing for uses such as signal processing, networking, and AI. And there is an entire generation of AI chips designed to run directly on devices rather than through the cloud — a segment led by companies like Qualcomm and Apple.
CNBC spoke with experts and insiders at major tech companies to break down this crowded landscape and the different kinds of AI chips.
GPUs for general-purpose computing
GPUs were once used mainly for video games, but they turned Nvidia into the world’s most valuable public company once they became the engine of modern AI. Nvidia shipped roughly 6 million units of its current-generation “Blackwell” GPUs last year.
The shift from gaming to AI began in 2012, when researchers trained the neural network AlexNet using Nvidia GPUs — a breakthrough that many view as the spark of the modern AI revolution. AlexNet competed in a high-profile image-recognition contest and relied on GPUs rather than CPUs, delivering stunning accuracy and a major competitive edge.
The same parallel-processing ability that makes GPUs capable of rendering lifelike graphics also makes them ideal for training deep-learning models, which learn from data rather than explicit programming.
Today, GPUs are sold into data-center systems paired with CPUs to run cloud-based AI workloads. CPUs have a handful of powerful cores for sequential tasks, while GPUs have thousands of smaller cores specialized in parallel operations such as matrix multiplication.
Because they can execute massive numbers of operations simultaneously, GPUs are ideal for both training and inference. Training teaches AI models to find patterns in huge datasets; inference uses those models to make decisions on new information.
GPUs remain the primary engine for Nvidia and its closest competitor AMD. Software is a key differentiator between them: Nvidia relies on its CUDA ecosystem, while AMD offers a largely open-source stack.
Both companies sell cloud GPUs to providers such as Amazon, Microsoft, Google, Oracle, and CoreWeave, which then rent out the computing power to AI developers.
Anthropic’s $30 billion agreement with Nvidia and Microsoft, for example, includes the equivalent of 1 gigawatt of computing capacity built on Nvidia hardware. AMD has recently secured major commitments from OpenAI and Oracle as well.
Nvidia also sells directly to governments and AI companies — including at least 4 million GPUs to OpenAI — and to foreign governments such as South Korea, Saudi Arabia, and the UK.
The company told CNBC that it charges roughly $3 million per server cabinet containing 72 Blackwell GPUs, and it is shipping around 1,000 such cabinets every week.
Dion Harris, Nvidia’s senior director for AI infrastructure, said he never imagined demand would grow to this level. “When we talked to companies about an eight-GPU system years ago, they thought it was excessive.”
ASICs for specialized cloud AI
GPU-based training fueled the first wave of large language models, but inference has become increasingly important as models mature. Inference can be run on less flexible, lower-cost chips built specifically for certain math operations — which is where ASICs come in.
If a GPU is a “Swiss Army knife” that can execute many different parallel tasks, an ASIC is a single-purpose tool — extremely fast and efficient but locked into one type of operation once manufactured.
“You can’t change these chips once they’re etched in silicon,” said Chris Miller, author of *Chip War*. “There’s a trade-off between efficiency and flexibility.”
Nvidia’s GPUs are versatile enough to meet countless AI needs, but they are expensive (up to $40,000 per unit) and difficult to obtain. Startups rely on them partly because designing a custom ASIC can cost tens of millions.
Cloud giants, however, are investing heavily in ASICs because they promise major savings at scale.
“These companies want more control over the workloads they build,” Newman said. “But they’ll continue to work with Nvidia and AMD — the computing demand is enormous.”
Google was the first to build a custom AI ASIC, launching the Tensor Processing Unit (TPU) in 2015. Work began in 2006, but it became urgent in 2013 when Google realized AI could double the size of its data-center footprint. In 2017, the TPU helped enable the Transformer architecture that underpins most modern AI.
Google revealed the seventh-generation TPU in November. Anthropic will train its Claude model on one million TPUs. Some believe TPUs rival — or outperform — Nvidia GPUs.
“A lot of people expect Google eventually to make TPUs available more broadly,” Miller said.
AWS followed with its own chips after acquiring Annapurna Labs in 2015. It launched Inferentia in 2018 and Trainium in 2022, with Trainium3 expected soon.
Amazon says Trainium delivers 30% to 40% better price-performance than alternatives. Anthropic currently uses half a million Trainium2 chips to train its models.
To build custom ASICs, cloud providers depend on companies like Broadcom and Marvell — which supply critical IP and networking expertise. “That’s why Broadcom has become one of the biggest winners of the AI boom,” Miller said.
Broadcom helped design Google’s TPUs and Meta’s 2023 accelerators and is building custom chips for OpenAI starting in 2026.
Microsoft has developed the Maia 100. Qualcomm has the A1200. Intel offers the Gaudi line. Tesla is working on its AI5 chip. Startups like Cerebras and Groq are pushing novel architectures.
In China, Huawei, ByteDance, and Alibaba are designing their own ASICs despite US export restrictions.
Device-level AI with NPUs and FPGAs
A third category of AI chips is built for running models directly on devices rather than through the cloud. These chips are typically integrated into system-on-a-chip (SoC) designs and are known as edge-AI processors. They allow AI features to run locally and efficiently, preserving battery life and privacy.
“You’ll be able to run AI tasks directly on your phone with extremely low latency,” said Saif Khan, former White House AI and technology advisor. “And without sending data to a data center.”
Neural Processing Units (NPUs) are a major part of this category, developed by Qualcomm, Intel, AMD, and others.
Apple doesn’t use the term NPU but embeds a “neural engine” in its M-series Mac chips and its A-series mobile chips.
“That approach has proven incredibly effective,” said Tim Millet, Apple’s VP of platform architecture. “It’s fast and gives us more control over the experience.”
Snapdragon chips in Android phones, Samsung’s custom NPUs, and edge-AI processors from NXP and Nvidia power AI in cars, robots, cameras, and smart-home devices.
“Most of the spending today is still in data centers,” Miller said. “But that will change as AI spreads into phones, cars, wearables, and everything else.”
FPGAs offer even more flexibility because they can be reprogrammed after manufacturing, though they are less power-efficient than ASICs or NPUs.
AMD became the largest FPGA maker after acquiring Xilinx for $49 billion in 2022. Intel ranks second after buying Altera for $16.7 billion in 2015.
Bottom line: Nvidia is still far ahead
All of these AI-chip companies rely on one manufacturer: TSMC in Taiwan.
TSMC is building a massive manufacturing site in Arizona, where Apple will move part of its production. Nvidia CEO Jensen Huang said in October that Blackwell GPUs have reached “full production” there as well.
Despite the increasingly crowded market, unseating Nvidia remains extremely difficult.
“Nvidia is in this position because it earned it,” Newman said. “It spent years building this developer ecosystem — and it’s the one that won.”
