Andrew FeldmanCo-founder & CEO

AI transitioned from a parlor trick to an active production tool in mid-2025. The minute you use AI in production, speed matters because fast tokens are directly more productive. Users get more done in less time, making fast inference the primary driver of value.

The key metric is tokens per second per user, measuring the speed from the first token to the last token in a response. This is especially critical for agentic workloads with multicycle turns where waiting is amplified. Real-time speed allows users to stay longer and tackle harder problems.

When the internet was slow, Netflix mailed DVDs. When the internet became fast, they did not optimize mailing envelopes; they became a movie studio. Speed completely changes how technology is used, opening up new domains for AI that are impossible under slow dial-up speeds.

We transitioned from one CPU doing everything to a multi-silicon landscape. Traditional GPUs like Nvidia and AMD handle generic workloads. Hyperscalers build internal ASICs like Google's TPU and AWS's Trainium, while independent players like Cerebras design chips optimized solely for AI.

It was a major validation of our vision. The 20 billion USD technology licensing agreement and acqui-hire made clear to the industry that traditional GPU architectures cannot deliver fast inference. It proved that the AI market requires dedicated inference processors.

Jalapeno is a custom ASIC co-developed by OpenAI and Broadcom to address their massive compute demands. OpenAI understands exponential adoption curves and has been visionary in securing capacity, striking massive compute and memory deals with us and other partners.

First, HBM memory is sold out, which we avoid by using SRAM. Second, TSMC's CoWoS packaging process is fully booked, which we do not use. Third, TSMC's 3-nanometer foundry space is highly congested, while we build our chips using their less constrained 5-nanometer process.

AI inference is dominated by moving weights from memory to compute. By building a dinner-plate-sized chip stuffed with fast SRAM instead of HBM, we keep the weights directly on the silicon. This allows us to move data to compute two thousand five hundred times faster than a GPU.

Agentic AI does not just provide answers; it takes actions in the digital world like fetching data or placing orders. The AI chip acts as the brain, directing the CPU as the body. This active loop is driving CPU consumption and demand through the roof.

Models evolve rapidly. If you build a specific model's architecture directly into your hardware circuitry, your chip will have a very short commercial life. We focus on accelerating the underlying mathematical calculations, like sparse linear algebra, which supports any model.

We were honest with our venture capitalists that we were attacking a massive engineering problem to build something fifty times better than a GPU. During our hardest phase, we spent eight million USD a month for eighteen months before successfully manufacturing our first wafer.

In graphics, you move data to the GPU, calculate for a long time, and send the result. Inference is the opposite: you move a massive volume of weights from memory to compute to calculate a single word, and then repeat. Traditional HBM memory bandwidth is too slow for this.

SRAM memory is exceptionally fast but cannot store much data per unit area. To hold the weights of modern AI models on fast SRAM, we had to build a chip that was forty-six thousand square millimeters, which is fifty-eight times larger than a standard GPU.

It was attaching a wafer-sized chip to a motherboard, supplying it with massive electrical power, and cooling it. No vendors could help us because nothing like it existed. We spent eighteen months failing, analyzing each failure, and inventing new materials to solve it.

Our architecture features a million identical tiles with built-in redundancies. If a single tile fails, we shut it down and route the calculations to a redundant tile. We also water-cool the systems, running them much colder than GPUs to reduce temperature-related failures.

CUDA is losing its dominance. Within a two-year period, seventy percent of state-of-the-art training models shifted away from CUDA, with Gemini training on TPUs and Claude on Trainiums. Furthermore, there is zero CUDA moat in inference, requiring only eight keystrokes to move.

OpenAI needed to secure data center capacity, which is measured in megawatts. We are delivering a full cloud solution, leasing two hundred fifty megawatts in 2026, 2027, and 2028. OpenAI connects directly to our clusters via an API to run their workloads.

We presented a bold proposal that allowed TSMC to leverage their strengths without changing too much. They recognized that AI would run better on giant chips and were willing to take a risk to learn. This collaborative spirit is how great companies win.