On Wednesday, OpenAI unveiled its first custom-built inference processor, designed and manufactured in collaboration with Broadcom. Named Jalapeño, the new processor was designed specifically for the unique needs of OpenAI’s inference systems. OpenAI’s own AI models assisted in the development of the chip, the company said.
While the chip is still being tested, OpenAI says early results show significantly better performance-per-watt than current state-of-the-art alternatives.
The partnership was officially announced in October, but OpenAI’s chip plans have long been rumored as a way to reduce the company’s dependence on Nvidia’s GPUs. Google and Amazon have both built custom chips to serve a similar purpose, often called “AI accelerators” — silicon designed specifically to speed up machine learning workloads.
OpenAI president Greg Brockman explained the company’s approach to chip development on its in-house podcast, shortly after the Broadcom partnership was announced.
“We have a deep understanding of the workload,” Brockman said in the episode. “We’ve really been looking for specific workloads that are underserved, [and asking] how can we build something that will be able to accelerate what’s possible?”
Jalapeño is specifically designed for inference, the process of running pre-built AI models in response to user commands. In the announcement, OpenAI emphasized the chip’s low operating cost when running real-time coding models. It’s likely that more performance-intensive tasks like pre-training will still rely on Nvidia hardware, but even small reductions in inference costs could do a lot to improve the company’s bottom line.
Optimizing that inference system may prove to be a crucial factor in the economics of AI going forward — and it’s likely to take place at every level of the stack. OpenAI is already building agentic products like Codex and the models that power them, as well as data centers to run those models. Moving into purpose-built chips lets the company go even further in that process, as the company explained in its announcement.
“OpenAI is not only developing frontier models or building products on top of them; it is designing the infrastructure underneath them: chip architecture, kernels, memory systems, networking, scheduling, deployment systems, and product experience,” the company wrote. “Because OpenAI operates across the stack, each layer can be optimized around the same goal: making its models faster, more reliable, and more affordable for users.”
Source: TechCrunch
