The editor of Downcodes learned that OpenAI is cooperating with Broadcom to develop customized inference chips, and Taiwan Semiconductor Manufacturing Company (TSMC) may serve as the foundry. This move has attracted widespread attention in the industry, and speculation about the future development direction of OpenAI is rampant. This article will delve into the reasons behind OpenAI’s development of specialized chips and its potential impact.
Recently, it was reported that OpenAI is working with Broadcom to develop a customized inference chip. According to Reuters, discussions between the two companies are very confidential and Taiwan Semiconductor Manufacturing Company (TSMC) may serve as the foundry for the project. This news triggered widespread speculation about the future development direction of OpenAI.
Picture source note: The picture is generated by AI, and the picture authorization service provider Midjourney
So, why does OpenAI need its own inference chip? First of all, OpenAI’s cloud computing costs are very huge. Although partners like Microsoft support part of the costs, controlling the hardware by yourself can undoubtedly significantly reduce operating costs. Many companies have discovered that building their own data centers is much more economical than renting cloud services.
In addition, developing special chips adapted to its own services may also be a strategic goal of OpenAI. As we all know, AI applications consume huge amounts of energy, so by optimizing the synergy between hardware and software, OpenAI's services will become more efficient.
OpenAI is also pitching investors the idea of building large-scale data centers dedicated to running AI services that could also be cheaper to build or operate if equipped with custom chips. What's more, the considerations of decentralized supply chains cannot be ignored. Due to limited global semiconductor production capacity, the risk of relying on external suppliers exists, and developing its own chips can reduce dependence on third-party products.
While we can't imagine OpenAI would be willing to get into the troublesome business of selling hardware, which requires a lot of real investment and would increase headcount, it might be deployed at the edge of the network where inference tasks often need to be as close to the user as possible. Related equipment, like many content delivery networks and Netflix, this architecture is definitely a good idea.
Speaking of inference chips, they are actually no stranger to the market. Things like AWS' Inferentia, Google's Tensor Processing Unit (TPU), and Microsoft's Maia silicon can handle inference and training workloads.
Interestingly, news of OpenAI’s partnership with Broadcom also pushed the latter’s stock price up slightly. Broadcom's latest quarterly earnings report showed that it expects to sell $12 billion in AI silicon wafers this fiscal year, a figure that is $1 billion higher than previous expectations, but investors responded with some disappointment. So working with one of the hottest names in AI software will undoubtedly make Wall Street even more excited.
The cooperation between OpenAI and Broadcom heralds the further integration of the AI industry chain and brings OpenAI stronger cost control capabilities and technological autonomy. In the future, customized AI chips may become one of the key factors for large AI companies to improve their competitiveness. This has far-reaching significance for the development of the entire AI industry.