On October 6th, it was reported that OpenAI, the American AI company behind ChatGPT, plans to develop its own AI chip. Internal discussions have been ongoing since last year, and they have even begun evaluating potential acquisition targets in the AI chip industry to address the shortage and high costs of the AI chips they rely on.
The report indicates that the OpenAI team is considering three procurement options for AI supercomputing power. These options include building their own AI chips, forming closer partnerships with chip companies like NVIDIA, and diversifying chip supply beyond NVIDIA—ultimately aiming to surpass NVIDIA.
As early as 2022, OpenAI CEO Sam Altman publicly voiced concerns about the scarcity of NVIDIA GPU chips, stating that the company was severely limited by GPU availability.
Since NVIDIA dominates 95% of the global AI training market, and with a shortage of NVIDIA GPU graphics cards coupled with rising AI computing costs, even a powerhouse like OpenAI is seeking new solutions to avoid long-term bottlenecks.
01
Compute Cost Soaring, OpenAI Seeks Custom AI Chips
Since 2023, the AI landscape, led by models like ChatGPT, has witnessed a surge in large AI models, driving AI towards greater versatility. However, the scarcity and high cost of computing power have become central factors limiting AI’s progress.
Currently, NVIDIA holds an 82% share of the global data center AI acceleration market and monopolizes 95% of the global AI training market, making it the biggest winner in this AI battle. The company’s founder and CEO, Huang Renxun, has amassed significant wealth, with a market value exceeding $1 trillion and a personal net worth of $39.9 billion.
Simultaneously, the surging demand for computing power has led to a scarcity of NVIDIA GPUs. The number of NVIDIA A100 graphics cards a company possesses has become a standard measure of its computing power.
According to statistics, OpenAI used between 10,000 and 30,000 NVIDIA GPUs to train the GPT-3.5 model. Reports from Union Consulting suggest that running ChatGPT could require up to 30,000 NVIDIA GPU graphics cards when considering the processing power of the NVIDIA A100 graphics card.
In terms of prices, the domestically available H800 graphics card is now priced at up to 200,000 RMB per card, while the A100/A800 prices have risen to around 150,000 to 100,000 RMB per card. Using 2,000 P computing power as an example, the H800 GPU offers 2 P per card and requires 1,000 cards, resulting in an estimated card cost of 200 million RMB. The A800 GPU offers approximately 0.625 P per card and requires 3,200 cards, resulting in an estimated card cost of as much as 320 million RMB.
In addition to purchasing GPU graphics cards, servers need to consider overall configuration, including CPUs, storage, NV-Link communication connections, as well as factors such as power consumption, site rental, and maintenance costs. The total cost can exceed 600 million RMB. OpenAI is currently using Microsoft’s supercomputer, which costs hundreds of millions of dollars and incorporates tens of thousands of NVIDIA chips for computing power training.
However, since OpenAI announced its GPT Enterprise Cloud business, its relationship with Microsoft has been gradually distancing, and reports are suggesting that Microsoft is also developing custom AI chips.
It’s not just OpenAI; in this wave of computing power demand, major tech giants such as Google, Amazon, Microsoft, and Meta have all been designing AI chips for their businesses. For example, Google’s latest Pixel 8 series smartphones use the Google Tensor G3 chip and the Titan M2 co-processor, both of which are in-house products. Amazon’s latest in-house ARM server CPU chip, Graviton 3E, is also used in its AWS cloud business.
However, OpenAI lacks experience in developing AI chips, and it takes at least two years or more from project initiation to mass production for a single chip. It remains unclear whether OpenAI will continue to pursue its custom chip plans and how effective the chips will be, which will require time to assess.
Reuters, citing industry insiders, reports that this is a massive investment, with annual development costs potentially reaching hundreds of millions of dollars. Meanwhile, OpenAI is considering acquiring some AI chip companies to expedite its in-house chip development process.
However, custom AI chips could take several years to materialize, meaning that OpenAI will continue to rely on chip suppliers like NVIDIA and AMD for the long term.
Interestingly, on October 4th, The Information reported that Meta has abandoned custom VR/AR chips, with the team disbanded and instead opting for processors provided by Qualcomm.
Therefore, whether OpenAI’s custom AI chip will ultimately prove effective remains to be seen.
There are also reports that SoftBank plans to invest $1 billion in establishing a new AI hardware company with Sam Altman, and Apple designer Jony Ive is reportedly discussing the creation of a new AI hardware product with Altman.
02
Microsoft to Unveil its First Custom AI Chip Next Month
According to a report from The Information, Microsoft is set to unveil its first custom-designed chip specifically for AI applications next month. This move comes as the global AI boom, sparked by the introduction of ChatGPT and the widespread discussion of generative AI technology, has led to a frenzied demand for AI chips, causing shortages in NVIDIA GPUs.
Insiders have revealed that Microsoft plans to introduce its inaugural AI-designed chip at its annual developer conference next month. This marks the culmination of several years of work by Microsoft and aims to reduce the company’s dependence on AI chips designed by NVIDIA.
Microsoft’s in-house chip is designed similarly to NVIDIA GPUs and is intended for use in data center servers dedicated to training and running large language models, such as OpenAI’s ChatGPT.
Previously, it was reported that Microsoft began developing its own AI chip, codenamed Athena, in 2019. The primary goal was not to directly replace NVIDIA chips but rather to reduce costs and continually incorporate AI capabilities into various services.
Microsoft’s annual report updated its risk factors in July, highlighting the importance of GPUs in its data centers, and underscoring the critical role GPUs play in its operations.
In August, UBS indicated that Microsoft’s limitations in terms of the number of GPUs it could use were a real concern and could impact its ability to generate AI revenue next year. Developing their GPUs could help mitigate this risk.
In September, rumors circulated that Microsoft had begun scaling back orders for NVIDIA H100 chips, indicating that Microsoft’s AI chip development and testing had matured, reducing their heavy reliance on NVIDIA.
If Microsoft does launch its own AI chip, it will be on par with Amazon and Google, both of which have developed their own AI chips.
Google was one of the earliest companies to venture into independent chip development back in 2013, using ARM technology to build chips for its extensive server network. By leveraging its designs, Google could better manage the interaction between hardware and software, achieving the efficiency expected in settings like car factories or industrial food processing. Amazon also has its hardware, claiming that its Graviton chips are 40% more efficient than comparable x86 chips.
Google began deploying its own Tensor Processing Units (TPUs) in 2015, custom-made for the branch of artificial intelligence known as neural network machine learning. In April of this year, The Information reported that Microsoft had recently entered this competition, intending to use its in-house processors more extensively internally and in collaboration with industry partner OpenAI.
Apart from custom design and greater control over the entire computing system, internet companies’ motivation to develop their chips is also financial. NVIDIA is projected to earn 56.51 cents in operating profit for every dollar of revenue this year, making it one of the most profitable tech companies globally. As Amazon’s founder, Jeff Bezos, once said, “Your margin is my opportunity.” Cloud providers don’t have to hand over those profits to chip vendors; they can instead control a crucial part of their cost structure, but it’s not easy. Eight years ago, Amazon acquired Israeli chip design company Annapurna Labs for $370 million, recognizing the challenge of building a development team from scratch.
NVIDIA is still likely to dominate the AI chip market, as a recent agreement that makes more chips available for use through Google Cloud shows that even Google isn’t ready to abandon chip manufacturers. However, driven by the demand for AI, the cloud computing industry is expected to be the largest single contributor to industry growth in the coming years. This means that chip suppliers like NVIDIA and AMD will have more opportunities, and the usage of in-house branded chips developed by service providers like Google, Amazon, and Microsoft will continue to grow.
03
Intensifying Competition in the Large Model Market: Google and Amazon Turn to OpenAI Competitor
Just days after Amazon announced its investment, news emerged that OpenAI’s competitor, the artificial intelligence (AI) unicorn company Anthropic, is undergoing another round of financing.
According to a report by The Information on October 4th, Anthropic, a company founded just two years ago, is in negotiations with Google and other investors, planning to complete a new round of financing of at least $2 billion, with a post-investment valuation estimated to be between $20 billion to $30 billion—more than five times its valuation of $4 billion in March this year.
Subsequently, the competition in the AI large model market has intensified, with both Google and Amazon showing interest in and making moves in the direction of OpenAI’s competitors.
Anthropic AI was founded in 2021 by former OpenAI Research Vice President Dario Amodei, Tom Brown, the first author of the GPT-3 paper, and others.
At that time, researchers led by Amodei left OpenAI due to differences in the company’s direction. They were concerned that Microsoft’s investment in OpenAI would lead it down a more commercial path, deviating from the company’s original vision.
In December 2022, the Anthropic team published a paper on arXiv titled ‘Constitutional AI: Harmlessness from AI Feedback,’ describing an unsupervised training-based model called AnthropicLM v4-s3 with 52 billion parameters, directly competing with OpenAI’s GPT-3 model.
In January of this year, Anthropic released a new AI chatbot model product called Claude, based on AnthropicLM v4-s3, which is considered a strong competitor to ChatGPT.
On February 4th, Google Cloud, a division of Google, announced a new partnership with Anthropic, and reports suggest that Google invested nearly $400 million (around ¥2.03 billion) in Anthropic. This new financing will increase Anthropic’s post-investment valuation to nearly $5 billion.
Before Google’s investment, Anthropic had already raised over $700 million in financing, with investors including Alameda Research, a cryptocurrency hedge fund founded by FTX founder SBF, among others. One of the reasons for Google’s investment in Anthropic is the CEO’s repeated expressions of concern about the threat and impact of ChatGPT on its search and advertising businesses.
According to The Information’s report, Anthropic has informed investors that its annual revenue has been around $100 million previously. However, due to its deep collaboration with Amazon, it is expected that its revenue will significantly increase in 2023, reaching $200 million, with monthly revenue approaching $17 million. By the end of 2024, Anthropic hopes to achieve annual revenue of $500 million, which is 1/200th of its valuation multiple, far higher than OpenAI’s valuation multiple.
Recommended Reading:
- Your Guide to Industrial Internet: 11 Crucial Questions
- AI Begins to Deliver Returns: Can It Really Make Money?
- Unveiling OpenAI CEO’s Ouster: Insider Details
- Breaking News: FedEx Takes on Amazon in E-Commerce
- GPT-4o Unveiled: Siri Faces Unprecedented Challenge
Disclaimer: This article is created by the original author. The content of the article represents their personal opinions. Our reposting is for sharing and discussion purposes only and does not imply our endorsement or agreement. If you have any objections, please contact us through the provided channels.