AWS launched Amazon EC2 Trn2 instances and Trn2UltraServers based on Trainium2 chips at the 2024 re:Invent conference, as well as the next-generation Trainium3 AI chip. The new generation of instances has significantly improved performance, and the price-performance ratio is 30-40% higher than the previous generation of GPU-based EC2 instances. This move marks an important step for AWS in the field of AI computing, providing more powerful computing capabilities for the training and deployment of large language models, and promoting the widespread application and development of AI technology through cooperation with multiple partners. Significantly improve cost-effectiveness.
At the 2024 AWS re:Invent conference, Amazon Web Services (AWS) announced the launch of Amazon Elastic Compute Cloud (EC2) instances based on Trainium2 chips, which are officially available to users. The price and performance of this new instance are 30-40% higher than the previous generation of GPU-based EC2 instances. "I'm excited to announce the general availability of Trainium2-powered Amazon EC2 Trn2 instances," said AWS CEO Matt Garman.
In addition to Trn2 instances, AWS also launched Trn2UltraServers and demonstrated the next generation Trainium3AI chip. Equipped with 16 Trainium2 chips, Trn2 instances are capable of delivering up to 20.8 petaflops of computing performance and are designed for training and deploying large language models (LLMs) with billions of parameters.
Trn2UltraServers combines four Trn2 servers into one system, providing up to 83.2 petaflops of computing power to achieve higher scalability. These UltraServers have 64 interconnected Trainium2 chips to meet customers' computing power needs during training and inference. "The launch of Trainium2 instances and Trn2UltraServers gives customers the computing power they need to solve the most complex AI models," said David Brown, vice president of Compute and Networking at AWS.
AWS has partnered with Anthropic to launch a large-scale AI computing cluster called Project Rainier, using hundreds of thousands of Trainium2 chips. This infrastructure will support Anthropic's development, including optimization of its flagship product Claude to run on Trainium2 hardware.
In addition, Databricks and Hugging Face are also working with AWS to leverage Trainium's capabilities to improve the performance and cost efficiency of their AI products. Databricks plans to use the hardware to enhance its Mosaic AI platform, while Hugging Face integrates Trainium2 into its AI development and deployment tools.
Other Trainium2 customers include Adobe, Poolside and Qualcomm. Garman mentioned that after Adobe used Trainium2 for early testing of the Firefly inference model, the results were very satisfactory and it is expected to save a lot. “Poolside expects to save 40% compared to other options,” he added. “Qualcomm is leveraging Trainium2 to develop AI systems that can be trained in the cloud and deployed at the edge.”
In addition, AWS also previewed its Trainium3 chip, which uses a 3-nanometer process. Trainium3-based UltraServers are expected to be launched by the end of 2025 and are designed to provide four times higher performance than Trn2 UltraServers.
To optimize the use of Trainium hardware, AWS also launched Neuron SDK, a software tool suite that helps developers optimize models to achieve optimal performance on Trainium chips. The SDK supports frameworks such as JAX and PyTorch, enabling customers to integrate the software into existing workflows with minimal code modifications.
Currently, Trn2 instances are available in the US East (Ohio) region and will be expanded to other regions in the future. UltraServers is currently in preview.
All in all, the launch of Trainium2 and its related products and services by AWS have provided strong impetus for the rapid development of the field of artificial intelligence and further consolidated AWS's leading position in the fields of cloud computing and AI. In the future, with the launch of Trainium3, its advantages in the field of AI computing will be even more significant.