NVIDIA announces the launch of AI service: NVIDIA AI Foundry and NVIDIA NIM inference microservices, which aim to enhance generative AI applications for enterprises globally. This initiative introduces the Llama 3.1 collection of models.
NVIDIA AI Foundry provides a platform enabling enterprises and national bodies to create custom “supermodels” tailored to specific industry applications. It leverages the Llama 3.1 models in combination with NVIDIA’s computational resources and expertise. These models are trained using both proprietary data and synthetic data from Llama 3.1 405B and the NVIDIA Nemotron™ Reward model.
Powered by the NVIDIA DGX™ Cloud AI platform and in collaboration with major public cloud providers, this service equips enterprises with scalable computational resources to meet evolving AI demands. Enterprises and nations are now exploring AI strategies to create domain-specific language models for unique generative AI tools.
“Meta’s openly available Llama 3.1 models mark a pivotal moment for the adoption of generative AI within the world’s enterprises,” said Jensen Huang, founder and CEO of NVIDIA. “Llama 3.1 opens the floodgates for every enterprise and industry to build state-of-the-art generative AI applications. NVIDIA AI Foundry has integrated Llama 3.1 throughout and is ready to help enterprises build and deploy custom Llama supermodels.”
Adding to this, Mark Zuckerberg, founder and CEO of Meta, commented, “The new Llama 3.1 models are a super-important step for open source AI. With NVIDIA AI Foundry, companies can easily create and customize the state-of-the-art AI services people want and deploy them with NVIDIA NIM. I’m excited to get this in people’s hands.”
NVIDIA has also released NIM inference microservices for the Llama 3.1 models through their website. These microservices deliver up to 2.5 times higher throughput compared to alternatives. Enterprises can use them alongside new NVIDIA NeMo Retriever NIM microservices to develop advanced retrieval pipelines for AI-driven applications like digital human avatars and AI assistants.
Accenture is the first to adopt NVIDIA AI Foundry. Using their Accenture AI Refinery™ framework, they will create custom Llama 3.1 models for both their own use and their clients. Julie Sweet, chair and CEO of Accenture, noted, “The world’s leading enterprises see how generative AI is transforming every industry and are eager to deploy applications powered by custom models. Accenture is leveraging NVIDIA NIM inference microservices for internal AI applications. Now, with NVIDIA AI Foundry, we can help clients quickly create and deploy custom Llama 3.1 models to power transformative AI applications for their business priorities.”
NVIDIA AI Foundry offers an end-to-end service combining NVIDIA software, infrastructure, and expertise. This service integrates open community models and technology from the NVIDIA AI ecosystem. Enterprises can develop custom models using the NVIDIA NeMo platform, including the NVIDIA Nemotron-4 340B Reward model, renowned for its top ranking on Hugging Face RewardBench.
The process allows enterprises to generate synthetic data using Llama 3.1 405B and Nemotron-4 340B to enhance model accuracy. Companies with their training data can use NVIDIA NeMo for domain-adaptive pretraining to refine model effectiveness.
The collaboration between NVIDIA and Meta extends to offering distillation recipes for Llama 3.1. This enables developers to build smaller, customized models for various AI infrastructure deployments, including workstations and laptops powered by NVIDIA RTX™ and GeForce RTX GPUs.
Industry leaders like Aramco, AT&T, and Uber are among the initial users of the new NIM microservices for Llama 3.1. They operate in sectors such as healthcare, energy, financial services, retail, transportation, and telecommunications.
NVIDIA has stated that production support for these microservices is available through NVIDIA AI Enterprise. Free access for research, development, and testing will soon be available for members of the NVIDIA Developer Program.
