🤗 Hugging Cast S2E2 - Accelerating AI with NVIDIA!

The host welcomes viewers to a live show focused on constructing AI with open models and open source. This season, the show shifts from news to more demos, aiming to provide practical examples for application in viewers' companies. The show is interactive, with a Q&A session planned after 30 minutes. The overarching goal is to facilitate the use of AI models across various compute stacks through partnerships with cloud, hardware, and on-premise platforms. The episode features a collaboration with Nvidia, highlighted by the unveiling of 'train on dgx cloud,' a service announced at GTC. This service allows for direct training on Hugging Face Hub with the latest GPU technologies, emphasizing accessibility and ease of use for the community.

Guests Abishek, Rafa, and Morgan join the show to discuss AI training innovations. They introduce 'train on dgx cloud,' a new service that simplifies training AI models using H100 GPUs on demand, directly from the Hugging Face Hub. The service is designed for Enterprise Hub organizations, offering fine-tuning of large language models (LLMs) without additional coding. The process is user-friendly, with options for different training tasks and models. The discussion also covers the benefits of using Optimum Nvidia, a toolkit for accelerating AI workloads, which can significantly reduce first token latency and increase throughput when used with Nvidia's latest hardware.

Abishek and Rafa provide a detailed walkthrough of Auto Train, an open-source project that has evolved to support various AI tasks beyond natural language processing. They demonstrate how to use Auto Train for training on DGX Cloud, showcasing the user interface and the simplicity of the training process. The discussion includes the ability to choose different hardware options, select training parameters, and upload datasets directly from the Hugging Face Hub. The session also addresses the efficiency of training, with examples of how the service can be cost-effective even for large models like Mistral 7B.

The conversation continues with a live demonstration of training a model using DGX Cloud. The process is shown to be quick and efficient, with real-time logs accessible to monitor training progress. Once training is complete, the model artifacts are pushed to the Hugging Face Hub, where they can be accessed and utilized. The model card is automatically generated, and training metrics are available for review. The discussion emphasizes the ease of use and the speed at which a model can be trained and deployed using the new service.

The hosts discuss the flexibility of training models with different datasets and parameters. They address questions about the maximum size of datasets that can be used with the training service, emphasizing that there is no hard limit, though it depends on GPU size and batch size. The conversation also covers data privacy, ensuring that only the user has access to their training datasets. Additionally, they touch on the support for various LLMs and the ability to train adapter models, which can be deployed without merging the weights.

The discussion highlights the global support for different LLMs and the community's role in shaping the service. The hosts mention the support for various models like YOZHA, Falcon, Mistral, and others, and invite feedback for additional models. They also discuss the different data formats supported by Auto Train and the availability of documentation to assist users in formatting their datasets correctly for training.

Morgan introduces Optimum Nvidia, an open-source toolkit that leverages Nvidia's TensorRT and Tensor Cores for accelerated AI workloads. The demo showcases how easy it is to integrate Optimum Nvidia into existing Transformer workflows with minimal code changes. The benefits include reduced first token latency and increased throughput, especially when using the latest Nvidia hardware. The presentation also covers the use of float8 quantization for faster inference and the potential for future integrations with other Hugging Face products.

The final segment explores the integration of Optimum Nvidia with Hugging Face's inference solutions. Morgan demonstrates a proof of concept for deploying an Optimum Nvidia-accelerated model on Hugging Face's Inference API. The discussion compares Optimum Nvidia with TGI, highlighting the benefits of using Optimum Nvidia for specific Nvidia GPUs that support float8 data types. The session concludes with a Q&A, addressing questions about recording availability, using live data for training, and the use of private link connections for endpoints.

The hosts wrap up the show by thanking the guests and participants. They confirm that the show will be available for on-demand viewing and will also be published on YouTube. The episode concludes with a reminder of the show's purpose: to demonstrate how to build AI with open source and open models in collaboration with partners.