Using Open Source AI Models with Hugging Face | Build Free AI Models

Alara, a PhD candidate at Imperial College London and a former machine learning engineer at Hugging Face, introduces a code along tutorial focused on using open source AI models with the Hugging Face project. She explains that Hugging Face is an AI company dedicated to simplifying access to state-of-the-art AI research through open source tools and libraries. The core of their ecosystem is The Hub, a platform for discovering and managing AI models and datasets, similar to GitHub. Alara outlines the various features of The Hub, including the ability to store large model files for free, and mentions other Hugging Face libraries like Transformers, diffusers, and datasets. The tutorial aims to teach attendees how to use these tools to create custom machine learning pipelines, resulting in multilingual text translation and image captioning models, and a custom dataset on The Hub.

The tutorial begins with setting up the coding environment by importing necessary libraries such as torch, Transformers, huggingface_hub, and datasets. Alara emphasizes the need for a Hugging Face account and token for uploading datasets. She guides users to ensure they have the latest versions of the libraries by running specific installation commands and restarting the kernel. The focus then shifts to importing the Transformers and huggingface_hub libraries, with Alara providing a brief explanation of the importance of these tools in the Hugging Face ecosystem.

Alara delves into the concept of tokenizers in natural language processing (NLP), explaining how they convert text into a mathematical format that models can process. She demonstrates how to load a pre-trained tokenizer from The Hub using the Transformers library. The paragraph also covers the loading of a pre-trained model, discussing the differences between using the base AutoModel class and task-specific classes like RobertaForSequenceClassification. Alara illustrates how to identify the correct class for a model using its configuration and how to load it explicitly for more control over the model's parameters.

This section discusses the integration of Hugging Face libraries with The Hub, which allows for the storage and easy retrieval of model checkpoints and configuration files. Alara explains how models are organized on The Hub, with each having its own folder and class structure. She introduces the concept of auto classes, which simplify the loading of models and their associated tokenizers by only requiring the repository name. The tutorial demonstrates how to load a text classification model trained to predict emoji labels from tweets, using the from_pretrained method for convenience.

Alara introduces the flan T5 base model by Google, a multilingual text-to-text generation model suitable for tasks like translation, question answering, and text completion. She explains the process of preparing input text for the model, including specifying source and target languages for translation. The tutorial covers the use of tokenizers to convert raw text into token IDs and attention masks, which are then used as input for the model. Alara demonstrates how to preprocess text and perform inference using the flan T5 model to translate an English sentence into German.

The tutorial shifts focus to the Hugging Face datasets library, which simplifies the process of discovering and loading datasets from The Hub. Alara introduces a fashion image captioning dataset with 100 samples, each containing an image and a corresponding text caption. She demonstrates how to load this dataset using the datasets library and explores its structure, showing how to access and visualize individual data samples. The paragraph also covers the option to load specific subsets of a dataset, such as only the training set, for more tailored data usage.

Alara introduces the BLIP model by Salesforce, an image captioning model that is not a language model but a multimodal model under the conditional generation class. She explains the need to import the BLIP processor for preprocessing images and the model for generating captions. The tutorial demonstrates how to preprocess an image, perform inference to generate token IDs, and decode these tokens into a human-readable caption. Alara also discusses the quality of the generated captions and how they can vary depending on the use case.

The tutorial concludes with creating a mapping function to preprocess and generate new captions for all samples in the dataset. Alara demonstrates how to use the mapping method of the datasets library to apply this function to the entire dataset. She then guides users through the process of pushing the updated dataset to The Hub, requiring a Hugging Face account and token. The paragraph covers the steps to log in to The Hub using the huggingface_hub library and the push_to_hub method to upload the dataset, allowing others to access and experiment with it.