Yi-1.5: True Apache 2.0 Competitor to LLAMA-3

The Ye model family has received a significant upgrade, now surpassing Long benchmarks with the release of Apache 2.0. Developed by 01 AI, a Chinese company, these models were pioneers in extending the context window of an open LLM to 200,000 tokens. They offer multimodal versions and are set to release Y-Large, which is expected to outperform the original GP4 Y Large. The release includes three models with 6 billion, 9 billion, and 34 billion parameters, all of which are upgraded versions of the original Ye models. These models have been trained on 4.1 trillion tokens and further fine-tuned on 3 million samples. Despite the context window being only 4,000 tokens, the models have the potential to expand this soon. All models are released under the Apache 2.0 license, allowing for commercial use. The models are also available in both base and chat versions, targeting different hardware segments, with the 6 billion parameter model being suitable for modern smartphones. Benchmarks show that the 9 billion parameter model outperforms others in its class, while the 34 billion parameter model competes closely with the LLaMa 370 billion model. The new release also excels in coding, math reasoning, and instruction following capabilities.

The video script details a series of tests conducted to assess the Ye model's reasoning abilities and understanding. The model was given a family scenario involving siblings and was able to provide logical responses based on the information given. It demonstrated the ability to remember details from previous statements and make accurate deductions. The model was also tested on its ability to keep track of multiple items and to understand mirror writing on a door, which it handled well. Additionally, the model's mathematical capabilities were tested with simple probability and arithmetic questions, which it answered correctly. The model's capacity to retrieve information from context was also evaluated, showing that it could provide accurate answers based on a hypothetical scientific paper provided as context. Furthermore, the model was tested on basic programming tasks, including identifying errors in a Python program and writing a function to download files from an S3 bucket, which it managed to do successfully.

The Ye model's proficiency in coding was evaluated by providing it with a Python program containing errors and asking it to identify them, which it did accurately. It was also tasked with writing HTML code for a web page featuring a button that changes the background color and displays a random joke upon click. The model provided code that partially worked; the button changed the background color, but the random joke functionality did not operate as expected, indicating a minor issue with the random number generator. Despite this, the model showed promising capabilities in basic web development and programming understanding.

The video concludes by emphasizing the impressive performance of the Ye model, especially considering its size. The main limitation highlighted is the model's limited context window of 4,000 tokens, but there is anticipation that this will be expanded to 200,000 tokens soon. The video recommends that those building LLM-based applications should test the Ye model alongside other models like LLaMa 3 or Meena to determine the best fit for their specific application. The Ye model is released under Apache 2.0, allowing for unrestricted commercial use, and the upcoming release of the Ye Large model is eagerly anticipated. The video ends by encouraging viewers to test the models and make an informed choice based on their application needs.