StabilityAI and CarperAI team has unveiled two new open-source Large Language Models (LLMs) named FreeWilly1 and FreeWilly2. These models stand out in the field of LLMs due to their enhanced reasoning capabilities.
FreeWilly1 is constructed on the LLaMA 65B model and has undergone fine-tuning with a synthetically generated dataset. FreeWilly2 is built on the LLaMA 2 70B model and exhibits performance comparable to GPT-3.5 for certain tasks. The training methodologies for these models were influenced by Microsoft’s research, as detailed in their paper titled “Orca: Progressive Learning from Complex Explanation Traces of GPT-4.” Stability AI’s approach involved prompting language models with high-quality instructions to create a dataset containing 600,000 data points. This dataset size is approximately 10% of what was used in the original Orca research. Despite this reduced dataset size, the FreeWilly models have shown exceptional performance across various benchmarks.
The data generation process involved creating 500,000 cases using a less intricate LLM model and an additional 100,000 cases with a more complex LLM model. To ensure valid comparisons, the datasets were meticulously screened to remove cases that originated from evaluation benchmarks. The effectiveness of this synthetically generated dataset is evident in the FreeWilly models’ performance, even though they were trained on a dataset only a tenth the size of the original Orca paper.
For the evaluation of these models, the researchers employed EleutherAI, supplemented with AGIEval. The findings indicate that both FreeWilly models excel in addressing challenging issues in specialized fields such as law and mathematics. They also demonstrate intricate reasoning and a keen understanding of language nuances. The CarperAI team is optimistic about the potential of these models to enhance our comprehension of spoken language and is eager to witness their innovative applications in the field of artificial intelligence.
For a comprehensive understanding of FreeWilly1 and FreeWilly2, the Reference Article and Project Page provide detailed insights.
LLaMa-2: A New Era in Public Domain Language Models
LLaMa-2 stands as the premier language model in the public domain today, paving the way for the continued evolution and deployment of Large Language Models (LLMs) across various products. Its predecessor, LLaMa-1, laid the foundation by inspiring numerous impactful projects. With the introduction of LLaMa-2, the prospects for utilization in diverse applications are even greater, especially given its provision for free commercial use.
In a recent dialogue with the BBC, Nick Clegg, a notable figure from Meta, discussed the decision to release LLMs as open-source. According to Clegg, such a move enhances the safety of these models, primarily because it facilitates in-depth research and analysis from external entities.
Some key observations from Clegg include:
Meta’s commitment to transparency and contribution to the broader community is evident in their decade-long track record. Over the last ten years, the company has made available over 1000 models, libraries, and datasets for public use. Prominent releases include React, PyTorch, and the more recent ‘Segment Anything’ model.
Source: mPost
