Stability AI, the generative AI company behind Stable Diffusion, today announced the release of its first Japanese Language Model (LM) named Japanese StableLM Alpha, accessible via Hugging Face.
The company claims that the 7 billion-parameter general-purpose language model is currently the only best-performing publicly available LM for Japanese speakers, according to a benchmark suite against four sets of other Japanese LMs.
A commercially available model, the Japanese StableLM Base Alpha 7B, will be released under the Apache License 2.0. The model is trained on 750 billion tokens of Japanese and English text using large scale data sourced from the web.
In addition to open datasets, training data includes datasets created by Stability AI’s Japanese community, in cooperation with the Japanese team of the EleutherAI Polyglot project. Stability AI used an extension of EleutherAI’s GPT-NeoX software to train the Japanese StableLM Base Alpha 7B model.
Another model, the Japanese StableLM Instruct Alpha 7B, is created solely for research purposes and released exclusively for research use. “This model is additionally tuned to follow user instructions, and trained with Supervised Fine-tuning (SFT) using multiple open datasets,” Stability AI tweeted.
Both models were tested using EleutherAI’s Language Model Evaluation Harness on tasks like sentence classification, sentence pair classification, question answering, and sentence summarization, with an average score 54.71%. Stability AI claims that this score puts its Japanese StableLM Instruct Alpha 7B far ahead of other Japanese models.
“We are proud of our first big step towards contributing to the Japanese generative AI ecosystem,” said Meng Lee, Project Lead of Japanese StableLM.
With the release of its Japanese LM, Stability AI has beaten SoftBank to the punch of releasing language models for the Japanese market. Last Friday, SoftBank announced that it has launched a new company to research and develop homegrown Large Language Models (LLM) for the Japanese market.
Furthermore, SoftBank is allocating around 20 billion JPY (more than $140 million) to its generative AI computing platform, set to launch in the fall of this year. It’s a waiting game to determine whose Japanese Language Model will emerge triumphant in the long run.
Source: mPost
