Koala: A Dialogue Model for Academic Researchhttps://bair.berkeley.edu/blog/2023/04/03/koala/
Koala is a new model fine-tuned on freely available interaction data scraped from the web, with a specific focus on data that includes interaction with highly capable closed-source models such as ChatGPT. The LLaMA base model is fine-tuned on dialogue data scraped from the web and public datasets, including high-quality responses to user queries from other large language models, question answering datasets, and human feedback datasets. The resulting model, Koala-13B, shows competitive performance compared to existing models as demonstrated by human evaluation on real-world user prompts. The post suggests that learning from high-quality datasets can mitigate some of the shortcomings of smaller models and may even match the capabilities of large closed-source models in the future. This implies that the community should put more effort into curating high-quality datasets to enable safer, more factual, and more capable models instead of simply increasing the size of existing systems.