Stanford HAI's Alpaca: A Game-Changing Instruction-Following Model
The Stanford Institute for Human-Centered Artificial Intelligence (HAI) has recently unveiled Alpaca, an innovative instruction-following model built on Meta AI LLaMA 7B. Utilizing OpenAI's text-da-Vinci-003, the researchers developed 52K demonstrations in a self-instruct style, which they used to train Alpaca. This model not only exhibits similar behaviors to OpenAI's text-DaVinci-003 on the self-instruct evaluation set, but it is also remarkably compact and cost-effective to reproduce.
Bridging the Budget Gap
The primary challenges of training high-quality instruction-following models on an academic budget include obtaining a strong pre-trained language model and high-quality instruction-following data. Alpaca
