With the meteoric rise of large language models (LLMs) like GPT-3, there has been an understandable scramble to find the best ways to tap into their vast potential.
The Latest Trend: Memory Augmentation
The latest trend in natural language processing seems to be an obsession with "adding memory" to large language models (LLMs) through retrieval augmentation techniques like RAG (Retrieval Augmented Generation). The idea is that by allowing LLMs to retrieve and incorporate external knowledge, we can enhance their already impressive capabilities even further. However, this risks overlooking the tremendous untapped potential still lying dormant within the base LLMs themselves.