Unveiling Retriever Augmented Generation

‍

This new approach, called Retriever Augmented Generation (RAG), unlocks new capabilities for conversational AI. By combining neural retrieval with conditional generation, RAG enables chatbots to provide more relevant, factual responses about external knowledge. This technique shows promise for creating domain specialized digital assistants that can have natural conversations and provide helpful information to users.

So imagine that I want to chat and ask questions about my fitness plan. There are two main methods for incorporating this external knowledge into a large language model (LLM):

• Fine-tuning, which involves further training the LLM on domain-specific data. A downside is that the model's knowledge becomes opaque - it's unclear what is learned versus hallucinated. Fine-tuned models may generate more false information. Another hard problem is how you can transform your knowledge base into a digestible format for model training.
• RAG pipelines query external knowledge sources to provide relevant context to the LLM. This approach is more interpretable and flexible. Knowledge can be added or removed rapidly without retraining. RAG enables granular access controls and avoids training costs.

In a RAG pipeline, given an input query, relevant contexts are retrieved from knowledge sources using semantic search, lexical search or both. These contexts are provided to the LLM in the prompt or directly in the model activations to guide its response. Overall, RAG provides a transparent and customizable approach to integrate external knowledge into LLMs.

‍

How would we do this in ChatFAQ?

‍

Once deployed ChatFAQ using our docker container, it would be as simple as running the following command from our command line interface:

Then, you are ready to open your chatbot and talk about you fitness plan!

‍

📚 If you're eager to explore the technical details, you can check out the first scientific article on RAG: https://arxiv.org/abs/2005.11401

‍