12-08, 13:30–14:00 (UTC), LLM Track
As we descend from the peak of the hype cycle around Large Language Models (LLMs), chat-based document interrogation systems have emerged as a high value practical use case. The ability to ask natural language questions and get relevant answers from a large corpus of documents has the potential to fundamentally transform organizations and make institutional knowledge accessible.
Retrieval-augmented generation (RAG) is a technique to make foundational LLMs more powerful and accurate, and a leading way to implement a personal or company-level chat-based document interrogation system. In this talk, we’ll understand RAG by creating a personal chat application. We’ll use a new OSS project called Ragna that provides a friendly Python and REST API, designed for this particular case. We’ll also demonstrate a web application that leverages the REST API built with Panel–a powerful OSS Python application development framework.
By the end of this talk, you will have an understanding of the fundamental components that form a RAG model as well as exposure to open source tools that can help you or your organization explore and build on your own applications.
Foundational LLM models like OpenAI’s GPT4 provide powerful capabilities but using them directly to answer questions accurately about a collection of documents is hard. Retrieval-augmented generation (RAG) is the leading approach to enhance the capabilities and usability of Large Language Models. It can provide more accurate results to questions by augmenting the LLM with relevant external information retrieved from vector storage.
This talk hopes to enable you to quickly understand core concepts of this approach by building RAG-powered projects with a new OSS tool called Ragna. We will:
- Understand RAGs and how they interact with LLMs
- Discuss OSS projects in this area align with their capabilities and challenges
- Experiment with generating answers based on your uploaded documents, in a familiar Jupyter Notebook and using a new project called Ragna
- Explore how choice of RAG components can affect the accuracy of answers
- Create a UI to build on the experiment, with Panel and Ragna’s REST API
Beginner-level understanding of Python is sufficient to follow along, and some prior knowledge of Panel is nice to have. This talk is designed for enthusiasts in our community, to explore an interesting topic using some beginner-friendly tools.
No previous knowledge expected
Philip is a Senior Software Engineer at Quansight.