Multi-model RAG assistant with quality control (RAGAS)
A Telegram assistant running on ChatGPT and GigaChat that answers from a knowledge base and scores every answer for accuracy with RAGAS.
Task
The client wanted an AI assistant for user questions and a way to tell whether each answer was actually right, without checking it by hand. It also had to run on more than one LLM.
Solution
I built a RAG system: before replying, the assistant searches the project's knowledge base and puts the answer together from what it finds. It works with two models, ChatGPT (OpenAI) and GigaChat (Sber). Each answer then goes through RAGAS, an open-source framework that scores accuracy automatically. Users talk to the assistant in Telegram.
Results
- 90% of questions handled without a human
- Accuracy of every answer scored automatically with RAGAS
- Runs on two LLMs: ChatGPT and GigaChat
Try the demo
The demo bot speaks Russian; I can build the same in English or Turkish. Its knowledge base covers a retreat, so ask it anything a guest might want to know.