Enhancing Sales Meeting Analysis with Advanced LLM-powered Pipeline
A demo RAG chatbot
01
Challenges
Large internal database
Corporate knowledge databases are often built of loads of unstructured data: files, reports, notes, etc. Most of the searching methods through these documents are ineffective as they require a lot of manual work.
Multi-level access
Depending on the companies’ internal policies, different departments, or even specific roles, may be authorized to access certain documents. When implementing generative AI to help users search for information through the knowledge base, it is crucial to indicate what pieces of information they are authorized to access, and develop a chatbot with multi-level information access.
LLM’s hallucinations
In general, large language models generate answers that are syntactically correct. If they don’t know the right answer, they return an answer that seems probable. In other words, they hallucinate. However, if we want to rely on the model’s responses, it needs to be trustworthy.
Trust
As hallucinations are a well-known problem of LLMs and many people were aware of it, eliminating it was not enough — because the users would never be sure if the model’s response is correct or just probable.
02
Solutions
Retrieval Augmented Generation (RAG)
Model returning answer with the source of the information
Proof of Concept to prove the technical feasibility of the chosen approach
Technology we used
GPT-4
Langchain
Chroma
Chainlit
Unstructured
03
Gains
The PoC proved the technical feasibility of the designed solution. When implemented, it can:
Be easily adopted thanks to a familiar interface
One of the objectives of incorporating new tools is to make work more efficient. To become a true alternative to previously used solutions, they need to be intuitive, and their onboarding time should be as short as possible. A well-known interface of a chatbot makes it possible to be used from day one by any employee.
Speed up finding information in unstructured data
Instead of searching for the right report and then looking for the information in that report, the users tell the Gen AI-powered chatbot what they need to know, and the model returns an answer with a reference to the original document — saving up hours of research work.
Help organizations tackle compliance issues
Assigning different levels of authorization to different roles or departments makes it easy to manage access to confidential information and comply with corporate policies.
