A solution for companies to extract information from internally stored text data is to utilize large languages models (LLM). These has proven to work excellent for various language tasks, such as for chatbots. However, to turn an LLM into a domain specific chatbot, large amount of data and computational power is needed, making it hard for companies to fully utilize these models. A potential solution to this is retrieval augmented generation (RAG), which utilize an external knowledge database to store internal company data, converting the LLM to a domain specific model.
However, one important aspect of RAG-LLM systems is for companies to understand if these systems can provide value and are useful. To better understand this, this thesis implements a proof-of-concept system in a company for engineers to retrieve internal text data. This project demonstrated that RAG could retrieve correct information with a hit rate and mean reciprocal rank (MRR) between 85%-100%, depending on the embedding model, the LLM can provide adequate summaries of the original data with a cosine similarity and F1 BERT-score of 0.92, respectively, and generate 5 summarized at around 18 seconds. In addition to this, a user evaluation was done with 4 engineers to understand what they thought about the system. In this case, the overall feedback was positive where it was easy to use and could enhance the process of finding information. Therefore, this thesis demonstrates that RAG-LLM systems can potentially be useful for companies to utilize internally stored text data.