Högskolan i Skövde

his.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • apa-cv
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
A comparative study of retrieval-augmented generation (RAG) and fine-tuned large language models in conversational AI
University of Skövde, School of Informatics.
2025 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesis
Abstract [en]

This thesis systematically compares Retrieval-Augmented Generation (RAG) and fine-tuning (FT) adaptation strategies for large language models (LLMs) in conversational AI, emphasizing practical enterprise deployment. Four strategies were benchmarked under reproducible conditions on the public OpenAssistant/oasst1 dataset: RAG implemented on Llama2-7B and Gemini 2.0 Flash, and fine-tuning of both models. Model performance was rigorously assessed using automated metrics (BLEU, ROUGE, METEOR, BERTScore, semantic similarity) alongside structured human evaluations conducted in collaboration with Sinch AB. Fine-tuning methods, particularly for Llama2-7B, demonstrated superior alignment with ground-truth responses and consistent brand-aligned tone, ideal for public-facing chatbots. Conversely, RAG excelled in factual accuracy and completeness, especially in dynamic domains. The results highlight complementary strengths, underscoring the importance of combined automated and human evaluation. Practically, the thesis provides actionable insights: RAG suits internal or rapidly evolving contexts, while fine-tuning excels for external communication with strict brand requirements. Future research should investigate hybrid approaches, integrating the best features of both strategies. This thesis thus bridges academic and practical deployment gaps, providing enterprise stakeholders clear guidance on adapting LLMs effectively.

Place, publisher, year, edition, pages
2025. , p. 1, 41
National Category
Information Systems, Social aspects
Identifiers
URN: urn:nbn:se:his:diva-25594OAI: oai:DiVA.org:his-25594DiVA, id: diva2:1985482
External cooperation
Sinch AB
Subject / course
Informationsteknologi
Educational program
Data Science - Master’s Programme
Supervisors
Examiners
Available from: 2025-07-24 Created: 2025-07-24 Last updated: 2025-09-29Bibliographically approved

Open Access in DiVA

fulltext(1669 kB)559 downloads
File information
File name FULLTEXT01.pdfFile size 1669 kBChecksum SHA-512
1a98f23dcc985c4dd5ec8121fac6cda42140b5a17d7371bcde95f36379d52732102af28421a15e367a9ec6612183892a69ad2fd8bfe602f074ccc10cc630b26f
Type fulltextMimetype application/pdf

By organisation
School of Informatics
Information Systems, Social aspects

Search outside of DiVA

GoogleGoogle Scholar
Total: 561 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 470 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • apa-cv
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf