Hello. We're looking to build an in-house document-based Q&A chatbot, and I'd like advice on whether it's better to adopt fine-tuning or RAG first.
The domain is specialized, so general LLMs fall short, and we have about 10,000 pages of documents. We're currently using the GPT-4o API.
Is the standard approach to start with RAG and fine-tune only the areas where it falls short? Or, given the large volume of documents, would it be better to fine-tune from the beginning? Cost and infrastructure aren't a concern, so I'd like to hear from those with actual experience.
I, on the other hand, spent three months struggling with RAG before eventually switching to fine-tuning. When the document structure is complex and there are lots of tables and charts, the retrieval kept pulling from the wrong places. Fine-tuning was also frustrating at first, but after training on summarized versions of the documents, it turned out to be more stable. However, if your document update cycle is short, the retraining cost can be a burden, so be sure to keep that in mind.
by 지나가던행인970 · ▲0