How I Doubled Chatbot Performance with LLM Fine-Tuning (FT)
I needed to build a customer service chatbot for my company, so I tried fine-tuning with LoRA.
1. Base model: Llama 3.1 8B (good Korean support)
2. Dataset: Refined 50,000 in-house CS logs and converted them into instruction format
3. Training: Ran QLoRA on 16GB VRAM (4-bit quantization)
4. Result: Accuracy increased from 68% to 89%, and hallucinations decreased
The most effective addition was including data on 'responses to incorrect questions.'
If anyone has FT experience, please share tips on data preprocessing.
5 answers
Oh, adding data to handle incorrect questions is the key point. I haven't tried that far yet, but I should give it a shot.
QLoRA only uses 16GB? That's less than I expected. But doesn't 4-bit quantization cause some performance loss? In my experience, 8-bit was more stable.
Wow, from 68 to 89 is almost a 30 percentage point increase—that's impressive. For me, the toughest part of data cleaning was removing duplicate inquiries. How do you handle it?
Well, I think claiming double the accuracy is a bit exaggerated... I'm curious about the before-and-after evaluation criteria.
Agreed, LoRA is really good lol. I also FT'd with 8B and performance definitely improved.