How I Doubled Chatbot Performance with LLM Fine-Tuning (FT)
I needed to build a customer service chatbot for my company, so I tried fine-tuning with LoRA.
1. Base model: Llama 3.1 8B (good Korean support)
2. Dataset: Refined 50,000 in-house CS logs and converted them into instruction format
3. Training: Ran QLoRA on 16GB VRAM (4-bit quantization)
4. Result: Accuracy increased from 68% to 89%, and hallucinations decreased
The most effective addition was including data on 'responses to incorrect questions.'
If anyone has FT experience, please share tips on data preprocessing.
5 answers
Agreed, LoRA is really good lol. I also FT'd with 8B and performance definitely improved.
Well, I think claiming double the accuracy is a bit exaggerated... I'm curious about the before-and-after evaluation criteria.
Wow, from 68 to 89 is almost a 30 percentage point increase—that's impressive. For me, the toughest part of data cleaning was removing duplicate inquiries. How do you handle it?
Oh, adding data to handle incorrect questions is the key point. I haven't tried that far yet, but I should give it a shot.
QLoRA only uses 16GB? That's less than I expected. But doesn't 4-bit quantization cause some performance loss? In my experience, 8-bit was more stable.