Finally saw why people swear by fine-tuning instead of basic RAG for document Q&A

I spent two weeks trying to get GPT-4 to answer questions from my company's policy PDFs using RAG and it kept hallucinating page numbers, but after feeding just 30 examples into a fine-tuned model yesterday it got every single citation right on the first try, has anyone else had that kind of jump in accuracy?

2 comments

2 Comments

charles_kelly439d ago

Dang that's exactly what happened with our HR docs too.

lewis.troy9d ago

Went through that same mess last month trying to organize old employee files. Ended up with three different versions of the same form and still can't tell which one is the real one. My desk looks like a paperwork bomb went off and I'm pretty sure I'm the one who set it off.