RAG and Fine-Tuning

How does AI answer from large documents?

You ask your company chatbot, "Can I carry forward unused leave?" In seconds, it quotes the right line from a 200-page handbook. It did not read all 200 pages, so how did it find the right part?

It uses RAG, short for Retrieval-Augmented Generation. This means AI first searches your documents for the parts related to your question, then uses only those parts to write its answer.

How does that work?

The system breaks your handbook into small sections and makes them searchable, like the contents page at the front of a book. When you ask about leave, it pulls out the leave sections, including exceptions, and gives them to the AI with your question. The AI then answers from those sections.

AI also uses RAG when it searches the web. A search engine keeps a ready list of web pages, so AI uses it to find the relevant pages, reads the useful parts and answers with links.

Does RAG teach the AI anything new?

No. RAG only gives the AI information for that one answer. To teach the AI a new skill, you need a different process called fine-tuning.

Say a hospital wants AI to turn doctors' quick notes, full of medical shorthand, into proper patient reports. A general AI often misreads that shorthand. So the hospital trains it further on thousands of past notes and the reports written from them.

This adjusts the AI's weights, also called parameters, which are the values inside the AI that hold what it has learned. Fine-tuning is a more complex process than RAG. It needs a large set of good examples, technical experts, time and money, and careful testing before use.

What to remember

RAG is used when AI needs to find answers from large documents or the web, without changing the AI. Fine-tuning changes the AI's weights, so it gets better at one specific task.