Fine-tuning

Why fine-tuning exists

Pre-trained LLMs like GPT-4, Claude, Gemini, and Llama are trained on vast internet text. They are generalists: good at many things, optimized for none. Fine-tuning exists because some use cases need the model to behave differently than its default—writing in a specific voice, following a particular format, knowing domain-specific terminology, or making decisions according to organization-specific rules. Rather than training from scratch (which can cost millions of dollars and take months), fine-tuning takes an existing model and adjusts its weights using a much smaller dataset, typically thousands to tens of thousands of examples.

Three families of fine-tuning methods

Modern fine-tuning falls into three categories, each solving a different problem:

1. Supervised Fine-Tuning (SFT): The most straightforward approach. You provide the model with labeled input-output pairs (e.g., a question and its ideal answer) and train it to produce the desired output. SFT is the standard entry point for adapting a model to a specific task, domain, or writing style. It works well when you have a clear “right answer” format and high-quality training examples.

2. Parameter-Efficient Fine-Tuning (PEFT): Methods like LoRA (Low-Rank Adaptation) and QLoRA modify only a small fraction of the model’s parameters—typically around 1%—while keeping the rest frozen. This dramatically reduces compute and memory requirements. QLoRA, which quantizes the base model to 4-bit precision and trains lightweight adapters on top, has become the default starting point for teams fine-tuning on a single GPU. Full fine-tuning of all parameters is rarely necessary for most use cases in 2026.

3. Alignment via Reinforcement Learning: These methods train the model to prefer certain outputs over others, typically to make it more helpful, less harmful, or more consistent with human preferences. Key approaches include:

• RLHF (Reinforcement Learning from Human Feedback): Trains a separate reward model on human preference data, then optimizes the LLM against that reward model. Powerful but complex and sometimes unstable.
• DPO (Direct Preference Optimization): Skips the explicit reward model and derives a policy update directly from preference pairs. Simpler, more stable, and now the standard alignment method.
• GRPO (Group Relative Policy Optimization): Samples multiple completions per prompt and uses within-group performance (e.g., correctness on math problems, passing unit tests) as rewards. Effective for reasoning and code tasks where the “right answer” is verifiable.

When to fine-tune (and when not to)

Fine-tuning is not always the right choice. The decision framework in 2026 generally looks like this:

Start with prompting and RAG first. Modern base models are remarkably capable out of the box. Strong prompt engineering and retrieval-augmented generation (RAG)—where the model fetches relevant information from external sources at query time—handle most use cases without any training.

Fine-tune when:

• Your domain is materially different from the base model’s training data (specialized legal, medical, or technical fields)
• You need consistent structured output (specific JSON schemas, code formats, report templates)
• You are targeting a language where base model performance is weak
• You have high-volume inference where a smaller fine-tuned model meaningfully reduces per-query cost and latency
• You have specific, measurable failures that prompting and RAG cannot fix

Do not fine-tune when:

• You are still defining your product requirements (fine-tuning locks in assumptions early)
• The task mainly needs up-to-date information (RAG is better)
• You are early in prototyping and iteration speed matters more than peak performance
• A new frontier model release might solve your problem next month

Costs and practical considerations

Fine-tuning has both direct and indirect costs:

Direct costs:

• Compute (GPU hours): Full fine-tuning of a large model can cost thousands of dollars per run. LoRA/QLoRA reduces this by an order of magnitude.
• Data preparation: Curating, labeling, and quality-checking training data is often the most time-consuming part of the process.
• Evaluation: You need a held-out test set and systematic evaluation process to verify the fine-tuned model actually improves on your target metrics.

Indirect costs:

• Maintenance: When the base model provider releases an update, your fine-tuned model may need retraining.
• Technical debt: A custom model variant must be hosted, versioned, and monitored separately from the base model.
• Obsolescence risk: A new base model release can leapfrog your fine-tuned version, making the investment partially or fully sunk.

Cost savings potential: At high inference volumes, a smaller fine-tuned model can reduce per-query cost by roughly an order of magnitude compared to calling a large general-purpose model via API.

Risks

Fine-tuning introduces several risks that teams should understand before investing:

• Overfitting: The model memorizes the training data instead of learning generalizable patterns. This is especially common with small training sets.
• Catastrophic forgetting: The model loses some of its original capabilities as it adapts to the new data. It might excel at your specific task but degrade on general reasoning.
• Distribution shift: The fine-tuned model performs well on inputs similar to its training data but poorly on inputs that differ—even slightly.
• Model obsolescence: The LLM landscape moves fast. A fine-tuned model that was state-of-the-art six months ago may be outperformed by a new base model release, making prior investment partially or fully sunk.
• Premature optimization: Fine-tuning early in product development can lock in assumptions about what the model should do before those assumptions are tested with real users.

Worked example: Fine-tuning a customer support agent

A SaaS company wants an AI agent to handle customer support tickets. Here is how the fine-tuning decision might unfold:

1. Start with prompting: The team writes a detailed system prompt with the company’s tone of voice, common issue categories, and escalation rules. GPT-4o handles 70% of tickets accurately.
2. Add RAG: The team connects the model to their knowledge base—product documentation, past tickets, policy documents. Accuracy improves to 82%.
3. Identify the gap: The remaining 18% failure rate clusters around three patterns: the model occasionally contradicts the company’s refund policy, uses inconsistent formatting in multi-step troubleshooting guides, and sometimes hallucinates product features that don’t exist.
4. Fine-tune with LoRA: The team curates 3,000 high-quality support interactions—questions paired with ideal responses—and fine-tunes a smaller model (Llama 3 70B) using QLoRA. Cost: approximately $50-100 in compute for a single training run.
5. Evaluate: The fine-tuned model’s accuracy on the held-out test set improves to 91%. Per-query inference cost drops by 60% because the smaller fine-tuned model replaces API calls to the larger base model.
6. Plan for maintenance: The team sets up a quarterly evaluation cycle—rerunning the test set against the latest base model to check whether a new release has closed the gap, potentially making the fine-tune unnecessary.

The 2026 recommended workflow

The widely recommended approach for teams considering fine-tuning in 2026 follows this sequence:

1. Start with strong prompting and few-shot examples on the best available base model.
2. Add RAG if the task needs proprietary or real-time data.
3. Invest in an evaluation harness before any fine-tuning—you need to measure whether fine-tuning actually helps.
4. Fine-tune (preferably with LoRA/QLoRA) only when evaluations show a persistent, measurable gap that prompting and RAG cannot close, and when inference volume justifies the engineering investment.
5. Plan for retraining as base models evolve—assume your fine-tune has a shelf life.

Common Questions

Is fine-tuning the same as training a model from scratch?

No. Training from scratch means building a model’s language understanding from raw text, which requires massive compute (millions of dollars) and huge datasets. Fine-tuning starts from a pre-trained model that already understands language and adjusts it with a much smaller dataset for a specific purpose. The difference is roughly analogous to building a car from raw materials versus modifying an existing car for a specific race.

How much data do I need to fine-tune a model?

It depends on the task and method, but practical fine-tuning datasets typically range from a few hundred to tens of thousands of examples. For simple style adaptation, 500-1,000 high-quality examples can be sufficient. For complex domain-specific tasks, teams often use 5,000-50,000 examples. Quality matters far more than quantity—a small, carefully curated dataset outperforms a large, noisy one.

Can I fine-tune a model to forget certain things?

Technically, there are methods for machine unlearning, but reliably making a model “forget” specific knowledge while preserving everything else remains an open research problem. Fine-tuning is better at adding behavior than removing it. If your goal is to prevent the model from using certain knowledge, guardrails and output filtering are more reliable than fine-tuning.

What is the difference between fine-tuning and RAG?

Fine-tuning changes the model’s internal weights—it permanently alters how the model generates responses. RAG (Retrieval-Augmented Generation) does not change the model at all; it injects relevant external information into the model’s context window at query time. Fine-tuning is better for changing behavior, style, and output format. RAG is better for incorporating up-to-date or proprietary information. Many production systems use both.