A few years ago, fine-tuning a large language model meant renting a rack of GPUs and copying billions of weights for every task. Today you can adapt a huge model on a single consumer GPU. The technique that made that leap is LoRA, and the broader family is parameter-efficient fine-tuning (PEFT). Here’s how it works.

First: what fine-tuning is, and why it’s expensive

Fine-tuning takes a pre-trained model and trains it further on smaller, task-specific data to specialise its behaviour. Full fine-tuning updates every weight in the model. That’s costly three ways:

  • Compute and memory: you need gradients and optimizer state for billions of parameters, which demands far more memory than just running the model.
  • Storage: every fine-tuned version is a full copy of the model — gigabytes per task.

For a modern LLM, that’s prohibitive for most teams. PEFT exists to avoid it.

The PEFT idea: freeze most of the model

Parameter-efficient fine-tuning freezes the pre-trained weights and trains only a small set of new parameters bolted onto the model. The base model’s knowledge stays intact; you’re just learning a small adjustment. Far less memory, far less storage, much faster.

How LoRA works

LoRA (Low-Rank Adaptation), introduced in arXiv:2106.09685, is the most widely used PEFT method. The insight: the change a task requires to a big weight matrix can be approximated by something much smaller — a low-rank update.

Instead of updating a weight matrix W directly, LoRA freezes W and learns two small matrices, A and B, whose product represents the update: the effective weight becomes W + B×A. Because the “rank” (the inner dimension shared by A and B) is tiny — often just 8 to 64 — you’re training only a few million parameters instead of billions. That cuts trainable parameters and optimizer memory by orders of magnitude, and you train much faster.

Why adapters are so practical

LoRA produces small, standalone adapter files — often just megabytes. This has lovely consequences:

  • Many tasks, one base model. Keep a separate LoRA for each customer, tone, or task, all sharing one frozen base. Storage per task collapses.
  • No inference penalty. You can merge the adapter back into W at inference time, so there’s no added latency once merged.
  • Swappable and shareable. Adapters are easy to distribute, version, and combine.

QLoRA: fine-tuning giants on one GPU

QLoRA (arXiv:2305.14314) pushed efficiency further by combining LoRA with a 4-bit quantized frozen base model. Quantization shrinks the base model’s memory footprint dramatically (storing weights at lower precision), while the LoRA adapters — kept at higher precision — do the learning. The result: you can fine-tune very large models on a single GPU with quality close to full fine-tuning. This is a big reason capable open-weight models became practical to customise for small teams.

When to reach for fine-tuning at all

Fine-tuning isn’t always the answer. The honest hierarchy:

  1. Prompting first — it solves more than people expect, instantly, with no training.
  2. RAG when the model needs knowledge it doesn’t have — grounding in your documents (see our RAG guide).
  3. Fine-tuning (LoRA) when you need consistent behaviour, style, or format the model won’t reliably follow through prompting alone.

They combine well: RAG for knowledge, a light LoRA for tone. Reach for the cheapest tool that works.

For interview-ready coverage of these trade-offs, see our LLMs & transformers interview questions and the beginner-level what is fine-tuning.