What is instruction tuning and how does it differ from standard pretraining?

Pretraining teaches a model general language and world knowledge by next-token prediction over massive unlabeled text; the result is a strong text completer but not a good assistant, it may continue a question rather than answer it. Instruction tuning is supervised fine-tuning on a curated dataset of (instruction, desired response) pairs spanning many task types, formatted as natural-language commands. This teaches the model to follow instructions and generalize to unseen tasks in a zero/few-shot manner, aligning its behavior with how users actually prompt. Datasets like FLAN, Alpaca, and Dolly popularized it. It typically precedes preference alignment (RLHF/DPO). Compared to pretraining, it uses far less data, is much cheaper, and changes behavior/format more than underlying knowledge. It's the step that turns a raw base model into a usable chat/assistant model.