What is transfer learning and when should you use it?

Transfer learning reuses a model pretrained on a large dataset as the starting point for a new, related task, rather than training from scratch. Early layers of a pretrained network learn general features (edges and textures in vision, or general language structure in NLP) that transfer across tasks. You typically take the pretrained backbone, replace the final task-specific head, and either freeze the early layers and train only the head (feature extraction) or fine-tune some or all layers at a lower learning rate. It is especially valuable when you have limited labeled data or compute, because it needs far fewer examples and training time to reach strong accuracy, and it reduces overfitting. It works best when the source and target domains are similar. This approach underpins modern practice, from ImageNet-pretrained CNNs to fine-tuning large pretrained language models.