How do LLMs handle out-of-vocabulary (OOV) words?

LLMs handle out-of-vocabulary words primarily through subword tokenization techniques such as Byte Pair Encoding (BPE) or WordPiece. These methods break words into smaller components or subwords. When an OOV word is encountered, it is decomposed into recognizable subword units, allowing the model to generate meaningful representations even for unseen words. This approach enhances the model's ability to tackle various linguistic constructs while retaining semantic coherence.