How do transformers handle variable-length inputs?
Transformers handle variable-length inputs through the use of padding and attention masks. During preprocessing, shorter input sequences are padded to the length of the longest sequence in a batch, ensuring uniformity. Attention masks are then used during training to instruct the model to ignore these padding tokens so they do not influence the output, enabling effective processing of inputs of varying lengths.