A token is a unit tightly bound to data. It can be used to measure how much corpus is needed to train a model, and also to measure the input and output length at inference time. 1. What a token is A token can be a whole word, a subword,