Token
A chunk of text — roughly a word or part of one. Models measure input and output length in tokens.
Also known as: token, tokens
A chunk of text — roughly a word or part of one. Models measure input and output length in tokens.
Also known as: token, tokens