What does retokenize mean?
Retokenize means (transitive, computing) To tokenize again..
Looking up this word…
Checking saved dictionary data and trusted language sources. This will stop automatically if a source does not respond.
Checking saved dictionary data and trusted language sources. This will stop automatically if a source does not respond.
/ɹiˈtoʊ.kən.aɪz/ · verb
(transitive, computing) To tokenize again..
Use retokenize when its meaning, tone and grammar fit the full sentence. A synonym is not always a direct replacement.
Retokenize means (transitive, computing) To tokenize again..
The closest synonym depends on the sentence and intended sense.
The opposite depends on the specific sense.
Use retokenize where its definition matches the intended context.
FreeDictionaryAPI.com / Wiktionary · Original source · CC BY-SA 4.0
Dictionary data is provided by FreeDictionaryAPI.com and sourced from Wiktionary under its stated license.