What does “tokenisation” mean?
-
noun
The process of turning text into tokens for computer analysis.
The tokenisation of the document helped the program analyze the text.
-
noun
The practice of replacing sensitive data with non-sensitive tokens to protect privacy.
Data security uses tokenisation to hide real card numbers.
Word forms
tokenisations
Often used as
- tokenisation process in NLP
- tokenisation of text data
- tokenisation step is required
- tokenisation method used today
- security tokenisation replaces sensitive data
- tokenisation helps protect privacy
Related words and nuances
- segmentation
- processing
- parsing
- encoding
- security
- analysis