tokenisation

/ˌtəʊ.kə.nɪˈzeɪ.ʃən/

B2 · Upper intermediateTechnologyCommon
Explore 20,000+ words in Discover Filter by topic and level, and save words with one tap. Sign up free

What does “tokenisation” mean?

  1. noun

    The process of turning text into tokens for computer analysis.

    The tokenisation of the document helped the program analyze the text.

  2. noun

    The practice of replacing sensitive data with non-sensitive tokens to protect privacy.

    Data security uses tokenisation to hide real card numbers.

Word forms

tokenisations

Often used as

  • tokenisation process in NLP
  • tokenisation of text data
  • tokenisation step is required
  • tokenisation method used today
  • security tokenisation replaces sensitive data
  • tokenisation helps protect privacy

Related words and nuances

  • segmentation
  • processing
  • parsing
  • encoding
  • security
  • analysis

Related words

Explore more words in Discover →