Angaben zum Begriff
Bevorzugte Bezeichnung
tokenisation
Definition
- process of breaking a stream of text up into words, phrases, symbols, or other meaningful elements called tokens
Oberbegriff
Quelle
- Wikipedia contributors, "Tokenization (lexical analysis)," Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=Tokenization_(lexical_analysis)&oldid=637328355 (accessed February 28, 2016).
Gehört zur Gruppe
Notation
- 48.01
URI
https://vocabs.acdh.oeaw.ac.at/dhataxonomy/Concept48.01
{{label}}
{{#each values }} {{! loop through ConceptPropertyValue objects }}
{{#if prefLabel }}
{{/if}}
{{/each}}
{{#if notation }}{{ notation }} {{/if}}{{ prefLabel }}
{{#ifDifferentLabelLang lang }} ({{ lang }}){{/ifDifferentLabelLang}}
{{#if vocabName }}
{{ vocabName }}
{{/if}}
