lexical token (Q5881)
in lexical analysis, atomic string with an assigned and thus identified meaning; can be either word, bi-gram or sentence
- token
| Language | Label | Description | Also known as |
|---|---|---|---|
| English | lexical token |
in lexical analysis, atomic string with an assigned and thus identified meaning; can be either word, bi-gram or sentence |
|