语料库中token是什么意思

[桃子伏特加628]

如题，请教语料库中token一词的含义，最好有英文解释

 

[李亮1975重庆]

useful links for glossary-driven entrance to corpus linguistics

http://appling.kent.edu/resources/glossary/corpus-glossary.cfm
http://wmtang.org/corpus-linguistics/glossary-of-corpus-linguistic-terms/
http://www.essex.ac.uk/linguistics/external/clmt/w3c/corpus_ling/content/glossary.html
http://www.cl2011.org.uk/corpus-linguistics-terms-and-their-meanings.html
http://language.worldofcomputing.net/nlp-glossary/glossary-of-nlp-terms.html