请大家帮我分析一下！

[ellen_bm]

range BNC 的三个基本词汇表，三级词汇表每级各1000词，共3000词，这个数量远远小于《大学英语课程教学要求》《量6674，其中一般要求4538词，较高要求1081词，更高要求1055词。

我将某套大学英语第一册书的前三单元用range 测量，结果如下：

WORD LIST TOKENS/% TYPES/% FAMILIES
one 4256/79.37 66/54.21 524
two 331/ 6.17 199/14.08 163
three 79/ 1.47 42/ 2.97 36
not in the lists 696/12.98 406/28.73 ?????

Total 5362 1413 723

我对tokens, types, families的概念有些糊涂
请哪位高手帮我分析一下，怎么总结呢？？非常感谢

 

[yamanda]

回复: 请大家帮我分析一下！

i say either and you say either.
此句中,
token (形符)有7个 i ,say ,either, and ,you, say ,either
type(类符)有5个 i ,say, either, and ,you
familiy 的例子如下:
work, works, working, worked是同一family.
我不是高手,是学习者关于这些概念前一向有人问过,许博给过很详尽的解释.我忘记在那里了.你可以找一下.

 

[yamanda]

回复: 请大家帮我分析一下！

/前是个数,/后是所占百分比,比较百分比比较有意义.not in the list是指不在三类词表中的单词.
rang 搞对比分析有意义些.最好再找个对照的文本分析出结果与这个结果做对比分析

 

[yamanda]

回复: 请大家帮我分析一下！

该讨论区的8月24日的帖子中有Dr.xu的详细解释.

 

[lilis]

回复: 请大家帮我分析一下！

我怎么也找不到Dr.xu 8月24日的帖子，哪位高手能从以下比较中说明tokens, types 和 family 的区别？回答以下问题？
1. The first raw material[FONT=宋体]①[/FONT] is one book has 12 texts, and the number of the words is 8,143.
Using Range, I’ve got the results are[FONT=宋体]：[/FONT]

WORD LIST​

TOKENS/%​

TYPES/%​

FAMILIES​

one ​

37967/99.76 ​

23/88.46 ​

23​

two ​

63/ 0.17 ​

2/ 7.69 ​

2​

three ​

0/ 0.00 ​

0/ 0.00 ​

0​

not in the lists ​

29/ 0.08 ​

1/ 3.85 ​

?????​

Total ​

38059 ​

26 ​

25​

2. The second raw material[FONT=宋体]②[/FONT] is just a paragraph taken from the first text of this book, and the number of the words is 83, but the results are different.
Also using Range, I’ve got the results are:

WORD LIST​

TOKENS/%​

TYPES/%​

FAMILIES​

one ​

74/89.16
52/85.25
47

two ​

3/
3.61
3/

three ​

2/
2.41
2/

not in the lists ​

4/
4.82

?????​

Total ​

83 ​

61 ​

52​

So the problems are:
1. What “token” and “type” mean?
2. For the second results, the number of tokens is the some as the number of the words, but for the first results, the two numbers are totally different.
3. In the second step, the small raw material[FONT=宋体]②[/FONT] comes from the big raw material[FONT=宋体]①[/FONT], that is, [FONT=宋体]②[/FONT] is a small part of[FONT=宋体]①[/FONT], but the number of types in the second results is larger than the first one.(61>26); the families also larger (52>25).

 

[清风出袖]

回复: 请大家帮我分析一下！

type是指词形。token 是词符，就是里面实际有多少单词。family指的是里面具体有的词族，比如
ABLE 0
ABILITY 0
ABLER 0
ABLEST 0
ABLY 0
ABILITIES 0
UNABLE 0
INABILITY 0
就是一个词族。