Zipf's Law Topics

• word occurrences
• frequent word
• brown corpus
• line segment
• single line
• 411
• vocabulary
• 1 million
• extent

Related Images

Source

Source

del.icio.us Tags

• statistics
• linguistics
• math
• zipf
• longtail
• stats
• wikipedia
• law
• mathematics
• science
• distribution
• distributions
• analytics
• language
• powerlaw
• systemdynamik
• transhumanismus
• soziale
• -
• soziosom
• $$tm
• individualittslge
• schwarm
• herde
• mememonie
• probability
• research
• tools
• theory
• nlp
• seo
• _toread
• textmining
• ir
• math-interesting

Zipf's Law, Zipf's Law

Looking for Zipf's Law?

Thus the most frequent word will occur approximately twice as often as the second most frequent word, which occurs twice as often as the fourth most frequent word, etc. For example, in the Brown Corpus "the" is the most frequently occurring word, and by itself accounts for nearly 7% of all word occurrences (69,971 out of slightly over 1 million). True to Zipf's Law, the second-place word "of" accounts for slightly over 3.5% of words (36,411 occurrences), followed by "and" (28,852). Only 135 vocabulary items are needed to account for half the Brown Corpus.

For example, "the" as described above would appear at x = log(1), y = log(69971). The data conform to Zipf's law to the extent that the plotted points appear to fall along a single line segment.

Source: Wikipedia > Zipf's Law