To create a list of frequent words:

./replaceunknown.pl wsj.0-18 > freqwords

To create the ngrams:

./replaceunknown2.pl wsj.0-18 freqwords | ./ngram.pl | sort | uniq -c | sort -nr > poscounts

