Search

In this paper, methods of feature selection used in the spam filtering are studied, including CHI square (CHI), Expected Cross Entropy (ECE), the Weight of Evidence for Text (WET) and Information Gain (IG) and a novel modified CHI feature selection method is proposed in spam filtering. The spam filter combined Support Vector Machine (SVM) is selected to evaluate the CHI square, Expected Cross Entropy, the Weight of Evidence for Text, Information Gain and modified CHI. The experiment proved that the modified CHI could improve the precision, recall and F test measure of spam filter and the modified CHI feature selection method is effective.

The problem of ambiguous word sense poses lots of difficulties for language automatic understanding. And the research of word sense discrimination is applied to the resolution of this problem. Statistics are booming in researching this problem. While due to limitation of the scale of training corpus, the method of Statistical word sense discrimination can not attain satisfying results yet. Therefore, under condition that only limit scale corpus is available, how to improve the efficiency and effectiveness of statistical learning method is a hotspot in supervised word sense recognition research. On the basis of the concept of word sense Category, a new word sense discrimination method using word sense Category Extending is proposed. Experiment results show that the proposed method can effectively improve the accuracy of word sense discrimination as the training corpus is not enlarged..