A Discovery Method for New Words From Mobile Product Comments.

A large number of new words in product reviews generated bymobile terminals are valuable indicators of the privacy preferences of customers. By clustering these privacy preferences, sufficient information can be collected to characterize users and provide a data basis for the research issues of privacy protection. The widespread use of mobile clients shortens the string length of the comment corpus generated by product reviews, resulting in a high repetition rate. Therefore, the effective and accurate recognition of new words is a problem that requires an urgent solution. Hence, in this paper, we propose a method for discovering new words from product comments based on Mutual Information and improved Branch Entropy. Firstly, by calculating the Co-occurrence Frequency and Mutual Information between words and adjacent words, the character strings of words after pre-processing and word segmentation are expanded left and right respectively to discover the potential word set. The candidate set of new words is obtained by means of an improved support filtering algorithm. Finally, a new word set is built by applying an improved Branch Entropy filtering algorithm and removing old words. The experimental results show that this method can accurately and effectively identify new words in product comments. [ABSTRACT FROM AUTHOR]