Search

To enable topical web crawling, link-context is the critical contextual information of anchor text for retrieving domain-specific resources. While some link-contexts may misguide topical web crawling and extract wrong web pages, because several relevant anchor texts become irrelevant or several irrelevant anchor texts become relevant after calculating the relevance between the link-contexts and the feature terms of the specific topic. In view of above, this paper presents a heuristic-based approach by selectively using link-context and implements DOM tree to locate the anchor text. Unlike previous crawling algorithms, which only zero in on link-context and ignore whether it is really needed or not. Our method cares both link-context and evaluating its necessity to correctly use link-context to guide topical crawling. Accordingly, our topical crawler can retrieve more relevant web pages. Experimental results indicate that this approach outperforms breadthfirst, best-first, anchor text only, link-context both in harvest rate and target recall. [ABSTRACT FROM AUTHOR]

The article focuses on a novel framework for locating Domain-Specific Deep Web entries based on ontology and crawling technique. It states that the framework consists of Web Page Classifier (WPC), Form Content Classifier (FCC) and Form Structure Classifier (FSC). It mentions that WPC works on interesting pages, which were analyzed by FSC, and identifies searchable forms that are suited for a domain given by FCC. Furthermore, the framework effectively determines Domain-Specific databases.

The article presents a study on data extraction and annotation techniques that are based from the domain-specific ontology evolution for deep web. The study has constructed a mini-ontology based from the information in query interface and query results in HTML pages, and has used constructed mini-ontology in the identification of data areas, and mapping of data annotations in data extraction. The study reveals that the evolved ontology has better performance in annotating data.

User-generated content on the Web has become an extremely valuable source for mining and analyzing user opinions on any topic. Recent years have seen an increasing body of work investigating methods to recognize favorable and unfavorable sentiments toward specific subjects from online text. However, most of these efforts focus on English and there have been very few studies on sentiment analysis of Chinese content. This paper aims to address the unique challenges posed by Chinese sentiment analysis. We propose a rule-based approach including two phases: (1) determining each sentence's sentiment based on word dependency, and (2) aggregating sentences to predict the document sentiment. We report the results of an experimental study comparing our approach with three machine learning-based approaches using two sets of Chinese articles. These results illustrate the effectiveness of our proposed method and its advantages against learning-based approaches. [ABSTRACT FROM AUTHOR]

Focused crawlers are important tools to support applications such as specialized Web portals, online searching, and Web search engines. A topic driven crawler chooses the best URLs and relevant pages to pursue during Web crawling. It is difficult to deal with irrelevant pages. This paper presents a novel focused crawler framework. In our focused crawler, we propose a method to overcome some of the limitations of dealing with the irrelevant pages. We also introduce the implementation of our focused crawler and present some important metrics and an evaluation function for ranking pages relevance. The experimental result shows that our crawler can obtain more “important” pages and has a high precision and recall value. [ABSTRACT FROM AUTHOR]

Feature selection, which can reduce the dimension of feature space without sacrificing the performance of the classifier, is an effective technique in text categorization. As many classifiers cannot deal with the features with large dimensions, the noisy, irrelevant and redundant information must be filtered from the original feature space. On this basis, a random walk algorithm based feature selection method (called RWFS) is proposed in this paper. Firstly, an optimal feature selection method (called OPFS) is used to select some features from the training set. Secondly, the redundant features are filtered by combining the random walk algorithm and a pre-determined threshold. Moreover, in order to search the optimal threshold, an improved artificial bee colony method (called IMABC) is proposed for parameter optimization. In the experiments, support vector machine (SVM) and k-Nearest Neighbor (KNN) classifiers are used on four corpuses. The experimental results show that, the proposed method is significantly superior to six typical feature selections, and can greatly reduce the dimension of vector space while guaranteeing the classification accuracy as measured by F1 measurement. [ABSTRACT FROM AUTHOR]

In order to improve the classification speed without sacrificing the email classification accuracy seriously, a novel online spam classification method is proposed. Firstly, the conceptions of term frequency based interest sets are introduced, and emails are classified by combining term frequency based interest sets and Naïve Bayes classifier. Secondly, based on the active learning theory, a novel boundary density based email classification certainty evaluating method is proposed to select and recommend emails to users for labeling by combining the user interests. Finally, the emails which are labeled and classified with the greatest possibilities are used for retraining based on the incremental learning theory. In the experiments, Support Vector Machine (SVM), Naïve Bayesian (NB) and K-Nearest Neighbors (KNN) classifiers are used on two corpuses:Trec2007 and Enron-spam. Comparing with six typical active learning based incremental learning methods, the proposed method greatly reduces the consuming time of email classification while guaranteeing the accuracy. Moreover, the proposed method brings very small sample labeling burden to the users, proving its high value on online application. [ABSTRACT FROM AUTHOR]