Research of the Distributed Focused Crawler for an Automotive News Push System

With the development of automobile industry, more and more people need automotive information. Meanwhile, there’s a great deal of network information which makes people gather information hard. To make people acquire the automotive information more efficiently, an Automotive News Push System is desperately needed. A distributed focused crawler is proposed for the system to assure that the information is accurate and timely. In this paper, distributed system framework and webpage classification are analyzed. Experimental results show that the frameworkare effective, the accuracy and efficiency of crawling are both improved.