Automatic abstracting and indexing. II. Production of indicative abstracts by application of contextual inference and syntactic coherence criteria

Together with the increasing shortage of qualified abstractors, the factors of time, cost and value have lent impetus to a trend toward the automatic generation of abstracts and indexes. This trend has caused increased emphasis to be placed on the abstract as the locus of data for automatic retrieval systems. This necessitates the creation of high quality abstracts. It is the purpose of this paper to report on the development of techniques for the automatic production of high quality abstracts from the full text of the original document. It is necessary to analyze the conditions under which various methods of sentence selection are successful, in order to develop criteria for selecting sentences to form an abstract. But clearly, an abstract can also be produced by rejecting sentences of the original which are irrelevant to the abstract. As will be seen, it is this point which is perhaps the most significant contribution of this paper. Methods of sentence selection and rejection are discussed. These include contextual inference, intersentence reference, frequency criteria, and coherency considerations. The automatic abstracting system we have developed consists basically of a dictionary, called the Word Control List, and of a set of rules for implementing certain functions specified for each WCL entry. The abstracts we have obtained so far are of sufficiently good quality to indicate that large-scale testing of the methods of the automatic abstracting system is warranted.