Search

This paper describes the development and application of an algorithm which generates non-linear representations of English text. The algorithm uses the results of a syntactic analysis system and a set of rules which prescribe linkages to generate a graph of a sentence. The shape of these graphs corresponds to the syntax of the sentence; the labels correspond to the vocabulary of the sentence and the edge types correspond to case grammar roles. The sentence graphs can then be interconnected at common nodes and analyzed according to common edges. Preliminary experimentation has yielded promising results. It appears that the algorithm produces a representation of English text which could be quite useful in automatic language processing.

As part of a research program aimed at determining the parameters influencing the effectiveness of a message retrieval system, a collection of 10, 000 technical abstracts was indexed and retrieval experiments were conducted with them. Since part of the work involved the development and test operation of an associative retrieval system, basic data about the distribution of words and word strings were gathered in preparing the system for test and trial. These statistics were thought to be of possible interest to other workers in the field and are gathered as a series of loosely connected papers in this volume under the following groupings: Characteristics and Indexing of GE Data Base; Comparison of Manual and Machine Selected Vocabularies; Vocabulary Distribution Studies, and Studies of Content Bearing Units in Text.

For the storage of information for subsequent retrieval of desired items, two stages of analysis are essential. The first is the determination of the subject content of a given article or paper; the second is the selection of certain words, groups of words, or classification headings by which the subject content is to be represented, either directly or by a suitable coding. Some workers still look forward to the day when it will be possible for the whole of a text to be read and ‘understood’ automatically by a machine; the ‘understanding’ process has been envisaged either as a process of selection of terms by the measure of word frequency, or word‐pair frequency (adjacent terms or terms not too far separated in one sentence) in the text, or by some process of automatic linguistic analysis. Such methods appear unsuitable for several reasons: language, as normally used, is a very difficult medium for exact expression (hence the value of mathematics) and few authors write well enough to avoid all ambiguities; a human reader accustomed to the subject can easily overcome any difficulties due to poor grammar, badly expressed arguments, excess brevity or prolixity in writing and even, sometimes, actual errors; a machine can not do so. Furthermore, the content of a paper is rarely of uniform importance throughout, and it is not worth recording, for subsequent retrieval, details which are merely repetitions of matters described earlier and better elsewhere, and not essential to the main purpose of the paper; for example, in a paper on evaporator design, a description of a standard method of analysis, applied to the contents of the evaporator in determining the efficiency of the design, will not be worth indexing; in a search for analytical methods, retrieval of such a paper would hardly be considered pertinent. A human reader, though far from infallible, can usefully make such judgments.

Together with the increasing shortage of qualified abstractors, the factors of time, cost and value have lent impetus to a trend toward the automatic generation of abstracts and indexes. This trend has caused increased emphasis to be placed on the abstract as the locus of data for automatic retrieval systems. This necessitates the creation of high quality abstracts. It is the purpose of this paper to report on the development of techniques for the automatic production of high quality abstracts from the full text of the original document. It is necessary to analyze the conditions under which various methods of sentence selection are successful, in order to develop criteria for selecting sentences to form an abstract. But clearly, an abstract can also be produced by rejecting sentences of the original which are irrelevant to the abstract. As will be seen, it is this point which is perhaps the most significant contribution of this paper. Methods of sentence selection and rejection are discussed. These include contextual inference, intersentence reference, frequency criteria, and coherency considerations. The automatic abstracting system we have developed consists basically of a dictionary, called the Word Control List, and of a set of rules for implementing certain functions specified for each WCL entry. The abstracts we have obtained so far are of sufficiently good quality to indicate that large-scale testing of the methods of the automatic abstracting system is warranted.

The R2 Information Retrieval System, when complete, will be a question-answer information retrieval system in which a person, having questions over a pre-determined data base, will be able to receive answers to his questions. The paper concerns the initial processing of the data base for the R2 system and explores initially several simple strategies for retrieving relevant statements, rather than answers, for a given question over the data base. The paper is first concerned with the construction of a 'phase dictionary' from the data base, and, secondly, with the development of several initial attempts for information retrieval from the data base.

An approach to translation between natural language and an artificial language useful for information storage and retrieval is presented. The technique is based on the “functional” generative description of language elaborated by the Prague group of algebraic linguistics. This paper concentrates on the characteristics of the artificial language.

This paper examines the implications of the findings of evaluative tests regarding the retrieval performance of natural language in various subject fields. It suggests parallel investigations into the structure of natural language, with particular reference to terminology, as used in the different branches of basic science. The criteria for defining the terminological consistency of a subject are formulated and a measure suggested for determining the degree of terminological consistency.

Some new text searching retrieval techniques are described which retrieve not documents but sentences from documents and sometimes (on occasions determined by the computer) multi-sentence sequences. Since the goal of the techniques is retrieval of answer-providing documents, “answer-passages” are retrieved. An “answer-passage” is a passage which is either answer-providing or “answer-indicative,” i.e., it permits inferring that the document containing it is answer-provding. In most cases answer-sentences, i.e., single-sentence answer-passages, are retrieved. This has great advantages for screening retrieval output. Two new automatic procedures for measuring closeness of relation between clue words in a sentence are described. One approximates syntactic closeness by counting the number of intervening “syntactic joints” (roughly speaking, prepositions, conjunctions and punctuation marks) between successive clue words. The other measure uses word proximity in a new way. The two measures perform about equally well. The computer uses “enclosure” and “connector words” for determining when a multi-sentence passage should be retrieved. However, no procedure was found in this study for retrieving multi-paragraph answer-passages, which were the only answer-passages occurring in 6% of the papers. In a test of the techniques they failed to retrieve two answer-providing documents (7% of those to be retrieved) because of one multi-paragraph answer-passage and one complete failure of clue word selection. For the other answer-providing documents they retrieved at all recall levels with greater precision than SMART, which has produced the best previously reported recall-precision results. The retrieval questions (mostly from real users) and documents used in this study were from the field of information science. The results of the study are surprisingly good for retrieval in such a “soft science,” and it is reasonable to hope that in less “soft” sciences and technologies the techniques described will work even better. On this basis a dissemination and retrieval system of the near future is predicted.

This paper describes a method for automatically segmenting words into their stems and affixes. The process uses certain statistical properties of a corpus (successor and predecessor letter variety counts) to indicate where words should be divided. Consequently, this process is less reliant on human intervention than are other methods for automated stemming. The segmentation system is used to construct stem dictionaries for document classification. Information retrieval experiments are then performed using documents and queries so classified. Results show not only that this method is capable of high quality word segmentation, but also that its use in information retrieval produces results that are at least as good as those obtained using the more traditional stemming processes.

This paper describes a model of information retrieval system based on thesaurus. Definitions of: thesaurus, classification, document description, information query and response are given. Inclusiveness and two other properties of the considered system are proven.

Persons performing research in the field of information retrieval need to be concerned with likenesses as well as differences in various automated systems. This paper uses the same computer routines to compare two dictionaries of different sizes, having some similar features and some different features. It compares the use of the dictionaries in building their input file, and discusses the frequency of occurrence of descriptors and their combining power, with an analysis of the data generated.

This paper is an exhaustive description of a working program for collating texts. The procedure identifies text variants ranging from minute differences in punctuation and spelling to additions and deletions of larger portions of text. The aim of the procedure is to bring lineation of all texts to be compared into conformity with that of the base-text and to print out all variants in the texts beneath the base-text. The program accepts input in running text form from magnetic tape and does not require any editing or reorganizing of data prior to processing. Comparisons are carried out on a line for line basis, and searches for new correspondences when the texts are out of phase extend over an interval of twenty lines each side of the current line. The report also contains flowcharts and complete program listings, and fills a definite need in the area of variant identification since no other program written in a higher level language has so far been made public. Originally written in FORTRAN 32 for the CDC 3000 series computers, the program is so designed as to make the conversion to another machine a realistic enterprise. Both the problem of operation of the system and possible adaptation to other machines is discussed from the viewpoint of the user, and some suggestions for further development are offered.

This paper describes a computer system which uses a combination of coordinate indexing and structure matching techniques to extract from English questions many criteria which can be used for selecting and recognizing answers. A complete index of all content words in text is first searched to find information-rich statements which may be answers to the question. Each of these statements is then dependency analyzed to determine if the words (or synonyms) which correspond to question words maintain the dependency relations holding in the question. A simple semantic evaluation of structurally acceptable answers follows. A human editor working with the computer system helps to resolve syntactic ambiguities which are otherwise a major stumbling block in question-answering systems.

Excerpts of technical papers and magazine articles that serve the purposes of conventional abstracts have been created entirely by automatic means. In the exploratory research described, the complete text of an article in machine-readable form is scanned by an IBM 704 data-processing machine and analyzed in accordance with a standard program. Statistical information derived from word frequency and distribution is used by the machine to compute a relative measure of significance, first for individual words and then for sentences. Sentences scoring highest in significance are extracted and printed out to become the "auto-abstract."

The concept of “relevance”, sometimes also called “pertinence” or “aboutness”, is central to the theory of information retrieval. Unfortunately, however, there is at present no consensus as to how this notion should be defined. The purpose of this paper is to propose and defend a definition of what it means to say that a piece of stored information is “relevant” to the information need of a retrieval system user. The suggested definition explicates relevance in terms of logical implication. For any yes-or-no question answering system which operates with one of the standard formalized languages, the definition provides a mathematically precise criterion of relevance. For other types of fact retrieval systems and reference retrieval systems, including all systems whose stored information is expressed in natural language, the definition is not mathematically precise but is nevertheless still helpful on a conceptual level.

The paper presents a new point of view about the “relevance” concept. The logic of information retrieval is analysed (two‐valued, many‐valued and infinite‐valued logic) and the truth‐value is interpreted as the relevance value. The logical system which seems to be most appropriate to describe the process of information retrieval is the continuous logic suggested by McNoughton. In this case the answer of the information retrieval system is interpreted as a fuzzy set.

This paper describes a system for dealing with a certain kind of textual information. The system has been in operation for about one and a half years. It also indicates the nature of a new and greatly expanded system presently under development.The first system consists essentially of three parts, of which the most important is a thesaurus processor. The assumption is made that document content depends only on word content and that word relationships are defined by an hierarchical structure. The function of the thesaurus processor is to provide a simple language for developing and changing the thesaurus, whenever change is necessary. The remaining parts of the first system are a document processor for updating document files, and a search processor for batch requests which must scan the entire document file. Some statistics are given on performance of the system.The second system includes a thesaurus processor of expanded capability. It also includes a newly developed search language, which can be used to scan records for complex patterns of events. Input-output processing of records and of search results is left to the user of the system.

This paper describes the salient features of a Rome Air Development Center exploratory research effort aimed at developing new computer programs which can accomplish certain key processing functions on natural language text. The concept of an ultimate system includes a number of capabilities. First are those operations dealing with input and file search actions. Second are those operations that organize the output from the data handling system. In short, the system goal is to "cut out," automatically, the useful parts of each incoming document and then to "paste together" that cross section of document parts that forms the most relevant response to a particular file query. In addition to a description of the experimental computer programs developed thus far, two pilot studies are discussed. The first describes a test of the degree to which people could reconstitute the source documents from the fragments of three documents which were virtually identical in subject matter. The second pertains to a pilot test of some of the experimental computer programs already developed.

This paper describes a method of automatic indexing which may also be viewed as a simulation of how a person classifies documents. In this method, the sentences of a passage are isolated and converted into an intermediate predicate formalism. The verb, which names the predicate, is assigned a priority. Predicate priorities, derived primarily from the verb priorities, are used as distance indices in converting the predicates from an entire passage to a graph structure. Elementary notions from graph theory are applied to determine the descriptors of the passage. The method is illustrated with a story that has been processed automatically.

Computer-based semantic analyzers come from five basic areas: machine translation, question and answer systems, content analysis, bibliographic retrieval, and medical information retrieval systems. This paper brings together and reviews these five basic areas and looks at representative semantic analyzers from them. The diversity of these semantic analyzers makes precise comparison impossible, but it was useful to draw some general comparisons in the areas of subject matter and breadth of context.

This paper is concerned with the use of analog electrical networks for the automatic recognition of statistical word associations present in written English text. A general mathematical theory is proposed for the association of linguistic units by means of linear transformations, and it is shown that this theory can be realized through use of passive electrical networks. Small-scale and experimental associative networks have in fact been built for information retrieval applications. These devices generate measures of association among index terms used to characterize a document collection, and between the index terms and the documents themselves. Roughly speaking, measures of two distinct types of linguistic association can thus be generated-"synonymy" association, which reflects similarity of meaning, and "contiguity" association which reflects relationships among designates.

This paper is a glossary defining several of the basic terms germane to information retrieval. Data-processing terms are also explained emphasizing their relationship to problems in information retrieval.

A semantic group of words is one whose words are associated in meaning, in some way definable by the lexicographer. A thesaurus is a large collection of words, each of which is assigned to one or more semantic groups. The importance of thesauri for applications in library information storage and retrieval is evident from the number of centres interested in them. (No less than 24 centres are listed in the index of Current Research in Scientific Documentation No. 13, and thesuri in 10 disciplines are mentioned specifically.) The present paper describes aspects of an attempt to generate groups of words by purely mechanical means, which seem to have strong semantic connections within groups and which, therefore, are of potential value as thesauri. The lexicographer plays no part at all, and therefore no semantic processes can be used. However, the machine can make very many more binary decisions than a lexicographer, and there is the possibility that a machine generated “thesaurus” is at least as valuable for document processing as a man-made one.

This paper presents an overview of research in progress in which the principal aim is the achievement of more natural and expressive modes of on-line communication with complexly structured data bases. A natural-language compiler has been constructed that accepts sentences in a user-extendable English subset, produces surface and deep-structure syntactic analyses, and uses a network of concepts to construct semantic interpretations formalized as computable procedures. The procedures are evaluated by a data management system that updates, modifies, and searches data bases that can be formalized as finite models of states of affairs. The system has been designed and programmed to handle large vocabularies and large collections of facts efficiently. Plans for extending the research vehicle to interface with a deductive inference component and a voice input-output effort are briefly described.

The aim of the project was to determine how people store and retrieve factual (non-numerical) information and how they utilize this stored information in comprehending English text. Three of the studies investigated how people retrieve factual information, one evaluated possible strategies for interpreting text, and a final paper summarized conclusions about the requirements for building a computer-based, natural language-processing system. The results indicated that people use both deductive inference and inference by analogy in answering questions. The initial search for relevant facts is apparently a parallel process, while the checking of possible answers is a serial process. Depending on the information turned up by the parallel search and the constraints of syntax and context, people apply a variety of different specific decision rules in order to decide how to answer a question or how to interpret a sentence. Military operations in the future will utilize computer-based, question-answering systems that can store and retrieve factual information and that can interact with users in English. Knowledge gained from these experiments is being used in a computer project aimed toward developing such systems.