Search

In this paper, we propose a graph-based dependency parsing model using dynamic features. The dynamic features, which are derived from the partial tree during bottom-up chart parsing, play an important role in finding the correct heads. Based on the beneficial aspects of the head-final property of Korean, we suggest a variant of the CYK parsing algorithm with an O(n3) complexity which has the ability to search for the maximum spanning tree (MST) from all projective trees. Compared to Eisner's algorithm, this algorithm is time-efficient in parsing all other head-final languages. We also explain why the graph-based approaches are seldom attempted in parsing morphologically rich languages (MRLs) and suggest a morpheme-level dependency representation as a solution. Finally, we evaluate the proposed method on the KAIST corpus using two discriminative online learning methods (the averaged perceptron algorithm and passive-aggressive algorithm) and show that the trained parser achieves the best reported parsing accuracy.

This paper proposes a method that automatically creates a sentiment lexicon in a new language using a sentiment lexicon in a resource–rich language with only a bilingual dictionary. We resolve some of the difficulties in selecting appropriate senses when translating lexicon, and present a framework that sequentially applies an iterative link analysis algorithm to enhance the quality of lexicons of both the source and target languages. The experimental results have empirically shown to improve the sentiment lexicon in the source language as well as create a good quality lexicon in the new language.

Bilingual dictionary construction is a time-consuming job; therefore many studies have recently focused on automatically constructing bilingual dictionaries from bilingual texts. In this paper, we propose two novel approaches called dynamic window and tokenizer based on statistical machine transliteration model to efficiently extract English-Chinese transliteration pairs from parallel corpora. Our approaches achieve a significant improvement over previous methods without using resources such as a Chinese segmentor, pronunciation dictionaries; hence these approaches can be readily applied to other language pairs.

Centering Theory assumes that entities realized in an utterance can be ranked according to their relative salience degree. This ranking called Cf-ranking (ranking of forward-looking centers) determines the likelihood that the entities realized in an utterance will be the center of the subsequent utterance, and it is one of the central issues in Centering literatures. This paper deals with these Cf-ranking issues in Korean at the level of text structuring and pronominalization for coherent text generation. For text structuring, we compare several Cf-ranking methods by examining various Centering-based metrics to evaluate local coherence of text, and for pronominalization, we compare them by examining previous rules for Centering-based pronoun generation rules and our pronominalization model. In almost all previous works, surface word order was not solely employed for Cf-ranking, instead it was additionally considered to supplement the main ranking scheme based on the fact that linear order does not perform well alone. However, this study shows that due to the characteristics of the Korean language, ranking by surface word order is better than any other ranking method in most Centering-based Metrics which depend on Cf-ranking, and it is also reliable in terms of pronominalization accuracy. Additionally we found that based on the Cf-ranking by surface word order, it is the most effective way for text structuring to maximize simply the number of utterance pairs whose first realized nominal entity in adjacent utterances is identical.

Translation ambiguity is a major problem in dictionary-based cross-language information retrieval. To attack the problem, indirect disambiguation approaches, which do not explicitly resolve translation ambiguity, rely on query-structuring techniques such as a structured Boolean model and Pirkola's method. Direct disambiguation approaches try to assign translation probabilities to translation equivalents, normally by employing co-occurrence statistics of target language terms from target documents as disambiguation clues. Thus far, translation probabilities have not been well explored in terms of statistical query translation models, query formulation, or cross-lingual retrieval models, etc. In order to study the impact of translation probabilities on retrieval effectiveness in direct disambiguation approaches, this paper empirically investigates the following issues: different disambiguation factors affecting the calculation of translation probabilities, the comparison of cross-lingual query formulation techniques involving translation probabilities, the relationship between the accuracy of translation disambiguation and retrieval effectiveness, and the relationship between top n translations and retrieval effectiveness.

When a dependency parser analyzes long sentences with fewer subjects than predicates, it is difficult to recognize which predicate governs which subject. To handle such syntactic ambiguity between subjects and predicates, we define a "S(ubject)-clause" as a group of words containing several predicates and their common subject. This paper proposes a method to segment S-clauses and perform syntactic analysis of long sentences using S-clause segmentation. To segment S-clauses, various machine learning methods were applied. We found that the Logitboost method produced the best performance. In our experimental evaluation, S-clause information turned out to be effective in determining the governor of a subject and that of a predicate in dependency parsing. Further syntactic analysis using S-clauses achieved an improvement in precision by 5.25 percent.

Recent MT paradigms, such as example-based MT, pattern-based MT, and hybrid MT contribute to overcoming the shortcoming of rule-based MT by improving the quality of translations. However, a problem with erroneous selection of examples or patterns also exists in these types of MTs. In this paper, we discover a matching abnormality, which we term an Over-Specification Problem that occurs when an input sentence corresponding to the pattern of general sense is actually matched to the patterns of exceptional sense, and thus is incorrectly translated. We formally define the problem and propose a solution based on the similarity of dependency trees and lexical association ratio to detect whether the input tree is dragged into the exceptional range of example trees or not. Finally, promising experimental results are provided. We take a hybrid MT as a testing MT and Chinese-to-Korean as a sample language pair.

We propose a new prototype Chinese-to-Korean machine translation system called TOTAL-CK, a prototype transfer based MT system designed for the large-scale practical domain. TOTAL-CK consists of the components of analysis, transfer, and generation. In this paper, we mainly discuss the transfer issues resulting from stylistic structural differences between Chinese and Korean. The dependency grammar formalism is employed for Chinese parsed trees and their equivalent Korean parsed trees. We deal with structural transfer ambiguities that arise when a given source language syntactic pattern is potentially translated by a number of different target language syntactic patterns. Also, structural transfer ambiguities are the major complex transfer problem appearing during dependency transferring, in an SVO pattern that most frequently occurs in Chinese sentences. There are three types of structural transfer ambiguity: transitive transfer, subject-governing transfer and verb-governing transfer. They are discovered by analyzing the ambiguity and the related formalism. Finally, we present a thesaurus-based disambiguation approach for dependency structural transfer, which is adopted in TOTAL-CK. Our approach achieves 94.6% in precision for SVO sentences and 93.5% for overall sentences.

This paper introduces a mid-depth text understanding methodology. By mid-depth we mean that the methodology is: (1) based on shallow text understanding, (2) accomplished with currently available linguistic instruments, and (3) focused towards understanding texts as deeply as possible. To realize the methodology, we set up two new linguistic devices for sentence extraction — sentence abstraction and abductive chains. Sentence abstraction is processing for sentence revision with concept abstraction. Abductive chains are lexical chains with abduction. The chain consists of abductive links which connect coherent parts of abstracted sentences. It aims to identify the causal propensity described or implied in the text as well as to delineate its recurring concepts. With this methodology, a preliminary attempt at mid-depth text understanding is made to locate the topic sentences of a given text. We expect that this methodology will lead us to develop another practical text summarization system.

Due to grammatical similarities, even a one-to-one mapping between Korean and Japanese words (or morphemes) can usually result in a high quality Korean-to-Japanese machine translation. However, multi-word translation units (MWTU), such as idioms, compound words, etc., require n-to-m mapping. Often, the component words of such units do not appear adjacently, which results in a discontinuous MWTU. During translation, the MWTU should be treated as one lexical item rather than as a phrase. In this paper, we investigate units which should be treated as MWTUs in a Korean-to-Japanese MT system, examine discontinuity between their component morphemes, and propose a representation and recognition method, depending on their characteristics. In an experimental evaluation, the proposed method is very effective for handling MWTUs, showing high recall and precision of 99.8% and 99.5% and a fast recognition time.

This paper describes how to generate high quality Korean sentences from intermediate meaning representations. In previous research, there is no clear-cut separation between syntactic and morphological processing, and lexical information and rules are so tightly combined. This creates difficulties, such as attempting to enhance portability and extensibility, or handling complex linguistic phenomena in a systematic manner. We adopted Mel'cuk's meaning-text model as a linguistic model in Korean generation part, in which the complicated task of Korean generation can be broken up into logically independent subtasks, making the system highly modularized and robust. In a Korean generation experiment, the system showed a fidelity rate of 90% and an intelligibility rate of 85%, which is a promising result considering the difficulties with generating various linguistic phenomena.

As more and more speech systems require linguistic knowledge to accommodate various levels of applications, corpora that are tagged with linguistic annotations as well as signal-level annotations are highly recommended for the development of today's speech systems. Among the linguistic annotations, POS (part-of-speech) tag annotations are indispensable in speech corpora for most modern spoken language applications of morphologically complex agglutinative languages such as Korean. Considering the above demands, we have developed a single unified speech corpus annotation tool that enables corpus builders to link linguistic annotations to signal-level annotations using a morphological analyzer and a POS tagger as basic morpheme-based linguistic engines. Our tool integrates a syntactic analyzer, phrase break detector, grapheme-to-phoneme converter and automatic phonetic aligner together. Each engine automatically annotates its own linguistic and signal knowledge, and interacts with the corpus developers to revise and correct the annotations on demand. All the linguistic/phonetic engines were developed and merged with an interactive visualization tool in a client-server network communication model. The corpora that can be constructed using our annotation tool are multi-purpose and applicable to both speech recognition and text-to-speech (TTS) systems. Finally, since the linguistic and signal processing engines and user interactive visualization tool are implemented within a client-server model, the system loads can be reasonably distributed over several machines.

This paper describes a method for lexical transfer ambiguity resolution using corpus and concept information. Previous research has restricted the use of linguistic knowledge to the lexical level. Since the extracted knowledge is stored in words themselves, these methods require a large amount of space with a low recall rate. On the contrary, we resolve word sense ambiguity by using concept co-occurrence information extracted from an automatically sense-tagged corpus. The tested accuracy of our method exceeds 82.4% for nominal words, and 83% for verbal words.