Back to Search
Start Over
Reference-less Quality Estimation of Text Simplification Systems
Reference-less Quality Estimation of Text Simplification Systems
- Source :
- 1st Workshop on Automatic Text Adaptation (ATA), 1st Workshop on Automatic Text Adaptation (ATA), Nov 2018, Tilburg, Netherlands
- Publication Year :
- 2019
-
Abstract
- International audience; The evaluation of text simplification (TS) systems remains an open challenge. As the task has common points with machine translation (MT), TS is often evaluated using MT metrics such as BLEU. However, such metrics require high quality reference data, which is rarely available for TS. TS has the advantage over MT of being a monolingual task, which allows for direct comparisons to be made between the simplified text and its original version. In this paper, we compare multiple approaches to reference-less quality estimation of sentence-level text simplification systems, based on the dataset used for the QATS 2016 shared task. We distinguish three different dimensions: gram-maticality, meaning preservation and simplicity. We show that n-gram-based MT metrics such as BLEU and METEOR correlate the most with human judgment of grammaticality and meaning preservation, whereas simplicity is best evaluated by basic length-based metrics.
- Subjects :
- FOS: Computer and information sciences
Machine translation
Computer science
Text simplification
media_common.quotation_subject
02 engineering and technology
computer.software_genre
[INFO.INFO-CL]Computer Science [cs]/Computation and Language [cs.CL]
Task (project management)
03 medical and health sciences
0302 clinical medicine
0202 electrical engineering, electronic engineering, information engineering
Quality (business)
Simplicity
BLEU
media_common
Computer Science - Computation and Language
business.industry
Reference data
030221 ophthalmology & optometry
020201 artificial intelligence & image processing
Grammaticality
Artificial intelligence
business
Computation and Language (cs.CL)
computer
Natural language processing
Subjects
Details
- Language :
- English
- Database :
- OpenAIRE
- Journal :
- 1st Workshop on Automatic Text Adaptation (ATA), 1st Workshop on Automatic Text Adaptation (ATA), Nov 2018, Tilburg, Netherlands
- Accession number :
- edsair.doi.dedup.....dc47270f44e94550e8dcef574e57f794