Search

Rationale refers to the reasoning and justification behind human decisions, opinions, and beliefs. In software engineering, rationale management focuses on capturing design and requirements decisions and on organizing and reusing project knowledge. This paper takes a different view on rationale written by users in online reviews. We studied 32,414 reviews for 52 software applications in the Amazon Store. Through a grounded theory approach and peer content analysis, we investigated how users argue and justify their decisions, e.g., about upgrading, installing, or switching software applications. We also studied the occurrence frequency of rationale concepts such as issues encountered or alternatives considered in the reviews and found that assessment criteria like performance, compatibility, and usability represent the most pervasive concept. We identified a moderate positive correlation between issues and criteria and furthermore assessed the distribution of rationale concepts with respect to rating and verbosity. We found that issues tend to appear more in lower star rated reviews, while criteria, alternatives, and justifications seem to appear more in three star rated reviews. Also, reviews reporting alternatives seem to be more verbose than reviews reporting criteria. A follow-up qualitative study of sub-concepts revealed, that users also report other alternatives (e.g., alternative software provider), criteria (e.g., cost), and decisions (e.g., on rating software). We then used the truth set of manually labeled review sentences to explore how accurately we can mine rationale concepts from the reviews. We evaluated the classification algorithms Naive Bayes, Support Vector Machine, Logistic Regression, Decision Tree, Gaussian Process, Random Forest, and Multilayer Perceptron Classifier using a baseline and random configuration. Support Vector Classifier, Naive Bayes, and Logistic Regression, trained on the review metadata, syntax tree of the review text, and influential terms, achieved a precision around 80% for predicting sentences with alternatives and decisions, with top recall values of 98%. On the review level, precision was up to 13% higher with recall values reaching 99%. Using only word features, we achieved in most cases the highest precision and highest recall respectively using the Random Forest and Naive Bayes algorithm. We discuss the findings and the rationale importance for supporting deliberation in user communities and synthesizing the reviews for developers. [ABSTRACT FROM AUTHOR]

App stores like Google Play and Apple AppStore have over 3 million apps covering nearly every kind of software and service. Billions of users regularly download, use, and review these apps. Recent studies have shown that reviews written by the users represent a rich source of information for the app vendors and the developers, as they include information about bugs, ideas for new features, or documentation of released features. The majority of the reviews, however, is rather non-informative just praising the app and repeating to the star ratings in words. This paper introduces several probabilistic techniques to classify app reviews into four types: bug reports, feature requests, user experiences, and text ratings. For this, we use review metadata such as the star rating and the tense, as well as, text classification, natural language processing, and sentiment analysis techniques. We conducted a series of experiments to compare the accuracy of the techniques and compared them with simple string matching. We found that metadata alone results in a poor classification accuracy. When combined with simple text classification and natural language preprocessing of the text-particularly with bigrams and lemmatization-the classification precision for all review types got up to 88-92 % and the recall up to 90-99 %. Multiple binary classifiers outperformed single multiclass classifiers. Our results inspired the design of a review analytics tool, which should help app vendors and developers deal with the large amount of reviews, filter critical reviews, and assign them to the appropriate stakeholders. We describe the tool main features and summarize nine interviews with practitioners on how review analytics tools including ours could be used in practice. [ABSTRACT FROM AUTHOR]