Search

With the growing demand for natural language models capable of automated code summarization, the necessity for robust model evaluation metrics has increased. Current evaluation metrics such as BLEU and ROGUE-L focus on specific aspects of the model such as semantic equivalence and fail to account for other potential errors the model can make. The purpose of this study is to define new quantitative metrics developers can employ when training their models in order to improve performance across all error types as opposed to optimizing for generalized equivalence with ground truth captions. As a first step in this process, we employed a qualitative comparison of four recently proposed code summarization models that involved a rigorous manual classification of the errors each model made when compared to ground truth captions. According to our derived taxonomy of model errors, we then proceeded to derive metrics which addressed the most prevalent error categorizations. These new metrics can be used during model training as a loss function in order to optimize the performance of code summarization tasks. We hope that future research efforts will be able to build upon this work with the ultimate goal of driving the creation of more accurate and reproducible automated code summarization models., Journal of Student-Scientists' Research, Vol. 3 (2021)

As user-facing applications become more prevalent in modern society, there has been an increase in interest related to the development of efficient UX and GUI design, yet a lack of focus on ethical design practices. Perhaps one of the most damaging ethical issues currently facing users is the use of dark patterns, or interfaces designed to maliciously persuade users into taking actions benefiting the service. The goal of this project is to analyze GUIs in mobile and web applications and detect dark patterns using machine learning tools and techniques. To do this, we began by preparing a dataset of over 1500 screenshots containing dark patterns from popular shopping websites. In order to provide our machine learning models with precise information from which to learn the visual semantics of these dark patterns, we placed labels on all the instances of dark patterns within the screenshots. We plan to use this data to train a Faster-RCNN neural object detection algorithm to automatically detect the presence of dark patterns present within software GUIs while exploring the possibility of extracting textual information through Optical Character Recognition. The ultimate goal of our work is to design a browser or mobile phone plugin that can run in the background and automatically detect the presence of dark patterns when users encounter them, and provide guidance on the potential actions that can be taken., Journal of Student-Scientists' Research, Vol. 3 (2021)