Dodging Pitfalls in Packages for Artificial Intelligence and Machine Learning.

Recent years have seen an explosion in the application of artificial intelligence and machine learning (AI/ML) to practical problems from computer vision to game playing to algorithm design. This growth has been mirrored and, in many ways, been enabled by the development and maturity of publicly-available software packages that make model building, training, and testing easier than ever. While these packages provide tremendous power and flexibility to users, and greatly facilitate learning and deploying AI/ML techniques, they and the models they provide are extremely complicated and as a result can present a number of subtle but serious pitfalls. This paper presents three examples where obscure settings or bugs in these packages dramatically changed model behavior or performance – one from a deep learning regression application, one from reinforcement learning, and one from computer vision classification. These examples illustrate the importance of thinking carefully about the results that a model is producing and carefully checking each step in its development before trusting its output. [ABSTRACT FROM AUTHOR]