1. Comparative analysis of the Adam algorithm
- Subjects
алгоÑиÑÐ¼Ñ Ð¾Ð¿ÑимизаÑии ,глÑбокое обÑÑение ,rmsprop ,neural network ,adam ,deep learning ,adagrad ,optimization algorithms ,adadelta ,nadam ,нейÑÐ¾Ð½Ð½Ð°Ñ ÑеÑÑ - Abstract
ÐÑпÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑикаÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑабоÑа бакалавÑа поÑвÑÑена иÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ Ð°Ð»Ð³Ð¾ÑиÑма опÑимизаÑии Adam и ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ñ ÐµÐ³Ð¾ пÑоизводиÑелÑноÑÑи Ñ Ð¿ÑоизводиÑелÑноÑÑÑÑ Ð´ÑÑÐ³Ð¸Ñ Ð°Ð»Ð³Ð¾ÑиÑмов опÑимизаÑии, ÑÐ°ÐºÐ¸Ñ ÐºÐ°Ðº: Nadam, AdaGrad, RMSProp, AdaDelta, импÑлÑÑ ÐеÑÑеÑова. ТÑебÑеÑÑÑ Ð²ÑÑÑниÑÑ, ÑвлÑеÑÑÑ Ð»Ð¸ Adam наиболее ÑÑÑекÑивнÑм Ð´Ð»Ñ Ð³Ð»Ñбокого обÑÑÐµÐ½Ð¸Ñ Ð½ÐµÐ¹ÑоннÑÑ ÑеÑей. ÐÑоведен ÑÑавниÑелÑнÑй анализ алгоÑиÑмов, пÑÑем Ð¸Ñ ÑеализаÑии. ÐÑÐ¾Ð²ÐµÐ´ÐµÐ½Ñ ÑеÑÑÑ Ð´Ð»Ñ ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ñ ÑазлиÑнÑÑ Ð¼ÐµÑодов опÑимизаÑии. ÐÐ°Ð½Ð½Ð°Ñ Ñема ÑвлÑеÑÑÑ Ð°ÐºÑÑалÑной, Ñак как ÑейÑÐ°Ñ Ð¼Ð°Ñинное обÑÑение и алгоÑиÑÐ¼Ñ Ð¾Ð¿ÑимизаÑии глÑбокого обÑÑÐµÐ½Ð¸Ñ Ð½Ð°Ð¸Ð±Ð¾Ð»ÐµÐµ воÑÑÑебованÑ. Также, Ñак как Adam ÑвлÑеÑÑÑ Ð¾Ð´Ð½Ð¸Ð¼ из ÑамÑÑ Ð¿Ð¾Ð¿ÑлÑÑнÑÑ ÑпоÑобов опÑимизаÑии нейÑоннÑÑ ÑеÑей важно пÑовеÑÑи иÑÑледованиÑ, по ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ñ ÐµÐ³Ð¾ Ñ Ð´ÑÑгими алгоÑиÑмами и вÑÑÐ²Ð»ÐµÐ½Ð¸Ñ Ð½Ð°Ð¸Ð±Ð¾Ð»ÐµÐµ ÑÑÑекÑивного алгоÑиÑма. Ð ÑабоÑе бÑли ÑазÑабоÑÐ°Ð½Ñ Ð¸ ÑÐµÐ°Ð»Ð¸Ð·Ð¾Ð²Ð°Ð½Ñ Ð°Ð»Ð³Ð¾ÑиÑÐ¼Ñ Ð¾Ð¿ÑимизаÑии Ñ Ð¸ÑполÑзованием библиоÑеки Keras на ÑзÑке пÑогÑаммиÑÐ¾Ð²Ð°Ð½Ð¸Ñ Python в инÑегÑиÑованной ÑÑеде ÑазÑабоÑки PyCharm. ÐÑли пÑÐ¾Ð²ÐµÐ´ÐµÐ½Ñ ÑеÑÑÑ ÑаÑÐ¿Ð¾Ð·Ð½Ð°Ð²Ð°Ð½Ð¸Ñ ÑзÑковой модели и ÑÑандаÑÑнÑй ÑеÑÑ Ð¿Ð¾ ÑаÑÐ¿Ð¾Ð·Ð½Ð°Ð²Ð°Ð½Ð¸Ñ Ð¸Ð·Ð¾Ð±Ñажений. Также бÑл пÑоведен ÑкÑпеÑÐ¸Ð¼ÐµÐ½Ñ Ð´Ð»Ñ Ð±Ð¾Ð»ÑÑей наглÑдноÑÑи ÑабоÑÑ Ð²ÑбÑаннÑÑ Ð°Ð»Ð³Ð¾ÑиÑмов: ÑеÑена задаÑа Ð½Ð°Ñ Ð¾Ð¶Ð´ÐµÐ½Ð¸Ñ Ð¼Ð¸Ð½Ð¸Ð¼Ñма ÑÑнкÑии Ð¾Ñ Ð´Ð²ÑÑ Ð¿ÐµÑеменнÑÑ . Ð¡Ð´ÐµÐ»Ð°Ð½Ñ Ð²ÑÐ²Ð¾Ð´Ñ Ð¸Ð· ÑÑавниÑелÑного анализа об ÑÑÑекÑивноÑÑи ÑаÑÑмаÑÑиваемÑÑ Ð°Ð»Ð³Ð¾ÑиÑмов., The final qualification work of the bachelor is devoted to the study of the Adam optimization algorithm and the comparison of its performance with the performance of other optimization algorithms, such as: Nadam, AdaGrad, RMSProp, AdaDelta, Nesterov's impulse. It is necessary to find out whether Adam is the most effective for deep learning of neural networks. A comparative analysis of the algorithms, by their implementation, is carried out. Tests were conducted to compare different optimization methods. This topic is relevant, since now the topic of machine learning and deep learning optimization algorithms is most in demand. And also, since Adam is one of the most popular ways to optimize neural networks, it is important to conduct research comparing it with other algorithms and identifying the most effective algorithm. In this paper, optimization algorithms were developed and implemented using the Keras library in the Python programming language in the integrated development environment PyCharm. Language model recognition tests and a standard image recognition test were performed. An experiment was also conducted to make the selected algorithms more clear: the problem of finding the minimum of a function of two variables was solved. Conclusions are drawn from the comparative analysis of the effectiveness of the algorithms under consideration.
- Published
- 2021
- Full Text
- View/download PDF