Comparison of the Stochastic Gradient Descent Based Optimization Techniques
Küçük Resim Yok
Tarih
2017
Yazarlar
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
Ieee
Erişim Hakkı
info:eu-repo/semantics/closedAccess
Özet
The stochastic gradual descent method (SGD) is a popular optimization technique based on updating each theta(k) parameter in the partial derivative J(theta)/partial derivative theta(k) direction to minimize / maximize the (J theta) cost function. This technique is frequently used in current artificial learning methods such as convolutional learning and automatic encoders. In this study, five different approaches (Momentum, Adagrad, Adadelta, Rmsprop ve Adam) based on SDA used in updating the theta parameters were investigated. By selecting specific test functions, the advantages and disadvantages of each approach are compared with each other in terms of the number of oscillations, the parameter update rate and the minimum cost reached. The comparison results are shown graphically.
Açıklama
2017 International Artificial Intelligence and Data Processing Symposium (IDAP) -- SEP 16-17, 2017 -- Malatya, TURKEY
Anahtar Kelimeler
Gradient Descent, Momentum, Adagrad, Adadelta, Rmsprop, Adam
Kaynak
2017 International Artificial Intelligence and Data Processing Symposium (Idap)
WoS Q Değeri
N/A
Scopus Q Değeri
N/A
