Paolo Pavanelli
Optimization through control based gradient descent.
Rel. Alfredo Braunstein. Politecnico di Torino, Corso di laurea magistrale in Physics Of Complex Systems (Fisica Dei Sistemi Complessi), 2018
|
PDF (Tesi_di_laurea)
- Tesi
Licenza: Creative Commons Attribution Non-commercial No Derivatives. Download (732kB) | Preview |
|
Archive (ZIP) (Documenti_allegati)
- Other
Licenza: Creative Commons Attribution Non-commercial No Derivatives. Download (10kB) |
Abstract: |
Non-convex continuous optimization problems occur in many fields of engineering, ranging from operations research, control theory and neural network learning. The most common optimization approach are gradient based method that,however, for large optimization problems, these approaches are plagued by local minima and saddle points,that is suboptimal solutions where the gradients are zero and where optimization halts prematurely.Recently, Baldassi et al. [ https://doi.org/10.1073/pnas.1608103113] have introduced a local entropic measure for learning with discrete synapses that leads to unanticipated computational performance.Inspired by this line of work, Chaudhari [ arXiv:1611.01838] connects this idea to the solution of a Hamilton-Jacobi partial differential equation and stochastic optimal control theory, providing an algorithm to implement the descent along the gradient of the local entropy. Here we provide an algorithm based on the work on learning parametrized controllers done by Kappen [http://arxiv.org/abs/1406.4026] and to compare its performances to the one proposed by Chaudhari.We explore the validity of our method in relevant one dimensional cases and in multidimensional cases of both a convex and a non convex function, finding that in both cases the control descent gives a comparable extimate of the global minimum of the original function given the same fixed parameters and thus suggesting that this method could prove itself as a valid alternative with further reasearch. |
---|---|
Relators: | Alfredo Braunstein |
Academic year: | 2017/18 |
Publication type: | Electronic |
Number of Pages: | 28 |
Subjects: | |
Corso di laurea: | Corso di laurea magistrale in Physics Of Complex Systems (Fisica Dei Sistemi Complessi) |
Classe di laurea: | New organization > Master science > LM-44 - MATHEMATICAL MODELLING FOR ENGINEERING |
Ente in cotutela: | SNN Machine Learning - Department of Biophysics, DCN Radboud University (PAESI BASSI) |
Aziende collaboratrici: | UNSPECIFIED |
URI: | http://webthesis.biblio.polito.it/id/eprint/8026 |
Modify record (reserved for operators) |