Post-graduate theses
Search command : Author="Ταρουδάκης"
And Author="Μιχαήλ"
Current Record: 1 of 6695
|
Identifier |
000463342 |
Title |
Meta-learning synaptic plasticity rules to approximate gradient descent |
Alternative Title |
Μετα-μάθηση κανόνων συναπτικής πλαστικότητας για προσέγγιση της μεθόδου κατάβασης μέγιστης κλίσης |
Author
|
Καραγεωργίου Κανήν, Χρήστος
|
Thesis advisor
|
Ποϊράζη, Παναγιώτα
|
Reviewer
|
Kording, Konrad
Froudarakis, Emmanouil
|
Abstract |
When synaptic plasticity leads to effective learning, neurons are assigned the credit that
appropriately corresponds to their specific contribution. In hierarchical neural networks (NNs),
the difficulty of distinguishing between credit-related and non-credit-related activity render
credit assignment a non-trivial problem. Although speculative biological solutions have been
proposed, these are not as potent as the optimization algorithm most widely used in deep
learning. Gradient descent (GD) changes the synapses of an artificial NN using the gradient of
performance, a quantity that signifies the direction of most rapid improvement of performance.
GD is successful because it efficiently optimizes performance with the least modification to
irrelevant parameters of a NN. Consequently, many have proposed that the brain may use an
algorithm that approximates GD. What the proposed biologically-plausible implementations of
GD have in common is that they all assume that the brain uses a sufficiently simple scheme
that a human scientist can readily understand and describe and thus allows for mathematical
proof that the system does GD. However, there is little reason to assume that evolution might
derive learning mechanisms that are easily interpretable by humans. Instead, we may expect
that inhibition, bursting and reward, among many other neuronal elements, all jointly
contribute to efficient GD-like credit assignment. Here, we apply meta-learning, an algorithm
that can be seen as an approximation of what evolution does, on synaptic plasticity rules with
many degrees of freedom and explore how GD can best be approximated. We show how our
method can solve non-trivial problems better than the non-meta-learned rules with a
performance comparable to the performance of GD. Our results provide insights into how vastly
diverse mechanisms of physiology and plasticity may enable efficient biologically-plausible credit
assignment.
|
Language |
English |
Subject |
Learning |
|
Neural networks |
|
Neuron |
|
Μάθηση |
|
Νευρωνικά δίκτυα |
|
Νευρώνας |
Issue date |
2024-4-17 |
Collection
|
School/Department--School of Medicine--Department of Medicine--Post-graduate theses
|
|
Type of Work--Post-graduate theses
|
Permanent Link |
https://elocus.lib.uoc.gr//dlib/0/c/8/metadata-dlib-1713262345-844945-28206.tkl
|
Views |
1917 |