E-Locus - Institutional Repository of the University of Crete

Home Collections Type of Work Post-graduate theses

Post-graduate theses

Search command : Author="Στεφανίδης" And Author="Κωνσταντίνος"

Current Record: 1 of 6543

[Add to Basket]

Identifier

000463342

Title

Meta-learning synaptic plasticity rules to approximate gradient descent

Alternative Title

Μετα-μάθηση κανόνων συναπτικής πλαστικότητας για προσέγγιση της μεθόδου κατάβασης μέγιστης κλίσης

Author

Καραγεωργίου Κάνην, Χρήστος

Thesis advisor

Ποϊράζη, Παναγιώτα

Reviewer

Kording, Konrad
Froudarakis, Emmanouil

Abstract

When synaptic plasticity leads to effective learning, neurons are assigned the credit that appropriately corresponds to their specific contribution. In hierarchical neural networks (NNs), the difficulty of distinguishing between credit-related and non-credit-related activity render credit assignment a non-trivial problem. Although speculative biological solutions have been proposed, these are not as potent as the optimization algorithm most widely used in deep learning. Gradient descent (GD) changes the synapses of an artificial NN using the gradient of performance, a quantity that signifies the direction of most rapid improvement of performance. GD is successful because it efficiently optimizes performance with the least modification to irrelevant parameters of a NN. Consequently, many have proposed that the brain may use an algorithm that approximates GD. What the proposed biologically-plausible implementations of GD have in common is that they all assume that the brain uses a sufficiently simple scheme that a human scientist can readily understand and describe and thus allows for mathematical proof that the system does GD. However, there is little reason to assume that evolution might derive learning mechanisms that are easily interpretable by humans. Instead, we may expect that inhibition, bursting and reward, among many other neuronal elements, all jointly contribute to efficient GD-like credit assignment. Here, we apply meta-learning, an algorithm that can be seen as an approximation of what evolution does, on synaptic plasticity rules with many degrees of freedom and explore how GD can best be approximated. We show how our method can solve non-trivial problems better than the non-meta-learned rules with a performance comparable to the performance of GD. Our results provide insights into how vastly diverse mechanisms of physiology and plasticity may enable efficient biologically-plausible credit assignment.

Language

English

Subject

Learning

Neural networks

Neuron

Μάθηση

Νευρωνικά δίκτυα

Νευρώνας

Issue date

2024-4-17

Collection