E-Locus - Institutional Repository of the University of Crete

Home Search

Results - Details

Search command : Author="Νικολάου" And Author="Χριστόφορος"

Current Record: 2 of 29

[Add to Basket]

Identifier

000452678

Title

Effect of biologically inspired activation functions on artificial neural networks

Alternative Title

Επιδράσεις βιολογικά εμπνευσμένων συναρτήσεων ενεργοποίησης σε τεχνητά νευρωνικά δίκτυα

Author

Νούλη, Γεωργία

Thesis advisor

Ποϊράζη, Παναγιώτα

Reviewer

Νικολάου, Χριστόφορος
Παυλίδης, Παύλος

Abstract

Artificial Neural Networks (ANNs) were inspired by the structure and function of the brain. One main difference between biological neural networks and ANNs is the activation functions used by their respective computational units. We hypothesize that biologically plausible activation functions inspired by experimentally observed computations of biological neurons can improve the performance and internal properties of ANNs. In this work, we implemented functions that approximate dendritic operations and utilized them as activation functions on two hidden layer neural networks. Specifically, we tested functions that approximate supralinear and sublinear dendritic non-linearities, a function accounts for a maximum activity threshold due to biological constraints and a non-monotonic function where the response is dampened for stronger stimuli. We show that using such activation functions can result in significantly sparser activity in the hidden layers of ANNs than traditional activation functions without compromising task performance. This result is more pronounced when using the non-monotonic activation function and remains even for sparsely connected networks. Additionally, we observed a significantly increased number of neurons activated only for a limited number of classes. In some cases, we observed neurons that only fire solely for a particular class, even in the first hidden layer of fully connected ANNs, indicating that neurons that act as class identifiers for easily distinguishable classes were also created in shallow layers of the network. Overall, we show that ANNs using biologically inspired activation functions exhibit properties that align with biological observations in mammalian brains, such as activity sparsity and neural class selectivity.

Language

English

Subject

Action potentials

Artificial intelligence

Dendritic activations

Dendritic spikes

Neuroscience

Issue date

2022-12-07

Collection

School/Department--School of Medicine--Department of Medicine--Post-graduate theses