Results - Details
Search command : Author="Νικολάου"
And Author="Χριστόφορος"
Current Record: 3 of 30
|
Identifier |
000452678 |
Title |
Effect of biologically inspired activation functions on artificial neural networks |
Alternative Title |
Επιδράσεις βιολογικά εμπνευσμένων συναρτήσεων ενεργοποίησης σε τεχνητά νευρωνικά δίκτυα |
Author
|
Νούλη, Γεωργία
|
Thesis advisor
|
Ποϊράζη, Παναγιώτα
|
Reviewer
|
Νικολάου, Χριστόφορος
Παυλίδης, Παύλος
|
Abstract |
Artificial Neural Networks (ANNs) were inspired by the structure and function of
the brain. One main difference between biological neural networks and ANNs is the
activation functions used by their respective computational units. We hypothesize
that biologically plausible activation functions inspired by experimentally observed
computations of biological neurons can improve the performance and internal properties
of ANNs. In this work, we implemented functions that approximate dendritic
operations and utilized them as activation functions on two hidden layer neural networks.
Specifically, we tested functions that approximate supralinear and sublinear
dendritic non-linearities, a function accounts for a maximum activity threshold due
to biological constraints and a non-monotonic function where the response is dampened
for stronger stimuli. We show that using such activation functions can result in
significantly sparser activity in the hidden layers of ANNs than traditional activation
functions without compromising task performance. This result is more pronounced
when using the non-monotonic activation function and remains even for sparsely
connected networks. Additionally, we observed a significantly increased number of
neurons activated only for a limited number of classes. In some cases, we observed
neurons that only fire solely for a particular class, even in the first hidden layer of
fully connected ANNs, indicating that neurons that act as class identifiers for easily
distinguishable classes were also created in shallow layers of the network. Overall,
we show that ANNs using biologically inspired activation functions exhibit properties
that align with biological observations in mammalian brains, such as activity
sparsity and neural class selectivity.
|
Language |
English |
Subject |
Action potentials |
|
Artificial intelligence |
|
Dendritic activations |
|
Dendritic spikes |
|
Neuroscience |
Issue date |
2022-12-07 |
Collection
|
School/Department--School of Medicine--Department of Medicine--Post-graduate theses
|
|
Type of Work--Post-graduate theses
|
Permanent Link |
https://elocus.lib.uoc.gr//dlib/3/2/0/metadata-dlib-1671187152-124022-8631.tkl
|
Views |
371 |