Your browser does not support JavaScript!

Home    Collections    Type of Work    Post-graduate theses  

Post-graduate theses

Current Record: 7 of 5933

Back to Results Previous page
Next page
Add to Basket
[Add to Basket]
Identifier 000452678
Title Effect of biologically inspired activation functions on artificial neural networks
Alternative Title Επιδράσεις βιολογικά εμπνευσμένων συναρτήσεων ενεργοποίησης σε τεχνητά νευρωνικά δίκτυα
Author Νούλη, Γεωργία
Thesis advisor Ποϊράζη, Παναγιώτα
Reviewer Νικολάου, Χριστόφορος
Παυλίδης, Παύλος
Abstract Artificial Neural Networks (ANNs) were inspired by the structure and function of the brain. One main difference between biological neural networks and ANNs is the activation functions used by their respective computational units. We hypothesize that biologically plausible activation functions inspired by experimentally observed computations of biological neurons can improve the performance and internal properties of ANNs. In this work, we implemented functions that approximate dendritic operations and utilized them as activation functions on two hidden layer neural networks. Specifically, we tested functions that approximate supralinear and sublinear dendritic non-linearities, a function accounts for a maximum activity threshold due to biological constraints and a non-monotonic function where the response is dampened for stronger stimuli. We show that using such activation functions can result in significantly sparser activity in the hidden layers of ANNs than traditional activation functions without compromising task performance. This result is more pronounced when using the non-monotonic activation function and remains even for sparsely connected networks. Additionally, we observed a significantly increased number of neurons activated only for a limited number of classes. In some cases, we observed neurons that only fire solely for a particular class, even in the first hidden layer of fully connected ANNs, indicating that neurons that act as class identifiers for easily distinguishable classes were also created in shallow layers of the network. Overall, we show that ANNs using biologically inspired activation functions exhibit properties that align with biological observations in mammalian brains, such as activity sparsity and neural class selectivity.
Language English
Subject Action potentials
Artificial intelligence
Dendritic activations
Dendritic spikes
Issue date 2022-12-07
Collection   School/Department--School of Medicine--Department of Medicine--Post-graduate theses
  Type of Work--Post-graduate theses
Permanent Link Bookmark and Share
Views 24

Digital Documents
No preview available

Download document
View document
Views : 2