E-Locus - Institutional Repository of the University of Crete

Home Search

Results - Details

Search command : Author="Τσακαλίδης" And Author="Παναγιώτης"

Current Record: 22 of 66

[Add to Basket]

Identifier

000423220

Title

Architectures of distributed deep learning on commodity clusters

Alternative Title

Αρχιτεκτονικές κατανεμημένης εμβριθούς μάθησης για συστάδες μηχανημάτων περιορισμένων πόρων

Author

Άσπρη, Μαρία Π.

Thesis advisor

Τσακαλίδης, Παναγιώτης

Thesis advisor

Παπαδοπούλη, Μαρία
Τζαγκαράκης, Γεώργιος

Abstract

For the last few years, Deep Learning, is becoming an important tool in many computational applications, having trivialized the whole pipeline of feature extraction and, as a result, replacing other popular Machine Learning algorithms. For Deep Learning to be effective though, it needs not only access to vast amounts of data, but alsο devices with high computational performance. At the same time, commodity computers, with their high availability and low cost, are a popular choice of hardware and thus widely used, both in industry as well as in academia. However, they lack not only the required space for storing large volume datasets, but alsο the computational capacity to make Deep Learning a viable choice. In order to address this issue, the solution of commodity clusters was proposed. The main goal of the present thesis is the study and application of Distributed Deep Learning techniques, through the scope of both data and model parallelization, aiming to effectively migrate Deep Learning on commodity hardware. Our objective is the best possible management of the available Cluster resources, as well as to study and exploit the impact of distributed environments on the performance of Deep Learning algorithms. We conducted experiments on a five node CPU commodity cluster, and present the results of our research in the form of two case studies on the major research fields of cosmology and remote sensing. In the first case study, we address the problem of spectroscopic redshift estimation in astronomy, through a distributed perspective. We perform data distribution techniques in order to study the performance of a Convolutional Neural Network, considering both the number of training nodes and the way the data are distributed, while quantifying their effects via the metrics of training accuracy and training loss. In the second case study, we examine a research topic in the field of remote sensing. Our aim is to effectively split a multimodal Convolutional Neural Network used for multi-class land cover classification, that has a high number of parameters. Through model splits, we succeeded in effectively sharing the load of a Neural Network between the workers of our cluster and thus optimize CPU usage. We also managed to decrease the network traffic that happens due to frequent data transfers among the machines.

Language

English

Subject

Astrophysics

Distributed systems

Remote sensing

Αστροφυσική

Εμβριθής μάθηση

Κατανεμημένα συστήματα

Τηλεπισκόπιση

Issue date

2019-07-26