E-Locus - Institutional Repository of the University of Crete

Home Search

Results - Details

Search command : Author="Πρατικάκης" And Author="Πολύβιος"

Current Record: 19 of 82

[Add to Basket]

Identifier

000452895

Title

Genisys, a resource management and placement mechanism for HPC and datacenter applications under Kubernetes

Alternative Title

Genisys, ένας μηχανισμός διαχείρισης πόρων και τοποθέτησης εφαρμογών υψηλών επιδόσεων κάτω από το σύστημα Kubernetes

Author

Ζέρβας, Γεώργιος Μ.

Thesis advisor

Μπίλας, Άγγελος

Reviewer

Μαγκούτης, Κωνσταντίνος
Πρατικάκης, Πολύβιος

Abstract

Today, Cloud and HPC workloads tend to use different approaches for managing resources. However, as more and more applications require a mixture of both high- performance and data processing computation, convergence of Cloud and HPC resource management is becoming a necessity. Cloud-oriented resource management strives to share physical resources across applications to improve infrastructure efficiency. On the other hand, the HPC community prefers to rely on job queueing mechanisms to coordinate among tasks, favoring dedicated use of physical resources by each application. In this work, we design a combined Slurm-Kubernetes system that is able to run unmodified HPC workloads under Kubernetes, alongside other, non-HPC applications. First, we containerize the whole HPC execution environment into a virtual cluster, giving each user a private HPC context, with common libraries and utilities built-in, like the Slurm job scheduler. Second, we design a custom Slurm-Kubernetes protocol that allows Slurm to dynamically request resources from Kubernetes. Essentially, in our system the Slurm controller delegates placement and scheduling decisions to Kubernetes, thus establishing a centralized resource management endpoint for all available resources. Third, our custom Kubernetes scheduler applies different placement policies depending on the workload type. We evaluate the performance of our system compared to statically partitioned Kubernetes and Slurm-based HPC clusters and demonstrate its ability to allow the joint execution of applications with seemingly conflicting requirements on the same infrastructure with minimal interference.

Language

English

Subject

Cloud

Docker

Genisys

Integration

Scheduler

Slurm

Virtual-clusters

Υπολογιστικό νέφος

Issue date