E-Locus - Institutional Repository of the University of Crete

Home Collections School/Department School of Sciences and Engineering Department of Computer Science Post-graduate theses

Post-graduate theses

Current Record: 350 of 824

[Add to Basket]

Identifier

000370045

Title

Design and Implementation of a Scalable Storage System for Fully-Consistent Replicated Data Logging

Alternative Title

Σχεδιασμός και υλοποίηση ενός πλήρως-συνεπούς κλιμακώσιμου συστήματος αποθήκευσης για δεδομένα συνεχούς ροής

Author

Σαλούστρος, Γεώργιος Εμμανουήλ

Thesis advisor

Μπίλας, Άγγελος

Abstract

The rapid growth of Internet-scale applications built over commodity hard- ware for cost-efficiency has created the challenge for IT architects to design storage systems that are scalable and highly available. In addition, an in- creasingly competitive scalable software market raises the need for rapid pro- totyping of such systems. In this thesis we show a new approach for rapidly improving the robustness of an existing scalable file system, by designing and implementing a strongly-consistent replication primitive as a re-targetable building block that we then retrofit with minimal engineering effort into the scalable file system. Our replication primitive (ZKRL) extends a general- purpose open-source coordination service (Apache Zookeeper) to maintain consistent data replicas across read and append operations. By cleanly re- placing the ad-hoc replication protocol of the Hadoop scalable file system (HDFS) we achieve stronger overall data replication semantics. The result- ing system thus becomes a candidate for a wider range of applications than previously possible. This thesis further shows that this improvement need not incur a performance penalty: Our results show that we can achieve per- formance close to peak I/O (network or disk) capabilities when appending to replicated logs over 1-Gb/s and 10-Gb/s networks.

Language

English

Subject

Distributed Systems

Parallel File Systems

Replications

Scalability

Storage Systems

Αντίγραφα

Κλιμακωσιμότητα

Παράλληλα συστήματα αρχείων

Συστήματα αποθήκευσης