E-Locus - Institutional Repository of the University of Crete

Home Collections School/Department School of Sciences and Engineering Department of Computer Science Post-graduate theses

Post-graduate theses

Current Record: 55 of 824

[Add to Basket]

Identifier

000448383

Title

Increasing the efficiency of data rebalancing in the Redis distributed key-value store

Alternative Title

Αποδοτική επαναστάθμιση δεδομένων στο κατανεμημένο σύστημα αποθήκευσης κλειδιού-τιμής Redis

Author

Νταλλαρής, Ευστράτιος Ε.

Thesis advisor

Μαγκούτης, Κώστας

Reviewer

Μπίλας, Άγγελος
Μαρκάτος, Ευάγγελος

Abstract

Data rebalancing is an important mechanism used in data-intensive distributed systems when a change in the underlying resources or imbalance in the workload (e.g., a skew in the access pattern) necessitates re-distributing data across available storage nodes. Data rebalancing requires transferring (migrating) data between nodes to ensure a more equitable allocation in terms of space and/or processing capacity. The efficiency of the data rebalancing process, in terms of speed as well as impact on application performance is important, especially when used in an on-line fashion; in this case, the rebalancing process is also known as elasticity. Fast, low-impact data rebalancing actions are desirable; however, they require careful design and implementation to achieve in a distributed key-value store. In this thesis, we study data rebalancing in a widely-deployed real-world system, Redis Cluster. Our study starts by analyzing the default data rebalancing mechanism (data path) in Redis Cluster, exposing sources of inefficiency in its design and implementation, and demonstrating their impact in measured performance in experiments under an I/O-intensive workload, the Yahoo Cloud Serving Benchmark (YCSB). We then design and implement a version of Redis Cluster that features an improved data-rebalancing path, leveraging remote direct memory access (RDMA) for low-overhead data transfers. Our results show that our improved version of Redis Cluster can transfer data to a new (joining) node in nearly wire speed (1GB/s) over a RDMA-capable network connecting a 5-node cluster, while allowing sufficient spare resources for application (YCSB) processing. Our improved version of Redis Cluster achieves significantly faster data rebalancing compared to standard (and tuned) Redis in YCSB read-only workloads while also maintaining lower CPU and memory footprint during data transfers.

Language

English

Subject

Distributed systems

RDMA

Κατανεμημένα συστήματα

Issue date

2022-07-29

Collection

School/Department--School of Sciences and Engineering--Department of Computer Science--Post-graduate theses

Type of Work--Post-graduate theses

Permanent Link