Post-graduate theses
Current Record: 64 of 833
|
Identifier |
000448383 |
Title |
Increasing the efficiency of data rebalancing in the Redis distributed key-value store |
Alternative Title |
Αποδοτική επαναστάθμιση δεδομένων στο κατανεμημένο σύστημα αποθήκευσης κλειδιού-τιμής Redis |
Author
|
Νταλλαρής, Ευστράτιος Ε.
|
Thesis advisor
|
Μαγκούτης, Κώστας
|
Reviewer
|
Μπίλας, Άγγελος
Μαρκάτος, Ευάγγελος
|
Abstract |
Data rebalancing is an important mechanism used in data-intensive distributed systems when a change in the underlying resources or imbalance in the workload (e.g., a skew in the access pattern) necessitates re-distributing data across available storage nodes. Data rebalancing requires transferring (migrating) data between nodes to ensure a more equitable allocation in terms of space and/or processing capacity. The efficiency of the data rebalancing process, in terms of speed as well as impact on application performance is important, especially when used in an on-line fashion; in this case, the rebalancing process is also known as elasticity. Fast, low-impact data rebalancing actions are desirable; however, they require careful design and implementation to achieve in a distributed key-value store.
In this thesis, we study data rebalancing in a widely-deployed real-world system, Redis Cluster. Our study starts by analyzing the default data rebalancing mechanism (data path) in Redis Cluster, exposing sources of inefficiency in its design and implementation, and demonstrating their impact in measured performance in experiments under an I/O-intensive workload, the Yahoo Cloud Serving Benchmark (YCSB). We then design and implement a version of Redis Cluster that features an improved data-rebalancing path, leveraging remote direct memory access (RDMA) for low-overhead data transfers. Our results show that our improved version of Redis Cluster can transfer data to a new (joining) node in nearly wire speed (1GB/s) over a RDMA-capable network connecting a 5-node cluster, while allowing sufficient spare resources for application (YCSB) processing. Our improved version of Redis Cluster achieves significantly faster data rebalancing compared to standard (and tuned) Redis in YCSB read-only workloads while also maintaining lower CPU and memory footprint during data transfers.
|
Language |
English |
Subject |
Distributed systems |
|
RDMA |
|
Κατανεμημένα συστήματα |
Issue date |
2022-07-29 |
Collection
|
School/Department--School of Sciences and Engineering--Department of Computer Science--Post-graduate theses
|
|
Type of Work--Post-graduate theses
|
Permanent Link |
https://elocus.lib.uoc.gr//dlib/d/1/0/metadata-dlib-1652974662-525887-18328.tkl
|
Views |
702 |