Your browser does not support JavaScript!

Home    Scalable storage support and fault-tolerance for data stream processing systems  

Results - Details

Add to Basket
[Add to Basket]
Identifier 000358235
Title Scalable storage support and fault-tolerance for data stream processing systems
Alternative Title Κλιμακώσιμη αποθήκευση και ανοχή σφαλμάτων για συστήματα επεξεργασίας συνεχόμενης ροής δεδομένων
Author Σεμπέπου, Ζωή Ιωάννη
Thesis advisor Μπίλας, Άγγελος
Abstract Continuous data stream processing systems have offered limited support for data persistence in the past, for three main raisons: First, online, real-time queries examine current streaming data and under the assumption of no server failures, do not require access to past data. Second, most fault-tolerance solutions for streaming systems are currently memory-based, keeping state replicas in the memory of stream processing nodes. Finally, stable storage devices are commonly thought to be constraining system throughput and response times when compared to main memory, and are thus kept off the common path. In this thesis we provide data streaming systems with a scalable path to persistent storage. Our persistance path allows two fundamental enhancements to data streaming systems' capabilities: First, it allows stream persistence for reference/archival purposes. In other words, queries can now be applied an past data on-demand. Second, fault tolerance is achievable by checkpointing and stream replay schemes that are not constrained by the size of main memory. For stateful operators we present continuous eventual checkpoints (CEC), a novel mechanism to provide fault-tolerant guarantees by taking continuous state checkpoints without pausing the operator. We achieve this by separating state into parts and by treating these parts independently. Our results show that our system can achieve scalable stream persistence over a high-performance tunable paralles file system and low overhead fault-tolerance with an adjustable checkpoint inetrval, trading off recovery time with performance.
Language English
Issue date 2010-07-16
Collection   School/Department--School of Sciences and Engineering--Department of Computer Science--Post-graduate theses
  Type of Work--Post-graduate theses
Views 486

Digital Documents
No preview available

Download document
View document
Views : 13