Your browser does not support JavaScript!

Home    Execution of Recursive Queries in Apache Spark  

Results - Details

Add to Basket
[Add to Basket]
Identifier 000408044
Title Execution of Recursive Queries in Apache Spark
Alternative Title Εκτέλεση αναδρομικών ερωτημάτων στο Apache Spark
Author Κατσογριδάκης, Παύλος Σ.
Select a value Μπίλας, Άγγελος
Reviewer Πρατικάκης, Πολύβιος
Φατούρου, Παναγιώτα
Abstract MapReduce environments offer great scalability by restricting the programming model to only map and reduce operators. This abstraction simplifies many difficult problems occurring in generic distributed computations like fault tolerance and synchronization, hiding them from the programmer. There are, however, algorithms that cannot be easily or efficiently expressed in MapReduce, such as recursive functions. In this work we extend the Apache Spark runtime so that it can support recursive queries. Those queries produce a very large number of tasks, making scheduling a difficult and time consuming problem. To tackle this problem we also introd uce a new parallel and more lightweight scheduling mechanism, ideal for scheduling a very large set of tiny tasks. We implemented the aforementioned scheduler and found that it simplifies the code for recursive computation and can perform up to 2.5 times faster than the default Spark scheduler for certain kinds of benchmarks.
Language English
Issue date 2017-03-17
Collection   School/Department--School of Sciences and Engineering--Department of Computer Science--Post-graduate theses
  Type of Work--Post-graduate theses
Views 544

Digital Documents
No preview available

Download document
View document
Views : 77