International Workshop on Big Data in Motion and Big Data at Rest (BD-MR) (BD-MR)

Different Big Data paradigms co-exist nowadays, each carefully optimized in accordance with the final application goals and constraints. For instance, the stream computing paradigm is well adapted for continuous complex analysis of streaming data with low latency, the Map Reduce paradigm is better suited for parallel analysis of massive volumes of data at rest, while the relational paradigm delivers efficient access to structured information. The evolution of these disparate data management paradigms has resulted in an array of solutions catering to a wide range of diverse use-cases. Unfortunately, this has also fragmented the Big Data solutions that are now adapted to particular types of applications. At the same time, applications have moved towards leveraging multiple paradigms in conjunction, for instance to collate real time data (data in motion) and historical data (data at rest). This has led to an imminent need of solutions that seamlessly and transparently allow practitioners to mix different approaches, and that can function and provide answers as an all-in-one solution. Today, Big Data applications require the redesign of novel Big Data infrastructures and algorithms with the following underlying challenges: the systems must decide the analytics to be applied on data in motion and with very low latency, identify the relevant synopsis of data in motion that are important to be stored and make them available to interrogation at any moment, not compromise the consistency of the data, and ensure that data at rest processing does not slow down the overall system. Apache Spark, Apache Yarn, Apache Mesos, Microsoft Naiad, SummingBird, Stratosphere, Storm are just several examples of frameworks that, more or less, start looking into this direction.