Guardant Health: Fast, scalable, data processing with Alluxio, Mesos, and Minio

Alluxio Meetup features a chance to interact with other Alluxio (http://www.alluxio.com/) users and developers, along with two presentations. Registration is required.

Adit Madan from Alluxio and Jörg Schad from Mesosphere will co-present the Alluxio and Mesosphere joint solution

Omar from Guardant Health will be sharing their experience leveraging the Alluxio - Mesosphere joint solution to gain faster insights from mobile data.

Agenda:

6:30 - 7:00: Happy Hour & Networking

7:00 - 7:30: First talk + Q&A

7:30 - 8:00: Second talk + Q&A

8:00 - 8:30: Open Q&A & Networking

Food and drinks will be available starting at 6:30pm, presentations will begin at 7:00pm.

Special thanks to Mesosphere for hosting this meetup!

From SMACK to SMAACK Running Alluxio on DC/OS

Abstract

Speed is usually a key factor when analyzing large amounts of data. Alluxio enables analytics applications, such as Apache Spark, to retrieve stored data at memory speeds. DC/OS makes it easy to deploy distributed programs (such as Alluxio and Spark) and containers across large clusters.

In this talk, we will first discuss the development of the DC/OS Alluxio package, which deploys Alluxio on top of DC/OS, and then then demo the deployment a complete analytics stack, both with and without Alluxio, in order to see the benefits Alluxio provides.

Speaker Bios

Jörg is a software engineer at Mesosphere in San Francisco. In his previous life he implemented distributed and in memory databases and conducted research in the Hadoop and Cloud area. His speaking experience includes various Meetups, international conferences, and lecture halls.

Adit Madan is a software engineer at Alluxio. His experience is in distributed systems, storage systems, and large-scale data analytics. He has a M.S. from Carnegie Mellon University, and a B.S. from IIT.

Scalable Genomics Data Processing Pipeline with Alluxio, Mesos, and Minio

Abstract

Guardant Health leverages Alluxio, Mesos, and Minio to create an end-to-end processing solution that is performant, scalable, and cost optimal. We use Alluxio as the unified storage layer to connect disparate storage systems and bring memory performance, with Minio mounted as the under store to Alluxio to keep cold (infrequently accessed) data and to sync data to AWS S3. Apache Mesos abstracts CPU, memory, storage, and other compute resources away from machines, enabling fault-tolerant and elastic distributed systems to easily be built and run effectively.

In this talk I will share our experience using Alluxio, Mesos, and Minio to tame genomic data at Guardant Health.

Speaker Bio

Omar Sobh is a DevOps Engineer at Guardant Health leading the charge for storage and compute initiatives for the Guardant Health Genomic Processing Pipelines.