spark


Exploring Wikipedia with Apache Spark: A Live Coding Demo

Location: Salon D
April 12th, 2016
4:00 PM - 5:00 PM

The real power and value proposition of Apache Spark is in creating unified use cases combining batch analysis, stream analysis, SQL, machine learning, graph processing and visualizations. In this live coding demo, Sameer will use various Wikipedia datasets to build a dashboard about what is happening in the world during his talk. The application will connect to the live Edits stream of Wikipedia and join it with other Wikipedia datasets to derive interesting insights about what's trending on the planet.


NoLambda: A new architecture combining streaming, ad hoc, machine learning, and batch analytics

Location: Salon A
April 11th, 2016
1:30 PM - 2:30 PM

In today’s world of exploding big and fast data, developers who want both streaming analytics and ad hoc, OLAP-like analysis have often had to develop complex architectures such as Lambda—a path for fast streaming analytics using NoSQL stores such as Cassandra and HBase with a separate batch path involving HDFS and Parquet. While this approach works, it involves too many moving parts, too many technologies for ops, and too many engineering hours. Helena Edelson and Evan Chan highlight a much simpler approach to combine streaming and ad hoc/batch analysis using what they call the NoLambda stack (Apache Spark/Scala, Mesos, Akka,
Read more  »

Evan Chan

Creator, FiloDB

Untangling Healthcare with Spark and Dataflow

Location: Salon A
April 12th, 2016
2:45 PM - 3:45 PM

Spark is becoming a data processing giant, but it leaves much as an exercise for the user. Developers need to write specialized logic to move between batch and streaming modes, manually deal with late or out-of-order data, and explicitly wire complex flows together. This talk looks at how we tackled these problems over a multi-petabyte dataset at Cerner. We start with how hand-written solutions to these problems evolved to prescriptive practices, opening up development of such systems to a wider audience. From there we look at how the emergence of Google's Dataflow on Spark is helping us take the next
Read more  »

Ryan Brush

Engineer, Cerner