Spark and Cassandra: An Amazing Apache Love Story by Patrick McFadin

Spark and Cassandra: An Amazing Apache Love Story by Patrick McFadin SlideShare Explore You Successfully reported this slideshow.Spark and Cassandra: An Amazing Apache Love Story by Patrick McFadinUpcoming SlideShareLoading in …5× 5 Comments 11 Likes Statistics Notes Yuta Watanabe Luciano Resende , Open Source Data Science/AI Platform Architect at IBM Victor Coustenoble , Technical regional manager EMEA at Trifacta Ramin Orujov , Big Data Software Engineer at Luxoft Nitin Sareen , Manager at Accenture at Accenture Show More No DownloadsNo notes for slide 1. ©2013 DataStax Confidential. Do not distribute without consent.@PatrickMcFadinPatrick McFadin
Chief Evangelist, DataStaxSpark and Cassandra: An amazingApache love story1 2. Store a ton of data Analyze a ton of data 3. Community Response? 4. CassandraOnly DC 5. CassandraOnly DCCassandra+ Spark DCSparkJobs 6. CassandraOnly DCCassandra+ Spark DCSparkJobsSparkStreaming 7. WorkerWorkerWorker WorkerAnalytics WorkloadTransactional Workload 8. DataStax Enterprise 9. DataStax Enterprise 10. • 10T of high frequency event data daily• Constant increasing volume“The web server that powers the interface can query bothdatacenters, depending on which the user is closest to,”“A small set of signals tend to double every eight months. Sowe needed a model that can scale linearly.”- Arun Jayandra, Microsoft 11. REST APIO365 Event HubIngestion Worker(Azure worker role using DataStax C# driver)C* AnalyticsREST APIO365KafkaC*/Spark StreamingAnalyticsG4 – Local SSDKafka: G4 – Data DiskZooKeeper: A7 – Data DiskPaaS SmallG4 – Local SSDCluster 1:Cluster 2:20k – 50k events/sec200k+ events/sec 12. Data Protection• Maximilian Schrems v Data Protection Commissioner• No longer OK to ship EU data to US under “SafeHarbour”Product_Catalog RF=3Product_Catalog RF=3 Customer_Data RF=3Customer_Data RF=0Product_Catalog RF=3Customer_Data RF=3 13. • 300k customers• Report on energy usage• Predict boiler failure“We’re dealing largely with time series data, and Spark is 10 to 100times quicker as it is operating on data in-memory…Cassandradelivers what we need today and if you look at the Internet of Thingsspace; that is what is really useful right now.” - Jim Anning, British GasHive Active Heating™ 14. CassandraOnly DCCassandra+ Spark DCSparkJobsSparkStreamingHome Data CenterHive Active Heating™ 15. Store a ton of data Analyze a ton of dataThank you! Recommended Natural Sparksmanship – The Art of Making an Analytics Enterprise Cross the C...Spark Summit A Scaleable Implemenation of Deep Leaning on Spark- Alexander UlanovSpark Summit Netflix branding stumblesMayur Verma Distributed Heterogeneous Mixture Learning On SparkSpark Summit Distributed Data Processing using Spark by Panos Labropoulos_and Sarod Yataw...Spark Summit Breaking Down Analytical and Computational Barriers Across the Energy Industr...Spark Summit Sparkling Random Ferns by P Dendek and M FedoryszakSpark Summit Data Science at Scale by Sarah GuidoSpark Summit Improving the power of a picture at Netflix -- the Science and Engineering Be...Gopal Krishnan Integrating with Hadoop: Couchbase Connect 2014Couchbase About Blog Terms Privacy Copyright × Public clipboards featuring this slideNo public clipboards found for this slideSelect another clipboard ×Looks like you’ve clipped this slide to already.Create a clipboardYou just clipped your first slide! Clipping is a handy way to collect important slides you want to go back to later. Now customize the name of a clipboard to store your clips. Description Visibility Others can see my Clipboard