by Tomasz Lelek | Released January 2018 | ISBN: 9781787282032

Implement high velocity streaming for real-time data processing along with machine learning, graph analysis operations using Spark MLlib, GraphX, SparkR on Apache Spark and explore some Analytical use-cases on Spark.About This VideoLeverage the power of Apache Spark to perform efficient data processing and analytics on your data in real-timeProcess and analyze streams of data with ease and perform machine learning efficientlyA comprehensive tutorial to help you get the most out of the trending Big Data framework for all your data processing needsIn DetailThis comprehensive tutorial will acquaint you with all the aspects of real-time analytics with Apache Spark, one of the trending Big Data processing frameworks on the market today. It will show you how to leverage the features of various components of the Spark framework to efficiently process, analyze, and visualize your data.You will learn how to implement the high velocity streaming operation for data processing in order to perform efficient analytics on your real-time data. You'll analyze data using machine learning techniques and graphs. You'll learn about Spark Streaming and create real-world streaming processing that address all the problems that need to be solved. You'll solve problems using Machine Learning techniques and find out about all the tools available in the MLlibtoolkit. You'll find out how to leverage Graphs to solve real-world problems.At the end of this video, you'll also see some useful Machine Learning algorithms with the help of Spark MLlib and will integrate Spark with R. We'll also make sure you're confident and prepared for graph processing, as you'll learn more about the GraphX API. By the end, you'll be well-versed in the aspects of real-time analytics and implement them with Apache Spark. Show and hide more

1. Chapter 1 : Spark Streaming
   • The Course Overview 00:02:58
   • Introducing Spark Streaming 00:04:18
   • Streaming Context 00:03:39
   • Processing Streaming Data 00:02:40
   • Use Cases 00:03:13
   • Spark Streaming Word Count Hands-On 00:05:45
   • Spark Streaming – Understanding Master URL 00:05:26
   • Integrating Spark Streaming with Apache Kafka 00:05:26
   • mapWithState Operation 00:06:56
   • Transform and Window Operation 00:02:52
   • Join and Output Operations 00:02:41
   • Output Operations –Saving Results to Kafka Sink 00:02:56
2. Chapter 2 : Advance Streaming and Use Cases
   • Handling Time in High Velocity Streams 00:04:59
   • Connecting External Systems That Works in At Least Once Guarantee – Deduplicaion 00:05:52
   • Building Streaming Application –Handling Events That Are Not in Order 00:06:02
   • Filtering Bots from Stream of Page View Events 00:06:54
3. Chapter 3 : Spark MLlib and ML Pipelines
   • Introducing Machine Learning with Spark 00:05:54
   • Feature Extraction and Transformation 00:01:13
   • Transforming Text into Vector of Numbers – ML Bag-of-Words Technique 00:04:44
   • Logistic Regression 00:06:53
   • Model Evaluation 00:02:43
   • Clustering 00:02:42
   • Gaussian Mixture Model 00:05:11
   • Principal Component Analysis and Distributing the Singular Value Decomposition (SVD) 00:03:16
   • Collaborative Filtering – Building Recommendation Engine 00:07:33
4. Chapter 4 : Spark GraphX
   • Introducing Spark GraphX–How to Represent a Graph? 00:03:04
   • Limitations of Graph-Parallel System – Why Spark GraphX? 00:02:54
   • Importing GraphX 00:01:46
   • Create a Graph Using GraphX and Property Graph 00:05:03
   • List of Operators 00:03:48
   • Perform Graph Operations Using GraphX 00:04:08
   • Triplet View 00:03:28
5. Chapter 5 : Performing Spark GraphX Operations
   • Perform Subgraph Operations 00:04:28
   • Neighbourhood Aggregations – Collecting Neighbours 00:03:42
   • Counting Degree of Vertex 00:04:21
   • Caching and Uncaching 00:03:57
   • GraphBuilder 00:02:53
   • Vertex and Edge RDD 00:03:32
   • Structural Operators – Connected Components 00:03:04
6. Chapter 6 : SparkR
   • Introduction to SparkR and How It’s Used? 00:04:14
   • Setting Up from RStudio 00:01:57
   • Creating Spark DataFrames from Data Sources 00:03:32
   • SparkDataFrames Operations – Grouping, Aggregation 00:02:46
   • Run a Given Function on a Large Dataset Using dapply or dapplyCollect 00:04:14
   • Running Large Dataset by Input Column(s) and Using gapply or gapplyCollect 00:04:06
   • Run Local R Functions Distributed Using spark.lapply 00:02:05
   • Running SQL Queries from SparkR 00:03:02
7. Chapter 7 : Analytical Use Cases
   • PageRank Using Spark GraphX 00:06:54
   • Sending Real-Time NotificationWhen User Want to Buy a Product on the E-Commerce Site 00:08:46
Show and hide more