by Nishant Garg | Released June 2018 | ISBN: 9781787287402

Query data using Spark SQL, analyze data and perform real-time processing using Spark StreamingAbout This VideoQuery data using Spark SQL APIsHandle steaming data using Spark StreamingBest Practice for streaming dataIn DetailThis tutorial is focused on analytics and real-time data processing using Apache Spark. You will begin with Spark SQL, using the Spark SQL API and built-in functions; within Apache Spark, you will go through some interactive analysis and look at some integrations between Spark and Java/Scala/Python.You will explore Spark Streaming, streaming context, and DStreams. You will learn how Spark streaming works on top of the Spark core, thus inheriting its features. You will stream data and also learn best practices for managing high-velocity streaming and external data sources.By the end of this course, you will be able to load data from a variety of structured sources (for example, JSON, Hive, and Parquet) using Spark SQL and schema RDDs and will perform real-time data processing. Show and hide more

1. Chapter 1 : Spark SQL
   • The course overview 00:03:07
   • Spark SQL Introduction 00:09:05
   • Spark SQL – Core Abstractions 00:06:49
   • Creating DataFrames from RDD 00:03:55
   • Creating DataFrames from Files 00:07:21
   • Creating DataFrames from Data Sources 00:05:02
   • DataFrame API – Common Operations 00:03:03
   • DataFrame API – Query Operations 00:05:45
   • DataFrame API – Actions 00:04:38
   • DataFrame API – Built-In Functions 00:03:01
2. Chapter 2 : Spark Streaming
   • Spark Streaming – Introduction 00:03:56
   • Spark Streaming – Quick Example 00:05:54
   • Spark Streaming – Architecture 00:04:15
   • Spark Streaming – Transformations 00:06:12
   • Spark Streaming – Input Sources 00:04:53
   • Spark Streaming – Performance Considerations 00:04:04
3. Chapter 3 : Advance Spark Streaming
   • Best Practices for High Velocity Streams 00:07:27
   • Best Practices for External Data Sources 00:05:04
   • Design Patterns 00:04:51
Show and hide more