by Rich Morrow | Released June 2016 | ISBN: 9781771375658

You're new to Big Data, you've heard about Apache Spark and Apache Hadoop and you want to play. Big Data coach Rich Morrow gets you into the game via sixteen sprints (sixteen hands-on labs) across the Spark-Hadoop ball field. First, you'll create playing areas using Amazon Web Services EMR and Cloudera Quickstart VM. Then you'll install Hadoop, run basic HDFS commands, learn MapReduce, use Flume and Sqoop, run Spark and then run Spark again.You'll play with Spark SQL, learn common MLLib usage, do analysis with Hive, ETL with Pig, and then jog through Hadoop/Cloud use cases, Hbase basics, and enterprise integration. When practice is over, you'll know Spark, it's associated modules, the Hadoop ecosystem, and the when, where, how, and why each technology is used. Working files are included, allowing you to follow along with the author throughout the lessons. Play on.Understand Apache Spark and why it's Big Data's fastest growing open source projectLearn what Apache Hadoop is and how it's used in the world of Big DataMaster the basics of Hadoop - HDFS, YARN and MapReduceMaster the basics of Spark- Spark SQL, MLlib, Spark Streaming, Graphx and moreDiscover Sqoop, Flume, Hive, Pig, HBase, and Oozie - key components of HadoopGain direct experience with Spark and Hadoop with sixteen hands-on labsRich Morrow is a 20+ year veteran of IT and an expert in big data and cloud technologies. He's used Hadoop and AWS for over 6 years in his consulting practice, quicloud.com, and has taught Cloudera (Hadoop) and AWS for Global Knowledge (where he also serves as Course Director for Cloud and Big Data) for over 4 years. Rich retains all certifications for both AWS and Cloudera, and is a prolific writer and speaker on Cloud, Big Data, DevOps/Agile, Mobile, and IoT topics, including the O'Reilly titles Hands-on with Amazon Redshift, Learning Apache Hadoop and Cloud Computing With AWS. Show and hide more Publisher resources Download Example Code

1. Introduction
   • Course Introduction 00:04:21
   • About The Author 00:04:14
   • What Is Big Data 00:11:07
   • Historical Approaches 00:07:04
   • Modern-Day Approach 00:12:42
   • What Is Hadoop 00:11:05
   • Hadoop Core Vs Ecosystem 00:05:03
   • Hadoopable Problems 00:06:37
2. Hadoop Basics
   • HDFS And Yarn 00:08:14
   • Hive And Pig Interface Introduction 00:05:59
   • Introduction To Spark 00:04:37
   • Hadoop In The Cloud (Amazon Web Services Intro) 00:08:49
   • Installing Hadoop Into EMR Part - 1 00:15:31
   • Installing Hadoop Into EMR Part - 2 00:15:34
   • Installing Cloudera Quickstart VM 00:11:01
   • Web GUIs 00:11:06
3. Hadoop Distributed Filesystem (HDFS)
   • HDFS Architecture 00:10:05
   • HDFS File Write Walkthrough 00:17:57
   • Secondary Name Node 00:06:38
   • Basic HDFS Commands 00:09:23
   • Using HDFS Commands Part - 1 00:07:34
   • Using HDFS Commands Part - 2 00:09:27
   • HA And Federation Basics 00:12:48
   • HDFS Access Controls (Or Lack Thereof) 00:09:34
4. Yarn
   • Yarn Purpose 00:06:16
   • Yarn Architecture 00:07:25
   • Yarn With Spark 00:06:44
5. MapReduce
   • MapReduce Explained 00:11:52
   • MapReduce Architecture 00:07:36
   • MapReduce Code Walkthrough 00:11:59
   • MapReduce Details Walkthrough 00:04:45
   • Running MapReduce Job 00:08:59
6. HDFS Data Import And Export
   • Import/Export Options 00:11:12
   • Flume Introduction 00:10:53
   • Using Flume 00:13:43
   • Sqoop Introduction 00:09:25
   • Using Sqoop 00:17:01
   • HDFS Interaction Tools 00:06:01
   • Oozie Introduction 00:10:17
7. Spark Basics
   • Spark Value Propositions 00:08:30
   • Spark Run Modes (Yarn, Standalone, Mesos) 00:07:33
   • RDDs And Dataframes 00:17:24
   • Hands On Spark Part - 1 00:08:12
   • Hands On Spark Part - 2 00:10:38
   • Running Spark Part - 1 00:09:58
   • Running Spark Part - 2 00:13:55
   • Optimizing And Debugging Spark 00:18:17
   • Spark Libraries Overview 00:09:05
8. Spark Built-In Libraries
   • Spark SQL 00:09:01
   • Spark SQL Usage 00:12:02
   • MLlib Basics 00:15:30
   • Common MLlib Usage Part - 1 00:15:02
   • Common MLlib Usage Part - 2 00:08:23
   • Spark Streaming 00:12:43
   • GraphX 00:09:58
9. Hive And Pig
   • Hive Vs Pig 00:09:53
   • Hive Basics 00:11:53
   • Analysis With Hive 00:10:54
   • Pig Basics 00:14:38
   • ETL And Analytics With Pig 00:20:16
10. Hadoop In The Cloud
    • Hadoop/Cloud Use Cases 00:05:16
    • Elastic MapReduce (EMR) 00:12:47
11. Ecosystem
    • HBase Basics 00:11:16
    • Enterprise Integration 00:10:39
12. Wrap Up
    • Wrap Up 00:03:41
Show and hide more