by | Released October 2015 | ISBN: 9781771375214

In this Analytic Data Storage in Hadoop training course, expert author Ryan Blue will teach you about typical storage and ingest patterns in Hadoop. This course is designed for users that are already familiar with Hadoop. You will start by learning how to create the dataset, load sample data, and query a dataset. From there, Ryan will teach you about partitioning, formats, and Avro. This video tutorial also covers parquet, bulk data drops, and database snapshots and mirroring. Finally, you will learn about event stream processing, including how to build a test pipeline and move to production. Once you have completed this computer based training course, you will have gained a solid understanding of typical storage and ingest patterns in Hadoop. Working files are included, allowing you to follow along with the author throughout the lessons. Show and hide more Publisher resources Download Example Code

1. Introduction
   • Introduction And What To Expect 00:03:27
   • About The Author 00:04:43
   • Introduction To The Movielens Dataset 00:01:41
2. Getting Started With Hadoop
   • Your First Hadoop Dataset - CSV To SQL Query 00:06:45
   • Describing Your Data With A Schema 00:10:06
   • Creating The Dataset 00:10:20
   • Loading Sample Data 00:09:56
   • Querying A Dataset 00:05:58
3. Partitioning
   • Introduction to Partitioning 00:07:25
   • Testing Partition Strategies 00:17:13
   • Partitioning Patterns 00:13:17
4. Formats
   • Introduction To File Formats 00:14:02
   • Hadoop File Formats - Why Splitability Matters 00:05:48
   • Hadoop File Formats Review 00:13:44
5. Avro
   • Avro 00:03:46
   • Avro File Format 00:07:32
   • Avro Shemas 00:10:01
   • Avro Object Models 00:11:05
   • Avro Tools 00:07:44
6. Parquet
   • Parquet 00:14:57
   • Parquet File Formats 00:12:01
   • Parquet Object Models 00:05:15
   • Parquet Tools 00:10:24
7. Bulk Data Drops
   • Application Pattern Overview - Bulk Data Drops 00:08:34
   • Apache Nifi - Part 1 00:07:35
   • Apache Nifi - Part 2 00:06:39
   • Using Apache Nifi 00:05:13
   • Preparing The Datasets 00:11:24
   • Building The Data Flow - Part 1 00:15:10
   • Building The Data Flow - Part 2 00:14:07
8. Database Snapshots And Mirroring
   • Application Pattern Database Snapshots And Mirroring 00:06:14
   • Introduction To Apache Sqoop 00:08:51
   • Table Snapshots 00:10:57
   • Incremental Mirroring - Part 1 00:08:11
   • Incremental Mirroring - Part 2 00:10:55
   • Moving To Production 00:04:18
9. Event Stream Processing
   • Application Pattern - Event Streams 00:07:54
   • Introduction To Apache Flume 00:06:51
   • Building A Pipeline - Part 1 00:09:18
   • Building A Pipeline - Part 2 00:14:51
   • Moving to Production 00:14:14
10. Conclusion
    • Wrap Up 00:00:51
Show and hide more