by Stephane Maarek | Released January 2018 | ISBN: 9781838558765

Get hands-on with Kafka monitoring setup with Prometheus and Grafana, Kafka operations and Kafka cluster upgrades Setup in AWS. About This VideoSet up a multi-broker Kafka cluster in no time in AWS (using CloudFormation)Set up administration tools such as Kafka Manager, ZooNavigator, and LinkedIn's Kafka Monitor Set up monitoring using Grafana and PrometheusIn DetailThis course helps you learn Kafka administration, Kafka monitoring, Kafka operations, and Kafka upgrades. It covers the following topics:Administration for Apache KafkaSetup UI tools such as Kafka Manager, Zoo Navigator, and Kafka Monitor to get a full view of your clusterUnderstand basic operations you can perform with these tools Monitoring for Apache KafkaA Kafka that is not monitored is a ticking time-bomb. Set up proper monitoring for Kafka and ZookeeperStore your metrics in Prometheus and visualize them in GrafanaOperations for Apache KafkaPerform the most common and difficult operations in your Apache Kafka clusterRoll-restart your brokers in only one commandUpdate Kafka configurations safely with no downtimeRebalance partitions and change the replication factor of your topicsAdd, replace, and remove brokers.Upgrades for Apache KafkaUnderstand the subtlety of upgrading a Kafka clusterStep-by-step explanations on upgrading a cluster with no downtimeAll the code files are placed at https://github.com/PacktPublishing/Apache-Kafka-Series---Kafka-Monitoring-and-OperationsDownloading the example code for this course: You can download the example code files for all Packt video courses you have purchased from your account at http://www.PacktPub.com. If you purchased this course elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you. Show and hide more

1. Chapter 1 : Course Overview and Introduction
   • Course Overview 00:07:10
2. Chapter 2 : Kafka Quick Setup in AWS
   • What is AWS CloudFormation 00:01:32
   • Target Architecture and Setup 00:04:19
   • [Hands-On] Deploy Zookeeper Using CloudFormation 00:08:16
   • [Hands-On] Deploy Kafka Using CloudFormation 00:07:38
   • [Hands-On] Overview of our Zookeeper Cluster 00:09:26
   • [Hands-On] Overview of our Kafka Cluster 00:05:03
   • [Hands-On] Stopping the Kafka Cluster 00:02:50
3. Chapter 3 : Kafka Administration Setup
   • [Hands-On] Overview and Setup of Monitoring Machine 00:08:10
   • [Hands-On] Setup Zoonavigator 00:07:52
   • [Hands-On] Setup Kafka Manager 00:08:08
   • [Hands-On] Setup Kafka Monitor 00:09:28
4. Chapter 4 : Kafka Monitoring Setup: Grafana + Prometheus
   • Overview of Monitoring 00:05:06
   • [Hands-On] Setting up Prometheus + Kafka Broker 1 00:17:31
   • [Solution] Broker 2 & 3 Prometheus Setup 00:05:21
   • [Solution] Zookeeper & Prometheus Setup 00:09:40
   • [Hands-On] Setup Grafana on Administration Server 00:06:49
   • [Hands-On] Setup Kafka Dashboard on Grafana 00:05:44
   • Important Metrics to monitor 00:06:05
   • [Hands-On] Adding Graphs to Grafana 00:10:16
   • [Solution] Adding more graphs to Grafana 00:03:01
   • [Hands-On] Observing Grafana when we kill a broker 00:05:05
   • Considerations to make the setup production-ready 00:02:00
5. Chapter 5 : Kafka Operations
   • [Hands-On] Install Jolokia Agent on Broker 1 00:07:54
   • [[Solution] Install Jolokia on Broker 2 & 3 00:03:38
   • [Hands-On] Install Rolling Restart Script 00:14:04
   • [Hands-On] Run Roll Restart Script 00:07:21
   • [Hands-On] Updating Configurations 1st method (rolling restart) 00:03:45
   • [Hands-On] Updating Configurations 2nd method (kafka-configs command) 00:11:19
   • [Hands-On] Rebalancing Partitions using plain commands and JSON 00:12:10
   • [Hands-On] Rebalancing Partitions using Kafka Manager 00:02:41
   • [Hands-On] Rebalancing Partitions using LinkedIn tools 00:08:11
   • [Hands-On] Increasing Replication Factor using plain commands and JSON 00:06:38
   • [Hands-On] Increasing Replication Factor using LinkedIn tools 00:04:47
   • [Hands-On] Adding Broker 4 using CloudFormation 00:09:55
   • [Solution] Moving partitions over to Broker 4 using the CLI tools 00:04:30
   • [Hands-On] Replacing a Broker keeping EBS Volume 00:07:03
   • [Hands-On] Replacing a Broker losing an EBS Volume 00:03:41
   • Removing a Broker Overview 00:01:01
   • [Solution] Remove Broker 4 00:08:01
6. Chapter 6 : Kafka Cluster Upgrade
   • Upgrading a Kafka Cluster Overview 00:02:20
   • [Advanced] Zero Copy, Up and Down Conversions 00:08:06
   • [Hands-On] Kafka Broker Upgrade Step 1 00:04:02
   • [Hands-On] Kafka Broker Upgrade Step 2 00:02:58
   • [Hands-On] Kafka Broker Upgrade Step 3 00:03:03
   • [Hands-On] Kafka Broker Upgrade Step 4 00:01:40
   • [Hands-On] Kafka Broker Upgrade Step 5 00:03:15
7. Chapter 7 : Next Steps
   • Cleaning up AWS 00:01:17
   • Next Steps & Congratulations! 00:02:02
Show and hide more