Design Patterns for working with Fast Data in Kafka

Apache Kafka is an open-source message broker project that provides a platform for storing and processing real-time data feeds. In this presentation Ian Downard will describe the concepts that are important to understand in order to effectively use the Kafka API. You will see how to prepare a development environment from scratch, how to write a basic publish/subscribe application, and how to run it on a variety of cluster types, including simple single-node clusters, multi-node clusters using Heroku’s “Kafka as a Service”, and enterprise-grade multi-node clusters using MapR’s Converged Data Platform.

Ian will also discuss strategies for working with "fast data" and how to maximize the throughput of your Kafka pipeline. He'll describe which Kafka configurations and data types have the largest impact on performance and provide some useful JUnit tests, combined with statistical analysis in R, that can help quantify how various configurations effect throughput.

The code and presentation for this talk will be available at https://github.com/iandow/design-patterns-for-fast-data.

Speaker:

Ian Downard is a technical evangelist for MapR where he is focused on creating developer-friendly ways to use the MapR Converged Data Platform.

• Personal Blog: http://www.bigendiandata.com
• Twitter: https://twitter.com/iandownard