Kafka Summit Logo
Organized by

Kafka Summit SF 2019

September 30 - October 1, 2019 | San Francisco

What’s Inside the Black Box? Using ML to Tune and Manage Kafka.

Session Level: Intermediate

We use machine learning to delve deep into the internals of how systems like Kafka work. In this talk I’ll dive into what variables affect performance and reliability, including previously unknown leading indicators of major performance problems, failure conditions and how to tune for specific use cases. I’ll cover some of the specific methodology we use, including Bayesian optimization, and reinforcement learning. I’ll also talk about our own internal infrastructure that makes heavy use of Kafka and Kubernetes to deliver real-time predictions to our customers.


We use cookies to understand how you use our site and to improve your experience. Click here to learn more or change your cookie settings. By continuing to browse, you agree to our use of cookies.