Kafka Summit Logo

Kafka Summit London 2018

Streaming platforms at massive scale.

April 23-24, 2018 | London

Tuning a Kafka Cluster to Achieve High Partition Density and High Throughput

Session Level:
Video & Slides

In this session we will discuss the performance analysis we performed while scaling Kafka in order to achieve a high throughput cluster with a large number of partitions. Decisions like which disk storage options are the most cost effective and which OS tuning knobs we turned to optimize the throughput will be discussed. We will also talk about performance effects as the utilisation of a cluster increases and the practical limits we found. Benchmark test details will be presented as well as caveats encountered during the testing. We hope that this will be helpful to anyone trying to create a large Kafka cluster on the cloud or in the datacenter.


We use cookies to understand how you use our site and to improve your experience. Click here to learn more or change your cookie settings. By continuing to browse, you agree to our use of cookies.