Kafka Summit Logo
Organized by

Kafka Summit San Francisco 2018

October 16-17, 2018 | San Francisco

Kafka on ZFS: Better Living Through Filesystems

Session Level: Advanced
Video & Slides

You’re doing disk IO wrong, let ZFS show you the way. ZFS on Linux is now stable. Say goodbye to JBOD, to directories in your reassignment plans, to unevenly used disks. Instead, have 8K Cloud IOPS for $25, SSD speed reads on spinning disks, in-kernel LZ4 compression and the smartest page cache on the planet. (Fear compactions no more!)

Learn how Jet’s Kafka clusters squeeze every drop of disk performance out of Azure, all completely transparent to Kafka.

  • Striping cheap disks to maximize instance IOPS
  • Block compression to reduce disk usage by ~80% (JSON data)
  • Instance SSD as the secondary read cache (storing compressed data), eliminating >99% of disk reads and safe across host redeployments
  • Upcoming features: Compressed blocks in memory, potentially quadrupling your page cache (RAM) for free

We’ll cover:

  • Basic Principles
  • Adapting ZFS for cloud instances (gotchas)
  • Performance tuning for Kafka
  • Benchmarks

We use cookies to understand how you use our site and to improve your experience. Click here to learn more or change your cookie settings. By continuing to browse, you agree to our use of cookies.