Kafka Summit Logo
Organized by

Kafka Summit San Francisco 2018

Streaming platforms at massive scale.

October 16-17, 2018 | San Francisco

Real-Time Market Data Analytics Using Kafka Streams

Session Level: Advanced
Video & Slides

At Bloomberg, we are building a streaming platform with Apache Kafka, Kafka Streams and Spark Streaming to handle high volume, real-time processing with rapid derivative market data. In this talk, we’ll share the experience of how we utilize Kafka Streams Processor API to build pipelines that are capable of handling millions of market movements per second with ultra-low latency, as well as performing complex analytics like outlier detection, source confidence evaluation (scoring), arbitrage detection and other financial-related processing.

We’ll cover:

  • Our system architecture
  • Best practices of using the Processor API and State Store API
  • Dynamic gap session implementation
  • Historical data re-processing practice in KStreams app
  • Chaining multiple KStreams apps with Spark Streaming job

We use cookies to understand how you use our site and to improve your experience. Click here to learn more or change your cookie settings. By continuing to browse, you agree to our use of cookies.