High-speed and low footprint data stream processing is high in demand for Kafka Streams applications. However, how to write an efficient streaming application using the Streams DSL has been asked by many users in the past since it requires some deep knowledge about Kafka Streams internals. In this talk, I will talk about how to analyze your Kafka Streams applications, target performance bottlenecks and unnecessary storage costs, and optimize your application code accordingly using the Streams DSL.
In addition, I will talk about the new optimization framework that we have been developed inside Kafka Streams since the 2.1 release which replaced the in-place translation of the Streams DSL into a comprehensive process composed of streams topology compilation and rewriting phases, with a focus on reducing various storage footprints of Streams applications, such as state stores, internal topics etc.
This talk is aimed to give developers who are interested to write their first or second Streams applications using the Streams DSL, or those who have already launched several services written in Kafka Streams in production but wants to further optimize these applications a better understanding on how different Streams operators in the DSL are being translated into the streams topology during runtime. And by having a deep-dive into the newly added optimization framework for Streams DSL, audience will have more insight into the kinds of optimization opportunities that are possible for Kafka Streams, that the library is trying to tackle right now and in the near future.