Kafka Summit Logo
Organized by

Kafka Summit London 2018

April 23-24, 2018 | London

Using Multiple SQL Engines for Fun; Pushing Down Filters to KSQL from BEAM & Spark

Session Level:
Video & Slides

Push down filters are nothing new, but with KSQL we can now start to push down our filters for streaming analysis. As the old saying goes, the fastest data to process is data you don’t have to load, and this is especially true in streaming systems. This talk will look proof-of-concept (e.g. code, but not code you should run in production) data sources built to push down filtering operations into KSQL from other stream processing systems. During this talk you will learn more about how Spark Streaming & Apache BEAM data sources work, and the work required to add filter push down in Spark Structured Streaming. While this talk will do its best to avoid summoning Cthulhu, mixing evaluation engines is a delicate exercise.


We use cookies to understand how you use our site and to improve your experience. Click here to learn more or change your cookie settings. By continuing to browse, you agree to our use of cookies.