Kafka Summit Logo Register

Kafka Summit San Francisco 2018

Streaming platforms at massive scale.

October 16-17, 2018 | San Francisco

Register

The Big Picture: Monitoring and Orchestration of Your Microservices Landscape with Kafka and Zeebe

Session Level: Intermediate

A company’s business processes typically span more than one microservice. In an e-commerce company, for example, a customer order might involve microservices for payments, inventory, shipping and more. Implementing long-running, asynchronous and complex collaboration of distributed microservices is challenging. How can we ensure visibility of cross-microservice flows and provide status and error monitoring? How do we guarantee that overall flows always complete, even if single services fail? Or, how do we at least recognize stuck flows so that we can fix them?

In this talk, I’ll demonstrate an approach based on real-life projects using the open source workflow engine zeebe.io to orchestrate microservices. Zeebe can connect to Kafka to coordinate workflows that span many microservices, providing end-to-end process visibility without violating the principles of loose coupling and service independence. Once an orchestration flow starts, Zeebe ensures that it is eventually carried out, retrying steps upon failure. In a Kafka architecture, Zeebe can easily produce events (or commands) and subscribe to events that will be correlated to workflows. Along the way, Zeebe facilitates monitoring and visibility into the progress and status of orchestration flows. Internally, Zeebe works as a distributed, event-driven and event-sourced system, making it not only very fast but horizontally scalable and fault tolerant—and able to handle the throughput required to operate alongside Kafka in a microservices architecture. Expect not only slides but also fun little live-hacking sessions and real-life stories.


We use cookies to understand how you use our site and to improve your experience. Click here to learn more or change your cookie settings. By continuing to browse, you agree to our use of cookies.