Contrary to RPC-like applications, where communication and dependencies are explicitly defined between Services; data flowing between Event-Driven Applications is defined by how do they react to and emit events. A trade-off between Data-flow explicitness and Service autonomy becomes apparent between this two architectural-styles. The goal in this presentation is to demonstrate how Distributed-Tracing can help to cope with this trade-off, turning messaging exchange between decoupled, autonomous, Event-Driven Services, into explicit Data-flows. Zipkin project brings a Distributed-Tracing infrastructure that enables the collection, processing, and visualization of traces produced by RPC-based, as well as messaging-based applications.
This presentation includes demonstrations on how to enable Tracing for Kafka Streams applications, Kafka Connectors, and KSQL; evidencing how implicit Services behavior and communication through the event-log become can become explicit via Distributed-Tracing. But collecting and visualizing traces is just the first step. In order to create insights from tracing-data, models has to be built to enable an better understanding from the system, and improve our operational capabilities. Including research-based experiences from Netflix and Facebook on how tracing-data has been processed and polished with multiple purposes, this presentation will cover how service-dependency analysis and anomaly-detection models can be built on top of it.