You have data centers all over the world. You have set them up close to your customers and services. You have decentralized data handling with a network of data processing systems. The trouble is now how do you make sure your messages reach the appropriate data centers without getting into a complicated mess of dedicated topic-to-topic replication? What if your messaging bus was smart enough to figure out where to deliver each message? With Adobe’s Experience Cloud Pipeline, we have tackled this very issue.
Adobe’s Experience Cloud gives access to an integrated set of Adobe products and solutions that help Adobe’s customers to build campaigns, manage to advertise, gain profound insights and deliver personalized experiences all in near real time. To achieve this Adobe solutions must exchange data close to real time. Often data processed by the solutions need to travel across globally distributed data centers hosted on both public and private clouds.
Based on Apache Kafka, Adobe’s Experience Cloud Pipeline is a globally distributed, mission-critical messaging bus for asynchronous communication across Adobe solutions that process tens of billion messages each day. The Experience Cloud Pipeline also serves an entry point for the streaming data flowing from external systems into the Adobe Experience Cloud. Multi-data center deployment is commonplace to most large enterprises. Having a smart messaging bus that is capable of routing, filtering and enriching messages in near real time, in proximity to your solutions and customers is of paramount importance. Adobe’s Experience Cloud Pipeline demonstrates a successful large-scale deployment of such use case.