Breaking down monolithic applications into smaller manageable microservices can be a tough challenge. But the benefits are many. Faster changes, developer productivity, maintainability, scalability and high performance are a few of the motivators that make companies undertake this difficult journey.
At Intuit, we have our fair share of monolithic applications. One such application is Quickbooks Online, our accounting product for small businesses. In order to decompose the application, we needed to create new services, and reduce footprint of data in the monolith by moving it to new services in a phased manner. As more and more data and services keep moving out of the monolith, this data now distributed across multiple microservices needs to be synchronized in near real time to provide a seamless and fast experience to the customers of our product.
To achieve this, we are using Kafka as our eventing backbone that can aid us in keeping distributed data in sync, without compromising performance and user experience. Guaranteed publishing of financial events with no loss, high accuracy and performance is of utmost importance as majority of Intuit products deal with highly sensitive, financial data. Strong ordering guarantees is another important criteria that Kafka can provide with low latency and high throughput. Use cases for data and streaming analytics, insights, personalization, machine-learning-based predictions, can all be unlocked by adopting Kafka as our distributed streaming platform.
This talk will take you through Intuit’s journey of building a distributed, asynchronous system using Kafka. Specifically about the choices made, challenges faced, the adaptations clients had to make and how we see Kafka powering our future!