

It provides source connectors that have the ability to stream database changes to Kafka Topics. Kafka Connect is a framework for connecting databases, key-value stores and external systems to Kafka. Because of this interest, integration between Kafka and legacy systems usually happens at some point as companies move onto a newer system and off a legacy one.Ī useful part of the Kafka ecosystem is Kafka Connect. More and more companies are utilising Kafka and the ecosystem it provides. Interest in Apache Kafka is growing rapidly. I’ve been pondering in my head for a while now what would be the most pain free way and quickest way of consuming an Avro formatted message from a Kafka Topic, converting it into JSON and sending it to a RabbitMQ queue.

We could have one that consumes AVRO and produces XML the list goes on.

We could have a system that consumes an XML message and produces a message in JSON. Two of the main factors of system integration in my opinion are the conversion of messages between the system producing the message & system consuming it i.e a customer model to a canonical model and how the two systems actually communicate with one another. There are so many things that have to be taken into account when systems of this kind need to be integrated with other systems. Integrating messaging systems can be a pain.
