Kafka provides a common framework, called Kafka Connect, to standardize integration with other data systems.
Kafka Connectors are ready-to-use components built using
Connect framework. A connector can be a
Source Connector if it reads from an external system and write to Kafka or a
Sink Connector if it reads data from Kafka and write to external system.
In this post, we will see how to implement our own Kafka source connector. Our source connector will read data from an Apache Solr collection using CursorMark and write to a kafka topic. Full code for this post with deployment instructions is avaliable at https://github.com/saumitras/kafka-solr-connect