I have a Kafka_USA kafka system. It pulls data from other kafka systems like Kafka_NY, Kafka_CA, Kafka_TX, …so on.
We should be able to add or remove these kafka data sources (which represent data of particular US state. Like we may remove Kafka_TX, and add Kafka_FL, etc.). We need to maintain a config file to keep track of these current kafka data sources information, so that the Kafka_USA always pulls data from the data sources that are specified in the config file. If a data source is removed from the config file, then it’d stop pulling, and if a new data source is added in the config file, it should start pulling data for that.
Thoughts? How can I implement this custom data pull?
Thanks much.