I need to connect to a Kafka cluster using SSL certificates.
I have tried deploying the job by keeping the SSL files in GCS bucket, but that doesn’t work.
I also tried to copy the file from GCP to a "/tmp/"
location, assuming that file will be downloaded in the worker node, but to my surprise, it got downloaded in the local server from where I am submitting the job.
How do I make my SSL files available to the dataflow worked nodes?
I am using Python SDK.