I have set up a pipeline that uses EventBridge to trigger a Glue workflow when a new file is added to an S3 bucket. I’ve tested the pipeline and verified that when I add a new file to S3, EventBridge successfully triggers my Glue workflow and kicks off a Glue job.

Now I want to pass S3 bucket and S3 object parameters from the EventBridge event to my Glue job so it can process the newly added file. I edited the input path and input template of the EventBridge target input transformer to capture the parameters I want (bucket name and object name) but when I add a file to the bucket, the parameters don’t get picked up by my Glue job.

Starting from the working EventBridge event, I modified the target input transformer with the below configurations
Input Path
{ "bucket_name": "$.detail.bucket.name", "object_key": "$.detail.object.key" }

Input template
{"--bucket_name": <bucket_name>,"--object_key": <object_key>}

Note: I tried input template keys with and without ‘–‘.

When I add a file to my S3 bucket, it still triggers EventBridge and kicks off a Glue job. But the Glue job does not pick up my bucket_name and object_key parameters (also verified by listing sys.argv in the job itself).
glue job parameters