I have multiple glue jobs all of them have same job parameters
--param1
and --param2
which are being used inside the glue jobs.
I connected both the glue jobs in steps functions.
But not sure how we can pass the params and access them inside the glue job.
I tried configuring them in definition
like
{
"Type": "Task",
"Resource": "arn:aws:states:::glue:startJobRun.sync",
"Parameters": {
"JobName": "Test1",
"Arguments": {
"--param1": "$.param1",
"--param2": "$.param2"
}
},
"Next": "Test2"
}
and similar thing for Test2
job as well.
But when starting the execution, if I provide below json as input, it is still printing "$.param1"
and "$.param2"
as parameter values.
{
"param1": "value1",
"param2": "value2"
}
How do I pass value1 and value2 as values for param1 and param2 for all the glue jobs while starting the execution of step function and access them inside glue job.Currently I’m accessing job params using below code
args = getResolvedOptions(sys.argv, ['JOB_NAME', "param1", "param2"])
print("Test1", args["param1"], args["param2"])