I am currently using Prefect cloud to orchestrate different tasks, and I am deploying these tasks as jobs on Google Cloud Run using the CloudRun Push workpool.
Here’s the situation I am facing: every time a flow is run, it creates a new job on Cloud Run. However, this is not the behavior I want. Instead, I would like to have a single job for each deployment, having the deployment name, and each flow run should create a new execution ID within that job.
I’ve looked through the Prefect and Google Cloud Run documentation but haven’t found a clear way to achieve this.
Does anyone have experience with this kind of setup or could point me in the right direction? Any help would be greatly appreciated!
Thank you in advance.
I tried to use the following configuration in prefect.yaml:
infra_overrides:
env:
PREFECT_FLOW_NAME: prefect-daily-testrun
infrastructure:
job_variable_name: PREFECT_FLOW_NAME
but the result is the same: a new job is created having the name of the flow.