I’ve been racking my brain trying to figure out what’s wrong with this setup. I’ve managed to get things going but think I’m just missing this last part.

I have a setup with the follow:

1. Cloud Run Job A (“enqueue-tasks”)
2. Cloud Run Job B (“process-item”)
3. a Cloud Tasks queue (“process-items”)

My goal is to schedule a biweekly trigger of the “enqueue-tasks” job. This job does some stuff (not important), and then queues up some tasks in the “process-items” queue.

This seems to be working fine

Each tasks should call the “process-item” job with some overrides. The queue should manage the rate-limit at which each tasks runs.

This seems to be the problem.

IAM Setup:

• cloud task service account
  • Role: Cloud Run Invoker w/ Overrides
    • run.jobs.run
    • run.jobs.runWithOverrides
    • run.routes.invoke
• cloud run job service account (Default compute service account)
  • Cloud Tasks Enqueuer
  • Service Account Token Creator
  • Service Account User

I created a custom role with the run.jobs.runWithOverrides permission so that I may pass additional overrides to the “process-item” job via args in each task

Excerpt of how I create/enqueue the tasks (note, everything is in us-west1 region):

// NOTE: This code is inside the "enqueue-tasks" job

import { CloudTasksClient } from "@google-cloud/tasks"

const client = new CloudTasksClient()

const project = "my-project-id"
const location = "us-west1"
const queueName = "process-items"
const taskId = "task-123" // example for item #123
const url = "https://run.googleapis.com/v2/projects/my-project-id/locations/us-west1/jobs/process-item:run"

await client.createTask({
    parent: client.queuePath(project, location, queueName),
    task: {
        name: client.taskPath(project, location, queueName, taskId),
        httpRequest: {
            httpMethod: "POST",
            url: url,
            oidcToken: {
                serviceAccountEmail: "[email protected]",
                audience: url,
            },
            headers: {
                "Content-Type": "application/json",
            },
            body: Buffer.from(JSON.stringify({ overrides: { containerOverrides: [{args: "--item-id=123"}]}})).toString("base64"),
        },
    },
})

Now, even though I’m a little new to GCP, I believe I’m doing everything right! For some reason though, when the cloud tasks run, they always fails with status “UNAUTHORIZED” and when looking at the dashboard for my Cloud Run Jobs I don’t see any invocation for them

To test and make sure that my cloud task SA is correct, I’ve gone ahead and created an access-token as the SA from the gcloud CLI with the following:

gcloud auth print-access-token --impersonate-service-account=cloud-task-service-account@my-project.iam.gserviceaccount.com

Then used Bruno (postman/insomnia like app) to try making the call manually as the cloud task SA.

> POST https://run.googleapis.com/v2/projects/my-project-id/locations/us-west1/jobs/process-item:run
> authorization: Bearer ya29.c.c0AY...XYZ
> content-type: application/json
> data {"overrides":{"containerOverrides":[{"args":["--item-id=123"]}]}}

This succeeds just fine! This leads me to believe that there must be something wrong with how im enqueueing the tasks. The docs are incredibly confusing and even digging through YouTube videos I can’t seem to find any resources online that for from Cloud Run Job -> Cloud Tasks -> Cloud Run Job. Happy to add more details if needed but really curious if I’m just missing something here :/

After looking into this similar issue, I wonder if the issue is with the OIDC audience. I looked to the docs for the endpoint to trigger a cloud run job via REST and found this, but maybe there is another undocumented endpoint Im suppose to use with OIDC?