Moving a Cromwell WDL workflow to GCP. Cromwell appears to really love using the “e” (for “efficiency”, i.e. slow and worthless) machine types. I had a task of “extract 6 ~10gb compressed files, then process them”, that I ran on both “e2-highcpu-8” and “n2-standard-8” machines.

Moving a Cromwell WDL workflow to GCP. Cromwell appears to really love using the “e” (for “efficiency”, i.e. slow and worthless) machine types. I had a task of “extract 6 ~10gb compressed files, then process them”, that I ran on both “e2-highcpu-8” and “n2-standard-8” machines.

Cromwell RetryWithMoreMemory instructions say I should set system.memory-retry-error-keys and memory_retry_multiplier in order to get retries to allocate more memory for a task when it fails with one of the strings in system.memory-retry-error-keys appearing in stderr.