I cannot create a dataproc cluster on compute engine in a given vpc/subnet.
even though the firewall rule exists in said vpc.
Firewall rule
Initialisation failed. Enabling Diagnostic mode com.google.cloud.hadoop.services.agent.AgentException: Cannot start master: Timed out waiting for 2 nodes. This usually happens when VM to VM communications are blocked by firewall rules. For additional details, see https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/network#firewall_rule_requirement