We are trying to configure a private GKE cluster, using PSC and autopilot, so that the control plane can be accessed from PUPI addresses (because we have to). The google documentation suggest it should be possible but we just can’t see how to configure it.

If the cluster endpoint is public then we can set pupi addresses on the master authorised network configuration (because it allows public ip addresses) and kubectl can connect to the private endpoint (with a little hacking of the .kube/config) file.

However, if the cluster endpoint is private then it doesn’t allow PUPI addresses to be used on the master authorised network configuration (which seems reasonable).

This document suggests it is possible to use PUPIs by setting the -enable-google-cloud flag (assuming that is the –enable-google-cloud-access flag in gcloud) but when trying to set this with a private endpoint, it says it is invalid.

The following works but only allows private IP addresses on the master authorized networks

gcloud container clusters create-auto <CLUSTER_NAME> 
    --region=<REGION>  
    --network=<NETWORK_NAME>  
    --subnetwork=<SUBNET_NAME> 
    --service-account=<SERVICE_ACCOUNT_EMAIL> 
    --cluster-version=1.29.1 
    --enable-private-nodes 
    --enable-private-endpoint  
    --enable-master-authorized-networks 
    --master-authorized-networks 10.10.10.0/24

The following creates the cluster but fails to add the master authorized networks as they are publicly addressable

finished with error: Invalid master authorized networks: network "11.10.10.0/24" is not a reserved network, which is required for private endpoints.

gcloud container clusters create-auto <CLUSTER_NAME> 
    --region=<REGION>  
    --network=<NETWORK_NAME>  
    --subnetwork=<SUBNET_NAME> 
    --service-account=<SERVICE_ACCOUNT_EMAIL> 
    --cluster-version=1.29.1 
    --enable-private-nodes 
    --enable-private-endpoint  
    --enable-master-authorized-networks 
    --master-authorized-networks 11.10.10.0/24

The following fails because it is not valid

invalid value for "cluster.master_authorized_networks_config": "cluster.master_authorized_networks_config.gcp_public_cidrs_access_enabled" cannot be enabled if private endpoint is enabled

gcloud container clusters create-auto <CLUSTER_NAME> 
    --region=<REGION>  
    --network=<NETWORK_NAME>  
    --subnetwork=<SUBNET_NAME> 
    --service-account=<SERVICE_ACCOUNT_EMAIL> 
    --cluster-version=1.29.1 
    --enable-private-nodes 
    --enable-private-endpoint  
    --enable-master-authorized-networks 
    --enable-google-cloud-access