I am trying to figure out the root cause and a remedy path of a security vulnerability that is flagged by the GCP security scanner for one of my GCP Python Cloud Functions.
NOTE:
I understand that Cloud Functions deployment/containers are supposed to be used as “black magic” with no knowledge of internals (as explained here: SO similar question) – but this mode of operation fails when GCP’s own security scanning is reporting security vulnerabilities on its own containers – and redeployment of CFs does not “solve” the problem. Thus, I am trying to dig deeper to understand how it actually works.
What is puzzling in this case is that, while my CF is a Python one – the security flag came for a Go library:
CVE-2024-24790 - Critical
Package: go
Package type: Go stdlib
First, all my questions – and then below are details of why and how I am trying to find the answers , and where I’ve got so far (not too far 🙁 )
- Main question 1: how can one find which specific COS (Container Optimized OS) version is used to host a specific Cloud Function? (see below why I think it is important)
- Main question 2: how to force GCP deploy to use the latest container image ? I see available container images for Python 39 and 310 runtimes as of 07/07/2024 (Tag = python39_20240707_3_9_19_RC00) but an older image is used based on what I see in the container build logs (v-python39_20240630_3_9_19_RC00) ?
- Other sub-questions that I think will help to find the root of the problem:
-
- is it possible to specify which COS to use when deploying your CF via ‘gcloud deploy’ command?
-
- how to find which COS is used in a specific CF Runtime docker container image – that are published here: https://cloud.google.com/functions/docs/concepts/execution-environment#python
-
- given a running Docker container with Ubuntu (specific to some Runtime version of Python) – how can I find if there is Go installed and what version? Disclaimer: I know nothing about Go, so this might be a very simple question! See below my attempts of finding this out – but basically I pulled the image and ran a container from it locally – ‘sh’ to it – and tried to search for ‘go’ or ‘stdlib’ via ‘find’ and ‘dpkg’ – didn’t find any. ‘go’ is also not on PATH.
Details:
Why COS:
Since my CF is Python, with no Go dependencies – it is clear that the Go library came into play from somewhere down the stack – most likely the container that hosts my CF. Searching Google docs for any mention of this specific vulnerability led me to this link: https://cloud.google.com/release-notes
where I noticed this entry:
July 01, 2024
Container Optimized OS
Security: Upgraded dev-lang/go to v1.21.11. This fixes CVE-2024-24790 and CVE-2024-24789.
So, breadcrumb #1: I need to use COS older than 07/01/2024 !
But, how do I find which COS is used? – that info I could not find, thus the question #1.
Digging further – back to the Vulnerability info:
CVE-2024-24790 - Critical
Package: go
Package type: Go stdlib
Clicking through to the “VIEW FIX” link – shows more info:
Fixed in: 1.21.11
Details: Version 1.21.7
Affected location: cpe:/o:canonical:ubuntu_linux:18.04
Package: go
Package type: Go stdlib
File location(s): /cnb/lifecycle/launcher
…
Breadcrumb #2: looks like this deployed CF is running on a container that is using Ubuntu 18.04 with Go stdlib version of 1.21.7 – but the fix is in the stdlib version 1.21.11
Next step: find a version of Runtime that has Go stdlib version 1.21.11
easier said than done: could not find this info directly, but:
What I could find is what specific Runtime version and Docker image is most likely used for deployed CFs:
https://cloud.google.com/functions/docs/concepts/execution-environment#python
has links to docker images for each supported Python version:
clicking on the image link – gives you the latest image IDs:
Latest python310 images:
— BTW – today is 07/15/2024 – so 7 days ago would be at least 07/08/2024 – why is 07/02/2024 date in the Tag??
Latest python39 images:
This image ID could also be spotted in the CF container build logs:
"timestamp": "2024-07-11T14:57:24.194595801Z",
"severity": "INFO",
"labels": {
"build_step": "Step #0 - "fetch"",
"build_tags": "p-gcf,r-python39,v-python39_20240630_3_9_19_RC00,b-python_20240708_RC00"
}
Interesting that even though I deployed my CF on 07/11/2024 – it did not use the latest image from 07/07/2024, but rather from 06/30/2024 …
Deploying on 07/15/2024 – same result – an older Docker image for the container seems to be used:
textPayload: "Step #2 - "build": Adding image label google.run-image: us-east4-docker.pkg.dev/serverless-runtimes/google-18-full/runtimes/python39:python39_20240630_3_9_19_RC00"
timestamp: "2024-07-15T10:05:36.600844638Z"
And the security scan still reports the same vulnerability present.
Assumption: latest versions of the container images, created AFTER 07/01/2024, should NOT have this vulnerability – based on the release note about COS, where it says that the CVE-2024-24790 vulnerability was fixed in the COS on 07/01/2024
Since I could not force GCP to deploy my CF using the latest container version available:
Next step: build and run a Docker container from the latest images locally – both 07/07/2024 (python39) and 07/02/2024 (python310) – and see what version of Go and ‘stdlib’ are installed
Results: could not find anything ‘go’-related, but since I have never used Go – I might be doing this wrong – this is what I tried:
python39 runtime – container created from the latest image, then ‘sh’ to the running instance:
confirmed Ubuntu version of the container:
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.6 LTS
Release: 18.04
Codename: bionic
$
search for installed ‘go’ packages/libs:
$ dpkg -l | grep go
-- nothing related to Go
$ dpkg -l | grep std
-- also nothing related to Go - here are full results:
ii libpython3-stdlib:amd64 3.6.7-1~18.04 amd64 interactive high-level object-oriented language (default python3 version)
ii libpython3.6-stdlib:amd64 3.6.9-1~18.04ubuntu1.13+esm1 amd64 Interactive high-level object-oriented language (standard library, version 3.6)
ii libstdc++6:amd64 8.4.0-1ubuntu1~18.04 amd64 GNU Standard C++ Library v3
ii libzstd1:amd64 1.3.3+dfsg-2ubuntu1.2 amd64 fast lossless compression algorithm
$
Next step: Assumption: since Python3.9 runtime is on Ubuntu18.04, maybe the Python3.10 runtime, which uses Ubuntu 20.04 (per https://cloud.google.com/functions/docs/concepts/execution-environment#python) uses a later version of Go ??
Repeated the same steps as for python39 – redeployed CF with python310 runtime, built and ran local Docker container from the latest (07/02/2024) Docker image version
Same results:
- same vulnerability is present
- the 06/30/2024 container image is used when redeploying the CF
- local container from 310 image – still do not see anything related to Go using ‘find’ and ‘dpkg’
One difference between vulnerability report for python39 vs python310 deployed CFs:
-
in python39, the ‘VIEW FIX’ shows this location of the offender: Affected location: cpe:/o:canonical:ubuntu_linux:18.04
-
in python310:
Fixed in: 1.21.11
Details: Version: 1.21.7
Affected location: Unsupported OS
Package: go
Package type: Go stdlib
Why “Unsupported OS” if GCP builds this Runtime container from its own “supported” images??
Also , it is still not clear how to match a specific Docker image to a COS version – nothing about COS is logged anywhere, no docs either
3