How to determine COS version of a deployed Python Cloud Function to debug security vulnerability report?

I am trying to figure out the root cause and a remedy path of a security vulnerability that is flagged by the GCP security scanner for one of my GCP Python Cloud Functions.

NOTE:

I understand that Cloud Functions deployment/containers are supposed to be used as “black magic” with no knowledge of internals (as explained here: SO similar question) – but this mode of operation fails when GCP’s own security scanning is reporting security vulnerabilities on its own containers – and redeployment of CFs does not “solve” the problem. Thus, I am trying to dig deeper to understand how it actually works.

What is puzzling in this case is that, while my CF is a Python one – the security flag came for a Go library:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
<code>CVE-2024-24790 - Critical
Package: go
Package type: Go stdlib
</code>
<code>CVE-2024-24790 - Critical Package: go Package type: Go stdlib </code>
CVE-2024-24790 - Critical
Package: go 
Package type: Go stdlib

First, all my questions – and then below are details of why and how I am trying to find the answers , and where I’ve got so far (not too far 🙁 )

  • Main question 1: how can one find which specific COS (Container Optimized OS) version is used to host a specific Cloud Function? (see below why I think it is important)
  • Main question 2: how to force GCP deploy to use the latest container image ? I see available container images for Python 39 and 310 runtimes as of 07/07/2024 (Tag = python39_20240707_3_9_19_RC00) but an older image is used based on what I see in the container build logs (v-python39_20240630_3_9_19_RC00) ?
  • Other sub-questions that I think will help to find the root of the problem:
    • is it possible to specify which COS to use when deploying your CF via ‘gcloud deploy’ command?
    • how to find which COS is used in a specific CF Runtime docker container image – that are published here: https://cloud.google.com/functions/docs/concepts/execution-environment#python
    • given a running Docker container with Ubuntu (specific to some Runtime version of Python) – how can I find if there is Go installed and what version? Disclaimer: I know nothing about Go, so this might be a very simple question! See below my attempts of finding this out – but basically I pulled the image and ran a container from it locally – ‘sh’ to it – and tried to search for ‘go’ or ‘stdlib’ via ‘find’ and ‘dpkg’ – didn’t find any. ‘go’ is also not on PATH.

Details:

Why COS:

Since my CF is Python, with no Go dependencies – it is clear that the Go library came into play from somewhere down the stack – most likely the container that hosts my CF. Searching Google docs for any mention of this specific vulnerability led me to this link: https://cloud.google.com/release-notes
where I noticed this entry:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
<code>July 01, 2024
Container Optimized OS
Security: Upgraded dev-lang/go to v1.21.11. This fixes CVE-2024-24790 and CVE-2024-24789.
</code>
<code>July 01, 2024 Container Optimized OS Security: Upgraded dev-lang/go to v1.21.11. This fixes CVE-2024-24790 and CVE-2024-24789. </code>
July 01, 2024
Container Optimized OS
Security: Upgraded dev-lang/go to v1.21.11. This fixes CVE-2024-24790 and CVE-2024-24789.

So, breadcrumb #1: I need to use COS older than 07/01/2024 !

But, how do I find which COS is used? – that info I could not find, thus the question #1.

Digging further – back to the Vulnerability info:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
<code>CVE-2024-24790 - Critical
Package: go
Package type: Go stdlib
</code>
<code>CVE-2024-24790 - Critical Package: go Package type: Go stdlib </code>
CVE-2024-24790 - Critical
Package: go 
Package type: Go stdlib

Clicking through to the “VIEW FIX” link – shows more info:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
<code>Fixed in: 1.21.11
Details: Version 1.21.7
Affected location: cpe:/o:canonical:ubuntu_linux:18.04
Package: go
Package type: Go stdlib
File location(s): /cnb/lifecycle/launcher
</code>
<code>Fixed in: 1.21.11 Details: Version 1.21.7 Affected location: cpe:/o:canonical:ubuntu_linux:18.04 Package: go Package type: Go stdlib File location(s): /cnb/lifecycle/launcher … </code>
Fixed in: 1.21.11
Details: Version 1.21.7
Affected location:  cpe:/o:canonical:ubuntu_linux:18.04
Package: go
Package type: Go stdlib
File location(s):   /cnb/lifecycle/launcher 
…

Breadcrumb #2: looks like this deployed CF is running on a container that is using Ubuntu 18.04 with Go stdlib version of 1.21.7 – but the fix is in the stdlib version 1.21.11

Next step: find a version of Runtime that has Go stdlib version 1.21.11

easier said than done: could not find this info directly, but:

What I could find is what specific Runtime version and Docker image is most likely used for deployed CFs:
https://cloud.google.com/functions/docs/concepts/execution-environment#python
has links to docker images for each supported Python version:

clicking on the image link – gives you the latest image IDs:

Latest python310 images:

— BTW – today is 07/15/2024 – so 7 days ago would be at least 07/08/2024 – why is 07/02/2024 date in the Tag??

Latest python39 images:

This image ID could also be spotted in the CF container build logs:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
<code>"timestamp": "2024-07-11T14:57:24.194595801Z",
"severity": "INFO",
"labels": {
"build_step": "Step #0 - "fetch"",
"build_tags": "p-gcf,r-python39,v-python39_20240630_3_9_19_RC00,b-python_20240708_RC00"
}
</code>
<code>"timestamp": "2024-07-11T14:57:24.194595801Z", "severity": "INFO", "labels": { "build_step": "Step #0 - "fetch"", "build_tags": "p-gcf,r-python39,v-python39_20240630_3_9_19_RC00,b-python_20240708_RC00" } </code>
"timestamp": "2024-07-11T14:57:24.194595801Z",
"severity": "INFO",
"labels": {
  "build_step": "Step #0 - "fetch"",
  "build_tags": "p-gcf,r-python39,v-python39_20240630_3_9_19_RC00,b-python_20240708_RC00"
 }

Interesting that even though I deployed my CF on 07/11/2024 – it did not use the latest image from 07/07/2024, but rather from 06/30/2024 …

Deploying on 07/15/2024 – same result – an older Docker image for the container seems to be used:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
<code>textPayload: "Step #2 - "build": Adding image label google.run-image: us-east4-docker.pkg.dev/serverless-runtimes/google-18-full/runtimes/python39:python39_20240630_3_9_19_RC00"
timestamp: "2024-07-15T10:05:36.600844638Z"
</code>
<code>textPayload: "Step #2 - "build": Adding image label google.run-image: us-east4-docker.pkg.dev/serverless-runtimes/google-18-full/runtimes/python39:python39_20240630_3_9_19_RC00" timestamp: "2024-07-15T10:05:36.600844638Z" </code>
textPayload: "Step #2 - "build": Adding image label google.run-image: us-east4-docker.pkg.dev/serverless-runtimes/google-18-full/runtimes/python39:python39_20240630_3_9_19_RC00"
timestamp: "2024-07-15T10:05:36.600844638Z"

And the security scan still reports the same vulnerability present.

Assumption: latest versions of the container images, created AFTER 07/01/2024, should NOT have this vulnerability – based on the release note about COS, where it says that the CVE-2024-24790 vulnerability was fixed in the COS on 07/01/2024

Since I could not force GCP to deploy my CF using the latest container version available:

Next step: build and run a Docker container from the latest images locally – both 07/07/2024 (python39) and 07/02/2024 (python310) – and see what version of Go and ‘stdlib’ are installed

Results: could not find anything ‘go’-related, but since I have never used Go – I might be doing this wrong – this is what I tried:

python39 runtime – container created from the latest image, then ‘sh’ to the running instance:

confirmed Ubuntu version of the container:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
<code>$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.6 LTS
Release: 18.04
Codename: bionic
$
</code>
<code>$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04.6 LTS Release: 18.04 Codename: bionic $ </code>
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.6 LTS
Release:    18.04
Codename:   bionic
$

search for installed ‘go’ packages/libs:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
<code>$ dpkg -l | grep go
-- nothing related to Go
$ dpkg -l | grep std
-- also nothing related to Go - here are full results:
ii libpython3-stdlib:amd64 3.6.7-1~18.04 amd64 interactive high-level object-oriented language (default python3 version)
ii libpython3.6-stdlib:amd64 3.6.9-1~18.04ubuntu1.13+esm1 amd64 Interactive high-level object-oriented language (standard library, version 3.6)
ii libstdc++6:amd64 8.4.0-1ubuntu1~18.04 amd64 GNU Standard C++ Library v3
ii libzstd1:amd64 1.3.3+dfsg-2ubuntu1.2 amd64 fast lossless compression algorithm
$
</code>
<code>$ dpkg -l | grep go -- nothing related to Go $ dpkg -l | grep std -- also nothing related to Go - here are full results: ii libpython3-stdlib:amd64 3.6.7-1~18.04 amd64 interactive high-level object-oriented language (default python3 version) ii libpython3.6-stdlib:amd64 3.6.9-1~18.04ubuntu1.13+esm1 amd64 Interactive high-level object-oriented language (standard library, version 3.6) ii libstdc++6:amd64 8.4.0-1ubuntu1~18.04 amd64 GNU Standard C++ Library v3 ii libzstd1:amd64 1.3.3+dfsg-2ubuntu1.2 amd64 fast lossless compression algorithm $ </code>
$ dpkg -l | grep go
-- nothing related to Go

$ dpkg -l | grep std
-- also nothing related to Go - here are full results:
ii  libpython3-stdlib:amd64           3.6.7-1~18.04                               amd64        interactive high-level object-oriented language (default python3 version)
ii  libpython3.6-stdlib:amd64         3.6.9-1~18.04ubuntu1.13+esm1                amd64        Interactive high-level object-oriented language (standard library, version 3.6)
ii  libstdc++6:amd64                  8.4.0-1ubuntu1~18.04                        amd64        GNU Standard C++ Library v3
ii  libzstd1:amd64                    1.3.3+dfsg-2ubuntu1.2                       amd64        fast lossless compression algorithm
$

Next step: Assumption: since Python3.9 runtime is on Ubuntu18.04, maybe the Python3.10 runtime, which uses Ubuntu 20.04 (per https://cloud.google.com/functions/docs/concepts/execution-environment#python) uses a later version of Go ??

Repeated the same steps as for python39 – redeployed CF with python310 runtime, built and ran local Docker container from the latest (07/02/2024) Docker image version

Same results:

  • same vulnerability is present
  • the 06/30/2024 container image is used when redeploying the CF
  • local container from 310 image – still do not see anything related to Go using ‘find’ and ‘dpkg’

One difference between vulnerability report for python39 vs python310 deployed CFs:

  • in python39, the ‘VIEW FIX’ shows this location of the offender: Affected location: cpe:/o:canonical:ubuntu_linux:18.04

  • in python310:

    Fixed in: 1.21.11
    Details: Version: 1.21.7
    Affected location: Unsupported OS
    Package: go
    Package type: Go stdlib

Why “Unsupported OS” if GCP builds this Runtime container from its own “supported” images??

Also , it is still not clear how to match a specific Docker image to a COS version – nothing about COS is logged anywhere, no docs either

3

Trang chủ Giới thiệu Sinh nhật bé trai Sinh nhật bé gái Tổ chức sự kiện Biểu diễn giải trí Dịch vụ khác Trang trí tiệc cưới Tổ chức khai trương Tư vấn dịch vụ Thư viện ảnh Tin tức - sự kiện Liên hệ Chú hề sinh nhật Trang trí YEAR END PARTY công ty Trang trí tất niên cuối năm Trang trí tất niên xu hướng mới nhất Trang trí sinh nhật bé trai Hải Đăng Trang trí sinh nhật bé Khánh Vân Trang trí sinh nhật Bích Ngân Trang trí sinh nhật bé Thanh Trang Thuê ông già Noel phát quà Biểu diễn xiếc khỉ Xiếc quay đĩa Dịch vụ tổ chức sự kiện 5 sao Thông tin về chúng tôi Dịch vụ sinh nhật bé trai Dịch vụ sinh nhật bé gái Sự kiện trọn gói Các tiết mục giải trí Dịch vụ bổ trợ Tiệc cưới sang trọng Dịch vụ khai trương Tư vấn tổ chức sự kiện Hình ảnh sự kiện Cập nhật tin tức Liên hệ ngay Thuê chú hề chuyên nghiệp Tiệc tất niên cho công ty Trang trí tiệc cuối năm Tiệc tất niên độc đáo Sinh nhật bé Hải Đăng Sinh nhật đáng yêu bé Khánh Vân Sinh nhật sang trọng Bích Ngân Tiệc sinh nhật bé Thanh Trang Dịch vụ ông già Noel Xiếc thú vui nhộn Biểu diễn xiếc quay đĩa Dịch vụ tổ chức tiệc uy tín Khám phá dịch vụ của chúng tôi Tiệc sinh nhật cho bé trai Trang trí tiệc cho bé gái Gói sự kiện chuyên nghiệp Chương trình giải trí hấp dẫn Dịch vụ hỗ trợ sự kiện Trang trí tiệc cưới đẹp Khởi đầu thành công với khai trương Chuyên gia tư vấn sự kiện Xem ảnh các sự kiện đẹp Tin mới về sự kiện Kết nối với đội ngũ chuyên gia Chú hề vui nhộn cho tiệc sinh nhật Ý tưởng tiệc cuối năm Tất niên độc đáo Trang trí tiệc hiện đại Tổ chức sinh nhật cho Hải Đăng Sinh nhật độc quyền Khánh Vân Phong cách tiệc Bích Ngân Trang trí tiệc bé Thanh Trang Thuê dịch vụ ông già Noel chuyên nghiệp Xem xiếc khỉ đặc sắc Xiếc quay đĩa thú vị
Trang chủ Giới thiệu Sinh nhật bé trai Sinh nhật bé gái Tổ chức sự kiện Biểu diễn giải trí Dịch vụ khác Trang trí tiệc cưới Tổ chức khai trương Tư vấn dịch vụ Thư viện ảnh Tin tức - sự kiện Liên hệ Chú hề sinh nhật Trang trí YEAR END PARTY công ty Trang trí tất niên cuối năm Trang trí tất niên xu hướng mới nhất Trang trí sinh nhật bé trai Hải Đăng Trang trí sinh nhật bé Khánh Vân Trang trí sinh nhật Bích Ngân Trang trí sinh nhật bé Thanh Trang Thuê ông già Noel phát quà Biểu diễn xiếc khỉ Xiếc quay đĩa
Thiết kế website Thiết kế website Thiết kế website Cách kháng tài khoản quảng cáo Mua bán Fanpage Facebook Dịch vụ SEO Tổ chức sinh nhật