I’m working on a web-scrape project using Python and Selenium with a Chrome driver, which requires client certificates to access pages. I have 2 scenarios it must handle:
- Different certificates allow access to different URLs (e.g. Certificate A accesses URLs 1, 2 and 3, and Certificate B accesses URLs 4, 5 and 6)
- Multiple certificates can access the same URL (e.g. Certificate A and B both can access URLs 7, 8 and 9 – those URLs return different company-specific data with each different cert)
I’m on Windows/Windows Server, and have used the Registry entry AutoSelectCertificateForUrls, which auto-selects a certificate, based on URL (or wildcard). But for scenario #2 above, it does no good.
So ideally, I’d like to pass the URL and Cert name to the Python script, then have Chrome use that Cert when accessing the specified URL, but I’m not seeing a way to do that. So far, I have:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait, Select
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--allow-insecure-localhost')
chrome_options.add_argument('--ignore-ssl-errors=yes')
chrome_options.add_argument('--ignore-certificate-errors')
driver = webdriver.Chrome()
driver.get(url)
:
:
# scrape code here
Does anyone have good step-by-step instructions to handle this?