Issue with updating Selenium ChromeDriverManager through a proxy network

I am trying to use selenium and Chrome with Python to access a website. Below is a snippet of my code:

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.service import Service

# PROXY='https://myproxy.com:3128'
PROXY = 'https://username:<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="3949584a4a4e564b5d795440494b564140175a5654">[email protected]</a>:3128'
proxyuser='username' #this is the proxy user name used in above string
proxypwd='password' #this is the proxy password used in above string


chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % PROXY)
chrome_options.add_argument("ignore-certificate-errors");

chrome = webdriver.Chrome(options=chrome_options,service=Service(ChromeDriverManager().install()))

chrome.get("https://www.google.com")
while True:
    print('ok')

I need to authenticate with a corporate proxy server to make this work. I'm not sure how to pass login credentials and proxy settings for installation of chromedriver. When I run the code without a proxy, everything works fine. However, when using a proxy connection, I encounter the following error:

[WDM] - ====== WebDriver manager ======
[WDM] - Current google-chrome version is 105.0.5195
[WDM] - Get LATEST chromedriver version for 105.0.5195 google-chrome
Traceback (most recent call last):
  File "D:\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
...
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

Due to corporate policies, bypassing the proxy is not an option.

request("get", url, params=params, **kwargs)

The highlighted line in the errors seems to be causing the issue. It appears that the driver cannot establish a connection to determine the new version available online because the request function should have proxy settings - potentially missing the necessary **kwargs values which ideally should include proxy argument. But I'm uncertain about this.

pythonseleniumselenium-webdriverproxywebdriver-manager

Answer №1

After some adjustments, the code has been updated with a custom download manager implementation which is now functioning correctly.

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from webdriver_manager.core.download_manager import WDMDownloadManager
from webdriver_manager.core.http import HttpClient
from selenium.webdriver.chrome.service import Service
from requests import Response
import urllib3
import requests
import os
os.environ['WDM_SSL_VERIFY'] = '0'
capabilities = webdriver.DesiredCapabilities.CHROME
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

PROXY = "https://username:[email protected]:3128"
proxyuser='username'
proxypwd='password'
opt = webdriver.ChromeOptions()
opt.add_argument('--proxy-server=%s' % PROXY)
opt.add_argument("ignore-certificate-errors")

class CustomHttpClient(HttpClient):
    def get(self, url, params=None) -> Response:
        proxies={'http': 'http://username:[email protected]:3128',
         'https': 'http://username:[email protected]:3128',
         }
        return requests.get(url, params,proxies=proxies,verify=False)

http_client = CustomHttpClient()
download_manager = WDMDownloadManager(http_client)
chrome = webdriver.Chrome(service=Service(ChromeDriverManager(download_manager=download_manager).install()),options=opt)

chrome.get("https://www.google.com")
while True:
    print('ok')

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Dimensions of PyTorch model's input

I have a unique PyTorch model that I loaded and I am curious about determining its input shape. Is there a method like this? model.input_shape Can anyone provide assistance in obtaining this information? Updated: The methods print() and summary() do not ...

pythondeep-learningpytorchconv-neural-network

TestNG is failing to run after the setup

What could be the issue in the code that is preventing it from running? Please provide guidance on how to resolve this. package tc1; import java.util.concurrent.TimeUnit; import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org ...

seleniumselenium-webdriverautomationtestng

What is the ideal location for storing Django manager code?

I have a question about Django patterns that is quite simple. Typically, my manager code is located in models.py. However, what should be done when models.py becomes overly extensive? Are there any alternative patterns available to keep the manager code se ...

pythondjangodesign-patternsdjango-modelsdjango-managers

Executing UI automation with Selenium and Java involves the automation of two separate Chrome browsers simultaneously

I currently have 3 classes structured like this: `public class SubmitOrderTest extends DefaultTest { @Test public void submitOrder() { LandingPage landingPage = launchWebsite(); landingPage.login("standard_user", "secret_sauce" ...

javaselenium-webdriverwebautomation

What is the proper way to store strings in variables within Python, and subsequently access and utilize those variables in various functions or methods within the

Recently, I encountered a straightforward issue regarding printing emojis in Python. After some research, I discovered three main methods to achieve this: Using UNICODE representation of the emoji Using CLDR names of the emoji Utilizing the emoji module ...

pythonstringunicodeemojirawstring

Error encountered: Element cannot be clicked on at specified coordinates - Angular/Protractor

Recently, I have been testing out CRUD functionality in an Angular app using Protractor. One recurring issue I've encountered is with the create/edit buttons, which all open the same modal regardless of the page you're on. The frustrating part i ...

javascriptangularjsseleniumautomationprotractor

using conditions AND in Selenium and Java

When searching for an item, I want the result to appear in a specific element if found. If not found, another element should appear. I attempted to use OR with expected conditions as shown below: wait.until(ExpectedConditions.or(ExpectedConditions.visibil ...

javaseleniumexpected-exception

Selenium is having trouble finding the iframe element

I'm struggling to locate the input element within an iframe using Selenium. Even after trying switchTo().frame(id) and switchTo().frame(index), I am still unable to find it. This is the code snippet I have been using: driver.switchTo().defaultConten ...

javaseleniumiframe

Utilize Python's Selenium webdriver to operate Firefox in the background

Currently, I am involved in a website scraping endeavor utilizing Selenium with Python. One question that has crossed my mind is whether it's feasible to initiate the Firefox browser in the background or launch Firefox on a separate workspace within U ...

pythonseleniumwebdriverweb-scrapinguser-agent

Is there a way to display a list of dropdown menu options using Selenium with Python?

I am currently attempting to utilize Selenium Python and its PhantomJS function in order to print out all the items from the first drop-down menu on this particular page (). Unfortunately, I keep encountering a No Attribute error message. If anyone could ...

pythonhtmlseleniumselenium-webdriverweb-crawler

What is the reason behind the lack of granularity in the ruby `require` statement?

Unique example for better understanding In my script file, labeled as c.py, I have defined two classes: Elephant and Giraffe. When working with Python, to utilize the Elephant class from the c.py file, an import statement is used: import c print(Elepha ...

pythonrubynode.js

Handling Exceptions in TkinterHaving a Tkinter Exception

Currently, I am still in the process of working on a small Tkinter project that involves creating a simple YouTube video downloader. However, every time I attempt to use the tkinter window, I encounter an exception callback and a value error. Exception in ...

pythontkinterpytube

The system does not identify 'ffmpeg' as a command that can be executed internally or externally

I need to convert a .TS file to an MP4 File using subprocess in Python. Here is the code I've written: import subprocess infile = 'vidl.ts' subprocess.run(['ffmpeg', '-i', infile, 'out.mp4']) I have also made s ...

pythonffmpegmp4

Experiencing difficulties launching InstaPy

After successfully setting up selenium and geckodriver to work with firefox, I decided to give InstaPy a try for the first time. With firefox installed, I was able to launch the browser, go to Instagram, and log in with the code snippet below: browser ...

pythonseleniumwebdriverinstagramgeckodriver

It appears that the :first / :first-child selector is not functioning properly with Selenium 2

During my transition from Selenium 1 to Selenium 2, I encountered an issue. The structure I currently have is similar to the following: <ul id="documentType"> <li><a href="http://somelink.com">first</a></li> <li& ...

javaseleniumcss-selectors

Is it possible to create HTML tables without including paragraphs within the cells using docutils rst2html?

If my input rst file looks like this: +--------------------------------+ | Table H1 | +-------------+------------------+ | Table H2a | Table H2b | +=============+==================+ | a1 | b1 | +------ ...

pythonhtmlcssrestructuredtext

Unable to find element to click on "Join Now" button in Google Meet using Selenium and Python for browser automation

I am working on creating a web automation tool that can automatically join Google Meet using Selenium with the Firefox driver in Python. So far, I have successfully signed up, muted the microphone, and turned off the camera without any issues. However, I a ...

pythonseleniumwebfirefoxwebautomation

Leveraging PyTables for creating an index on a massive 500 gigabyte HDF5 file

Is it possible to transfer a large 500GB-800GB indexed table into HDF5 and then query for specific records based on keys? In an HDF5 file, data access is based on integer "row" numbers, which means that an external 'key to row number map' would ...

pythonhdf5pytablesh5pybigdata

The method in the calling class cannot be resolved, even though it exists and compiles successfully in both the interface and implementation classes

I am currently working on developing a series of Selenium tests using Java. One crucial part of my program involves the utilization of a class named ResponseInterceptor.java. This particular class leverages methods from both my IReporter.java interface and ...

javaseleniuminterface

Python code: Attempting to retrieve the hour, minute, and second values from a datetime object

As a newcomer to the Python environment, I am working on manipulating datetime objects by using the method replace(hour=...,minute=...,second=...) in an iterative manner and storing the results at each step in a pd.Series. The objective is to have a pd.Ser ...

pythonpython-3.xdatetime