Using unique headers with Phantomjs in Selenium WebDriver

As per the information provided here, it is now feasible to modify headers. Currently, I am attempting to adjust the Accept-Language in the PhantomJS webdriver. The code snippet below does not seem to be effective:

DesiredCapabilities.PHANTOMJS['phantomjs.page.customHeaders.Accept-Language'] = 'ru-RU'
driver = webdriver.PhantomJS()

Is there a way to configure PhantomJS to include my desired header? It does not matter if it is within ghostdriver, phantomjs, or phantomjs-webdriver.

Answer №1

The most recent release of PhantomJS (1.9.1) was on Jun/5/2013. The pull request was merged on Jun/23/2013.

If you have version 1.9.1 of PhantomJS, custom headers will not function correctly.

You'll need to either build PhantomJS yourself or wait for the integration of ghostdriver changes and the release of a new version by PhantomJS.

  • Clone the PhantomJS repository
  • Clone the ghostdriver repository
  • Copy ghostdriver/src/* to phantomjs/src/ghostdriver recursively
  • Build PhantomJS

After building PhantomJS myself, I achieved the following result:

from selenium import webdriver

webdriver.DesiredCapabilities.PHANTOMJS['phantomjs.page.customHeaders.Accept-Language'] = 'ru-RU'
driver = webdriver.PhantomJS()
driver.get('http://httpbin.org/headers')
print(driver.page_source)

...
{
  "headers": {
    "Connection": "close",
    "Host": "httpbin.org",
    "Accept-Encoding": "gzip",
    "Accept-Language": "ru-RU",
    "User-Agent": "Mozilla/5.0 (Unknown; Linux i686) AppleWebKit/534.34 (KHTML, like Gecko) PhantomJS/1.10.0 (development) Safari/534.34",
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
  }
 ...

UPDATE

It is recommended to use PhantomJS 1.9.2+.

Answer №2

To demonstrate how to configure all headers, window size, and proxy settings in Selenium PhantomJS, I have provided a comprehensive example:

from selenium import webdriver

def setup_phantomjs_driver(*args, **kwargs):

    headers = { 'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
        'Accept-Language':'zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3',
        'User-Agent': 'Mozilla/5.0 (Windows NT 6.2; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0',
        'Connection': 'keep-alive'
    }

    for key, value in headers.items():
        webdriver.DesiredCapabilities.PHANTOMJS['phantomjs.page.customHeaders.{}'.format(key)] = value

    webdriver.DesiredCapabilities.PHANTOMJS['phantomjs.page.settings.userAgent'] = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36'

    driver =  webdriver.PhantomJS(*args, **kwargs)
    driver.set_window_size(1400,1000)

    return driver


def run_main():
    service_args = [
        '--proxy=127.0.0.1:9999',
        '--proxy-type=http',
        '--ignore-ssl-errors=true'
        ]

    driver = setup_phantomjs_driver(service_args=service_args)

    driver.get('http://cn.bing.com')

Important Information:

UserAgent is now configured using

phantomjs.page.settings.userAgent
rather than phantomjs.page.customHeaders

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Selenium technique for navigating iframes without specific identifiers

I encountered an issue while trying to switch to an iframe on a page I am scraping. The ID has been removed from the iframe, making it difficult for me to switch to it. Unfortunately, I have not been able to find any helpful documentation on this matter. I ...

Understanding PyQt Documentation

After installing PyQt GPL v4.6.2 for Python v3.1 and Qt by Nokia v4.6.0 (OpenSource), I noticed that the documentation in PyQt is not displaying properly. Even the example docs are blank. I am seeking a step-by-step guide from someone on how to access the ...

Issue encountered with Chromedriver version 2.36 when paired with Chrome version 65 resulting in an

I recently updated my ChromeDriver to version 2.36 after Chrome automatically updated to v65. However, when running tests now, I am encountering the following exceptions: Starting ChromeDriver 2.36.540470 (e522d04694c7ebea4ba8821272dbef4f9b818c91) on por ...

Is it necessary to hover over the menu in order to access the clickable element?

Be sure to hover over the menu first before trying to click on the element! The element is currently not visible and cannot be interacted with (WARNING: The server did not provide any stacktrace information) error on Selenium I have attempted the followi ...

Selenium script encountered a error while boot layer was being initialized

Just starting with selenium and trying to execute my first test case in eclipse, but encountering some errors along the way. Error popped up during initialization of boot layer java.lang.module.FindException: Unable to infer module descriptor for C:& ...

Determine the duration of the repeating decimal fraction's cycle

I am working on a Python program (version 3.6.5) that calculates the length of a repeating decimal like 1/7. The output should display something like: "Length: 6, Repeated Numbers: 142857". Here is what I have so far: n = int(input("Enter numerator: ")) ...

Executing Python code in Visual Studio without the need for the Python extension by compiling and running the program

Currently, I am utilizing Visual Studio 2019 as my chosen text editor for writing Python programs. My goal is to compile and run the program directly within Visual Studio without having to download the VS Python extension. Python 3.8 has been successfully ...

Selenium objects disappear from view

My selenium test is currently running smoothly at a resolution of 1920 * 1080. However, I have been tasked with adapting this test to work on different common resolutions, such as 1366 * 768. The issue arises when running the Selenium test on smaller reso ...

Managing Errors in Loops: Addressing the issue of loops not terminating when conditions are met using Selenium and

Developing a custom script to extract user details from AD and transfer that information into an automation tool for creating user accounts on a website. I've successfully navigated through the entire process, but encountering an issue with handling u ...

Tips for closing file upload dialog after clicking the upload button using Protractor

Trying to figure out how to upload an image file in protractor without having to click on the upload button. Whenever I try to locate the element "input[type = "file"]", it only appears after clicking on the upload button. After clicking the upload button ...

A streamlined approach to tackling extensive if-elif chains for better performance

Currently, I am referencing a value obtained from an HTML form which corresponds to price ranges. For instance: 0 represents any price, 1 denotes $0 to $100, and so on. I can't help but feel that there must be a more Pythonic or efficient approach to ...

Removing rows based on conditions in python

I am facing a multilabel classification challenge. My goal is to remove rows that have a value of 0 in all 35 columns of the data frame, except for the ['Doc'] column. Below is an example of the dataframe: Doc Big Small Int Bor Dr ...

How can I fix the issues with my Caesar cipher?

Here's the code snippet I'm using: text = input("Enter your text: ") shift = int(input("Enter the shift value: ")) def caesar_shift(text, shift): cipher = "" for i in text: if i.isalpha(): stayIn = ord(i) + shift ...

I am encountering an issue with implementing the (EC.presence_of_element_located(By.class, "")) function

I recently encountered a problem while working on my Python Selenium project. It seems that Python is having trouble recognizing the code snippet EC.presence_of_element_located. I am not sure why this error is occurring. Below is the segment of code where ...

Getting rid of add-ins popup when using Selenium webdriver with a personalized Firefox profile

While utilizing selenium webdriver with a customized firefox profile, I encounter the firefox Add-ons popup displaying 2 extensions: Firefx WebDriver 2.5.0 and Ubuntu Firefox Modifications 0.9rc2. Is there a way to remove this popup? I have checked the se ...

Exploring the world of Python mix-ins through the main typing class

I have a group of mixins within my class: class MyObject(MyObjectFilesMixin, MyObjectProcessingMixin, ...): def __init__(self, value): self.value = self.preprocess(value) One of the mixins is structured like this: class MyObjectFilesMixin: ...

Obtain the text using Selenium WebDriver's getText() method

I am currently using Selenium WebDriver's .getText() function in Java to extract some text from a webpage, but I am facing difficulties as the text is not present within the WebElement object. The specific text I am trying to retrieve is: "eNewsletter ...

Issue encountered with geckodriver: Unexpected argument '--websocket-port' was detected

I recently updated my Selenium project using the Bonigarcia autodownload webdriver project to the latest versions. I upgraded to Selenium 4.0.0 from Maven repo and also updated the Bonigarcia project to version 5.0.3. However, now when I try to run my test ...

The driver information displayed is as follows: driver version - RemoteWebDriver org.openqa.selenium.json.JsonException: The type cannot be determined from: <. The last character read was: <

Recently, I encountered an issue while trying to send a long string of characters (100-4000 characters) using the sendKeys function in Selenium grid running in parallel. The process was breaking midway and throwing the following exception. Has anyone else ...

Every conceivable permutation of lists derived from a given list

I am seeking to generate all possible combinations of lists with a specific size. For instance, let's consider a list K, where K = [3, 5, 2]. The code snippet below accomplishes this task for the given list. However, I am interested in finding a way t ...