When Selenium is not in headless mode, it cannot capture a screenshot of the entire website

Disclaimer: Although there is a similar question already out there, none of the answers provided worked for a headless browser. Therefore, I have decided to create a more detailed version addressing this specific issue (the original question I referred to can be found here: Take screenshot of full page with Selenium Python with chromedriver)

Greetings to all.

I've come across a seemingly simple yet challenging problem. I am trying to capture a screenshot of a NON-HEADLESS BROWSER on a display with dimensions of 1920x1080 (this will be relevant later), capturing the entire webpage rather than just the visible portion.

Here's what I've attempted:

import os
import time

from selenium import webdriver

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument("--start-maximized")
chromedriver = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'chromedriver.exe')
chrome = webdriver.Chrome(chromedriver, options=chrome_options)

url = 'https://stackoverflow.com/'

chrome.get(url)
time.sleep(2)

total_height = chrome.execute_script("return document.body.parentNode.scrollHeight") + 1000

chrome.set_window_size(1920, total_height)

time.sleep(2)
chrome.save_screenshot("screenshot1.png")
chrome.quit()

The above code works perfectly fine with Headless mode. However, if I remove the --headless option, Selenium attempts to resize itself, but because it tries to exceed the 1080 height of the display, it automatically adjusts to 1080, resulting in a screenshot size of 1920x1080. In an ideal scenario, I would want Selenium to go headless only during the moment when it takes the screenshot (which, unfortunately as per my knowledge, is not feasible).

Other commonly used methods that do not work when the browser is not headless:

el = driver.find_element_by_tag_name('body')
el.screenshot(path)
original_size = driver.get_window_size()
required_width = driver.execute_script('return document.body.parentNode.scrollWidth')
required_height = driver.execute_script('return document.body.parentNode.scrollHeight')
driver.set_window_size(required_width, required_height)
driver.find_element_by_tag_name('body').screenshot(path)  # avoids scrollbar
driver.set_window_size(original_size['width'], original_size['height'])
element = chrome.find_element_by_tag_name('body')
element_png = element.screenshot_as_png
with open("test2.png", "wb") as file:
    file.write(element_png)

With the headless option

https://i.stack.imgur.com/KV5Qg.png

Without the headless option

https://i.stack.imgur.com/mglqz.png

Answer №1

If you want to capture screenshots of a webpage while scrolling, you can utilize Screenshot_Clipping.

To do so in Python3, simply execute the following command:

pip install Selenium-Screenshot

Next, instantiate a Screenshot object:

ob=Screenshot_Clipping.Screenshot()

Once you have the object, you can use ob.full_Screenshot to take a screenshot of the entire screen.

For the full source code, check out the PythonFullPageScreenShot project on Github.

Keep in mind that the maximum height for a screenshot is limited to 10000 pixels. You may need to scale it to 100 to capture the full height of the website.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

What is the process for executing Selenium IDE scripts in Selenium RC?

As a newcomer to using the selenium testing tool, I am seeking guidance on running selenium IDE scripts within selenium RC. I would greatly appreciate examples and screenshots to help me better understand the process. ...

Trying to execute Python scripts in Selenium IDE as a test suite

I'm facing a couple of issues when trying to run multiple Python test scripts exported by the Selenium IDE Python Remote Control plugin formatter. 1) The browser window automatically closes after a python script is completed. I am using Firefox for m ...

Error encountered while executing Selenium Web Driver's executeScript method: [JavascriptError: missing ) after argument list] with name: 'JavascriptError'

return driver.executeScript("\ console.log('Error: Incorrect selection');\ $('option:selected', 'select[name='who']').removeAttr('selected');\ $('select[name='who'] ...

Error: Unable to locate template code within the Django program

I'm having trouble implementing a slider on my home page. Here is the code I have so far: model.py: class Static(models.Model): title = models.CharField(max_length=50) description = models.TextField() images1 = mode ...

Selenium in Python struggling to locate dynamically loaded JavaScript table following automated login process

I am currently utilizing Selenium with Python3 to navigate a Service Now Website. The process is as follows: Selenium loads the ServiceNow URL and I use sendKeys to automate entering the username and password. Once the page is loaded, there is a table of ...

Locate an element by its TEXT using SeleniumBase

When using SeleniumBase, I have a requirement to click on an element that is identified as shown below: <span class="text"> Contato PF e PJ </span> The specific text inside the element (Contato PF e PJ) needs to be ...

Develop a program to transform the octal permissions of a specified file into symbolic permissions

I am looking to develop a script that takes a file name as input and outputs its permissions in symbolic form. For example, if the octal permission is 755, the script should display: owner - rwx group - r-x others - r-x ...

A guide on extracting data from a Script element in an HTML file and saving it to a CSV file using Python Selenium

Looking for help with optimizing a Python script that extracts data from the Script tag on an HTML web page. The current output saved in a CSV file is not meeting my requirements: (1) the format is incorrect, and (2) there is unnecessary content included. ...

Error: Attempting to append to a `dict` object in JSON is not a valid operation

Why isn't the append method working on dictionary objects? Here is my code snippet: with open (file) as f: for line in f: review = json.loads(line) review.append((review['reviewText'],review['overall']))#append ...

Guide on filling in credentials in Facebook popup using Webdriver with Javascript

On my website, I have a feature where users can authenticate using Facebook. Currently, my site is not public, but you can see a similar process in action at agar.io. Just click on "Login and play" and then click on "Sign in with Facebook". This will open ...

An issue occurred while executing PhantomJS using Selenium's RemoteWebDriver

Launching a selenium grid hub using the default startup command: java -jar selenium-server-standalone-2.33.0.jar -role hub Also, initiating PhantomJS in its webdriver mode on the same machine with the following command: phantomjs --webdriver=8080 --web ...

The Python code is malfunctioning when used as a Geoprocessing Service

I've been struggling with this issue for quite some time now and I could really use some help. My goal is to create a geoprocessing service that takes permit information from a GDB, writes it to a file, and then opens it on the user's computer th ...

Add elements to a list in Python by incrementing the array value

Consider having a collection of lists: name_list = ['John', 'Smith'], ['Sarah', 'Nichols'], ['Alex', 'Johnson'] There is also another arbitrary list, for example, with colors: color_list = [&ap ...

Selenium - Comparing text_to_be_present_in_element with text_to_be_present_in_element_value

Would you be able to provide an example in Python that demonstrates the contrast between text_to_be_present_in_element and text_to_be_present_in_element_value? I came across this link which attempts to clarify how text_to_be_present_in_element functions, ...

What are some solutions for troubleshooting errors when attempting to delete sections of a string from a Python DataFrame?

After searching through multiple threads, I'm encountering issues with the pandas replace method and regex re.sub method (Python 3.x). Despite my efforts, I am unable to get the desired results. I am struggling to extract data from html due to its ir ...

Ways to eliminate the 3rd warning of collections.abc DeprecationWarning

My software is experiencing an influx of warnings originating from a third-party package transformers/modeling_deberta.py:18: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprec ...

Continuing in a loop until the error ceases or until 15 seconds have elapsed

My goal is to create a loop that attempts to click on an element until it is found. If this process exceeds a 15-second limit, a warning message will be displayed. The current issue is that the loop does not respect the 15-second limit and gets stuck in a ...

Is it possible to identify the beginning of a download using Selenium?

Currently, I am using Python and Selenium to download a large batch of files. To ensure that each file is successfully downloaded, I am implementing a basic time.sleep() function, but I want to enhance efficiency and guarantee the completion of each downlo ...

Multithreading in Python does not function properly in Visual Studio

I'm currently working on a project involving the multiprocessing library. However, when I try to run my code in Visual Studio on my Windows PC, it seems to get stuck and nothing is being printed. Here's the snippet of code that's causing me ...

Encountering a problem with serializing data.Json, unable to dump the data

import json data={"name":"John","age":30,"cars": [{ "name":"Ford", "models":["Fiesta", "Focus", "Mustang" ] },{ "name":"BMW", "models":[ "320", "X3", "X5"] },{ "name":"Fiat", "models":[ "500", "Panda" ] }]} with open('newjson','w') as ...