"Encountering a System.NullReferenceException while attempting to read browser logs using Selenium

Currently, I am using C#, selenium and chromeWebDriver. However, I am encountering an issue when attempting to access the browser console log file with selenium. The error message I receive is:

System.NullReferenceException: 'Object reference not set to an instance of an object.'

private void button1_Click(object sender, EventArgs e)
{
    ChromeOptions options = new ChromeOptions();
    options.SetLoggingPreference(LogType.Browser, LogLevel.Warning);
    IWebDriver driver = new ChromeDriver(options);

    driver.Url = "https://www.google.com/";

    var entries = driver.Manage().Logs.GetLog(LogType.Browser); // System.NullReferenceException
    foreach (var entry in entries)
    {
        Console.WriteLine(entry.ToString());
    }
}

Answer №1

Here's my temporary solution until Selenium 4 is released (which will also be compatible with Selenium 4). This code snippet is a quick and rough demonstration of how it can be achieved. Feel free to modify and enhance it as needed.

using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Remote;
using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Reflection;
using System.Text;

namespace GetChromeConsoleLog
{
    internal static class Program
    {
        private static void Main()
        {
            // configure options
            var options = new ChromeOptions();
            options.SetLoggingPreference(LogType.Browser, LogLevel.All);

            // perform necessary actions
            var driver = new ChromeDriver(options)
            {
                Url = "https://www.yahoo.com/"
            };

            var logs = driver.GetBrowserLogs();

            // retrieve and display logs using the extension method GetBrowserLogs
            foreach (var log in driver.GetBrowserLogs())
            {
                Console.WriteLine($"{log["timestamp"]}: {log["message"]}");
            }

            // cleanup
            driver.Dispose();

            // keep the console open
            Console.WriteLine();
            Console.WriteLine("Press any key to exit...");
            Console.ReadLine();
        }
    }

    public static class WebDriverExtensions
    {
        public static IEnumerable<IDictionary<string, object>> GetBrowserLogs(this IWebDriver driver)
        {
            // not a chrome driver
            if(driver.GetType() != typeof(ChromeDriver))
            {
                return Array.Empty<IDictionary<string, object>>();
            }

            // setup
            var endpoint = GetEndpoint(driver);
            var session = GetSession(driver);
            var resource = $"{endpoint}session/{session}/se/log";
            const string jsonBody = @"{""type"":""browser""}";

            // execute
            using (var httpClient = new HttpClient())
            {
                var content = new StringContent(jsonBody, Encoding.UTF8, "application/json");
                var response = httpClient.PostAsync(resource, content).GetAwaiter().GetResult();
                var responseBody = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
                return AsLogEntries(responseBody);
            }
        }

        private static string GetEndpoint(IWebDriver driver)
        {
            // setup
            const BindingFlags Flags = BindingFlags.Instance | BindingFlags.NonPublic;

            // get RemoteWebDriver type
            var remoteWebDriver = GetRemoteWebDriver(driver.GetType());

            // obtain executor instance > obtain internalExecutor from this instance
            var executor = remoteWebDriver.GetField("executor", Flags).GetValue(driver) as ICommandExecutor;

            // get URL
            var uri = executor.GetType().GetField("remoteServerUri", Flags).GetValue(executor) as Uri;

            // result
            return uri.AbsoluteUri;
        }

        private static Type GetRemoteWebDriver(Type type)
        {
            if (!typeof(RemoteWebDriver).IsAssignableFrom(type))
            {
                return type;
            }

            while (type != typeof(RemoteWebDriver))
            {
                type = type.BaseType;
            }

            return type;
        }

        private static SessionId GetSession(IWebDriver driver)
        {
            if (driver is IHasSessionId id)
            {
                return id.SessionId;
            }
            return new SessionId($"gravity-{Guid.NewGuid()}");
        }

        private static IEnumerable<IDictionary<string, object>> AsLogEntries(string responseBody)
        {
            // setup
            var value = $"{JToken.Parse(responseBody)["value"]}";
            return JsonConvert.DeserializeObject<IEnumerable<Dictionary<string, object>>>(value);
        }
    }
}

get logs

Answer №2

While I have successfully utilized the mentioned property in the past, I am currently facing difficulties getting it to function with ChromeDriver version 75 and above. Upon further investigation, I discovered that similar issues have been raised regarding this matter on this page.

Answer №3

Although the issue was reported as fixed in GitHub issue #7323 here, testing with ChromeDriver Nuget version 77.0.3865.4000 still shows that the issue persists.

Testing with the latest stable version of Chromedriver 78.0.3904.7000 confirms that the problem has not been resolved.

Attempts to use workarounds mentioned in Selenium issue #7335 here have been made, but even though they enable driver instantiation, the logs remain inaccessible and null.

One workaround involves setting

typeof(CapabilityType).GetField(nameof(CapabilityType.LoggingPreferences), BindingFlags.Static | BindingFlags.Public).SetValue(null, "goog:loggingPrefs");
when creating a chromedriver instance.

According to MatthewSteeples in the same issue, the fix has been implemented but not yet released on Nuget. It is expected to be included in the next release.

Edit: The reason for using an older ChromeDriver Nuget is to facilitate running automated tests locally and in the Hosted Azure Devops release pipeline without needing to manually adjust the Nuget version.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

The selector is not valid: Utilizing XPath and Selenium, the output of the xpath query "//*[@id='topstuff']/div/div/p[1]/text()[2]" is displayed as [object Text]

I seem to be facing a challenge in understanding how to achieve this task. What I'm trying to accomplish is relatively simple - I want my automated Google search to flag instances where it does not return any results. Here's an excerpt from my co ...

What is the best approach for finding the xPath of this specific element?

Take a look at this website Link I'm trying to capture the popup message on this site, but I can't seem to find the element for it in the code. Any ideas? ...

Is it possible to easily extract the text linked to a label that exists in a radio button using Selenium WebDriver?

Looking at the html snippet below, how can I determine if "Male" is already selected? <input id="radio1" type="radio" value="1" name="sex> <label class="radio" for="radio1">Female</label> <input id="radio2" type="radio" value="2" name ...

Automate button clicking on web pages using Python with Selenium

While I am able to access the tag below through css_selector, I am unsure how to perform a button click. <a title="Sign In" onclick="toggleLogin('signInBlock','regBenefitsBlock');" href="javascript:void(0);">Sign In</a> I ...

Untangling Internet JSON data

I am attempting to parse the JSON string into objects and then display it in a richtextbox public void Form1_Load(object sender, EventArgs e) { using (var webClient = new System.Net.WebClient()) { var json = webClient.Dow ...

What are the problems with Internet Explorer in Selenium WebDriver?

Currently dealing with a mouse-over issue in IE while using webdriver code. The functionality works smoothly in Chrome and Firefox, however, the problem arises only in IE. How can I resolve this? Initially focusing on an element and then clicking on a li ...

A guide on utilizing Xpath to reference a tag within a class

Trying to obtain the website link by using xpath and selenium with a reference to the class pv-contact-info__contact-type ci-websites. [Here is the html snippet being referenced][1] sel = Selector(text=driver.page_source) website = sel.xpath("//*[@class= ...

Error encountered with Selenium script execution

As a newcomer to selenium, I require assistance with the following script where I am attempting to input a value via a text field. Below you can find the code snippet. import org.openqa.selenium.By; import org.openqa.selenium.JavascriptExecutor; import or ...

An issue with Selenium web scraping arises: WebDriverException occurs after the initial iteration of the loop

Currently, I am executing a Selenium web-scraping loop on Chrome using MacOS arm64. The goal is to iterate through a list of keywords as inputs in an input box, search for each one, and retrieve the text of an attribute from the output. A few months ago, I ...

in Selenium, locate the immediate sibling of an element that contains a specific child element identified by a CSS selector

I am attempting to target a specific element within a table structure like the one below: <table> <tbody> <tr> <td> <span>MySpan1</span> </td> <tr> <tr> <td> </td> <td> <di ...

Selenium Chromedriver fails to redirect to specified website

Currently, my code looks like this: chrome_options = Options() chrome_options.add_extension(r"C:\Users\x\OneDrive\Desktop\pp\crxSolver.crx") driver = webdriver.Chrome(r'C:\Users\x\OneDrive\Desktop&bso ...

How can a Python program obtain the system username of the user currently running it?

Currently, I am working on developing a Python web scraping tool with Selenium that involves loading the uBlock Origin extension. I am facing difficulty in figuring out how to make the program recognize the user's system username so that I can specify ...

What could be causing the error stating that there is no module named 'msedge'?

My code snippet includes: from selenium import webdriver from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.common.by import By from selenium.webdriver.support import expected_conditions as EC from msedge.selenium_tools import ...

Can TestNG handle multiple test suites simultaneously?

Is it possible to run multiple test suites at once? I know we can have multiple tests in one suite, but I am looking to run more than one suite simultaneously. Previous answers on this topic seem unclear. ...

Populate the text box with the bb code using Selenium in Python

I'm in the process of creating a test for the discussion board, and I want to experiment with the forum's BB code. Here is an example of the BB code I am working with: [B][FONT=Trebuchet MS][SIZE=7]Meteor[/SIZE][/FONT][/B] [COLOR=#000000][FONT=V ...

The functionality of mouse hover is malfunctioning on IE11 and Firefox

When attempting to mouse hover on a particular element using the Action class in Selenium and then move to another sub-element to click, the mouse cursor ends up pointing elsewhere. These are the Internet Explorer capabilities that have been configured al ...

inexperienced with selenium, unfamiliar with potential errors

Hello, I am a Python 3.6 beginner seeking assistance with my code and the error it's generating. I would greatly appreciate any help in identifying what is going wrong. Thank you. #!/usr/bin/env python #coding: utf-8 from selenium import webdriver ...

The website's reaction to the request for URLs was not what was anticipated

I'm trying to load a website and scrape all of the links, which is usually simple but I encountered an unusual response today: links = WebDriverWait(web, 20).until(EC.presence_of_all_elements_located((By.XPATH,'//a[@href]'))) print("Thi ...

Accessing a website and retrieving a document with Python

I am seeking assistance in improving the following script to successfully click on the export button. Although the script navigates to the report's page, it does not actually interact with the export button: from selenium import webdriver options = ...

"Enhancing test automation in Selenium: A guide on sending multiple values to the same input

Is there a way to send multiple input values for the same input field in selenium using python? Currently, my code only sends a single value to the input field. I am looking to modify it to test and run for multiple values. from selenium import webdrive ...