Extracting information from a webpage

I am trying to extract the text from the XPath provided below using Selenium. However, I keep encountering the following traceback error:

date = driver.find_element_by_xpath('//[@id="General"]/fieldset/dl/dd[5]/text()').text()
print(date)

Error trace:

InvalidSelectorException: invalid selector: The result of the xpath expression "//*[@id="General"]/fieldset/dl/dd[5]/text()" is: [object Text]. It should be an element.

It seems that Selenium does not support parsing out text nodes (which is my end goal). Which other library would you recommend for this purpose?

pythonselenium

Answer №1

  • .text should be treated as an attribute, not a method, so there is no need for parentheses.

  • In xpath, the use of /text() should be avoided when working with Selenium as it cannot directly return text or attributes like @href. Instead, utilize .text for text and .get_attribute('href') for attributes.

date = driver.find_element_by_xpath('//[@id="General"]/fieldset/dl/dd[5]').text
print(date)

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Troubleshooting Timeout Problems with Selebiun Crawler in C#

I am encountering an error while running the following code. public void GetCategoriesSelenium() { string javascript = System.IO.File.ReadAllText(@"GetCategory.js"); CrawlerWebSeleniumJS.ExecuteScript("var finished;"); ...

javascriptc#seleniumselenium-webdriver

Testing a cucumber scenario by comparing the redirect URL with a different URL

Currently, I am working on writing Cucumber tests for my project. One issue I have encountered is that when clicking a certain button in my code, it redirects to another page with a fixed URL. How can I retrieve the current URL within the Cucumber test? I ...

javascriptseleniumcucumber

Can you tell me the distinction between using RemoteWebDriver's executeScript() and Selenium's getEval() for executing

Can you explain the distinction between these two pieces of code: RemoteWebDriver driver = new FirefoxDriver(); Object result = driver.executeScript("somefunction();"); and this: RemoteWebDriver driver = new FirefoxDriver(); Selenium seleniumDriver = ne ...

javajavascriptseleniumwebdriverselenium-webdriver

Encountered CSRF validation error while working with a Python Django backend in conjunction with React frontend using Axios for making POST requests

I recently completed a tutorial at and now I'm attempting to add a POST functionality to it. Despite obtaining the csrf from cookies and including it in the "csrfmiddlewaretoken" variable alongside a test message in json format for the axios function ...

javascriptpythondjangoreactjsaxios

The PhantomJs browser is not able to open my application URL

Recently, my scripts in PhantomJS browser have stopped running. Whenever I try to capture screens, all I get are black screens. To troubleshoot this, I manually opened a URL in PhantomJS using the command window and ran the script below to verify if it ope ...

javascriptseleniumphantomjs

What is the best approach for finding the xPath of this specific element?

Take a look at this website Link I'm trying to capture the popup message on this site, but I can't seem to find the element for it in the code. Any ideas? ...

javascriptseleniumxpathautomation

What's the best way to determine which of the two forms has been submitted in Django?

On my homepage, I have both a log_in and sign_up form. Initially, the log_in form is displayed by default, but when a user clicks on the Sign Up button, the sign_up form appears. These toggles switch depending on which button the user clicks. from django ...

javascriptjquerypythondjango

Executing selenium tests on Internet Explorer 11 on a Windows 10 1809 machine without encountering any new pop-up windows

While testing on my computer, I encountered an issue where my test would start successfully, but after opening and closing several Internet Explorer windows during the test, no new windows would open. There were no error messages displayed, and the test se ...

javascriptseleniumselenium-webdriverava

Selenium unable to interact with Javascript pop-up box

I am currently working on automating a feature for our web application, specifically a form of @mentioning similar to Facebook. On the front end, when a user types @ into a text input, the API is called to retrieve the list of users and display them in a b ...

javascriptpython-3.xseleniumpython-behave

Encountering an issue with WebDriver in the realm of JavaScript

I am struggling to use JavaScript to locate specific controls and send values to them. One example is changing the text in a textbox with the ID "ID" to "123456". Below is the code I tried: ((IJavaScriptExecutor)driver).ExecuteScript("document.getElement ...

c#javascripttestingseleniumwebdriver

Having trouble choosing an option from the dropdown menu with Puppeteer Js

I need help with Puppeteer JS to select the initial element in a dropdown. Any suggestions? Once I input the city name in the text field, I want to choose the first option from the dropdown menu. const puppeteer = require('puppeteer'); (async ...

javascriptseleniumpuppeteer

Nightwatch execute() function not technique following anticipate

After reviewing the documentation, I am confident that this code should work correctly. However, I am encountering an issue where something needs to run once the expect has finished, but it doesn't seem to be functioning as expected. Functioning Code ...

javascriptseleniumtestingnightwatch.js

I must interact with the video within the iframe by clicking on it

I am trying to interact with an iframe video on a webpage. Here is the code snippet for the video: <div class="videoWrapper" style="" xpath="1"> <iframe width="854" height="480" src="xxxxxxx" frameborder="0" allow="autoplay; encrypted-media" all ...

javascriptjavaseleniumselenium-webdriver

Enhance your JavaScript code by replacing Promise syntax with Async syntax and utilizing map() instead of a traditional For

I have a code snippet here that is functioning properly. However, I am interested in converting the Promise code in the middle of the function to Async code and replacing the for loop with map(). Can someone guide me on how to achieve this transformation ...

javascriptseleniumpromiseasync-awaites6-promise

Running only failed tests in Protractor can be achieved by implementing a custom script that

I've encountered a problem in my JavaScript file where some of the test cases fail intermittently, and I want to rerun only those that have failed. Is there any feature or solution available that can help with this issue? It's quite frustrating a ...

javascriptseleniumprotractor

Invoke a Python function from JavaScript

As I ask this question, I acknowledge that it may have been asked many times before. If I missed the answers due to my ignorance, I apologize. I have a hosting plan that restricts me from installing Django, which provided a convenient way to set up a REST ...

javascriptpythonangularjsclient-server

How can we use the Selenium JavascriptExecutor in Java to return an Object from JavaScript?

I've encountered an issue while running a JavaScript file using Java with Selenium in my application. When I execute the JavaScript file with JavascriptExecutor after logging in, I'm only getting a null return instead of a valid one. Below is a ...

javascriptjavaselenium

Selenium in C#: Timeout issue with SendKeys and Error thrown by JS Executor

Attempting to insert the large amount of data into the "Textarea1" control, I have tried two different methods. The first method successfully inserts the data but occasionally throws a timeout error, while the second method results in a JavaScript error. A ...

javascriptc#seleniumselenium-webdriver

A combination of Tor Browser, Selenium, and Javascript for

I have been attempting to use selenium with Tor, but unfortunately it is not functioning correctly. I have come across a library that allows for this functionality, however, it appears to only work with Python. Is there a way to accomplish this using Jav ...

javascriptseleniumtor

Error: The session ID is not currently active

I encountered a problem with my tests failing when running them on TFS, showing the following error: WebDriverError: No active session with ID Failed: No active session with ID Interestingly, these same tests are passing locally. Everything was working ...

javascriptseleniumtfsprotractorjasmine2.0