Having trouble locating element using xpath within a div containing ::before pseudo-element

When working with the web driver object, I often need to obtain a list of web elements.

findElements(By.xpath(""));

Usually, I use xpath like 

//*[@class=\"providers-list clearfix\"]
to retrieve the list. However, I encounter an error when trying to access an element within it:

<div class="providers-list clearfix">::before 
  <div class="data-container">..</div>
</div>

The xpath causing issues is:

//[@class=\"data-container\"]" as no such element: Unable to locate element: {"method":"xpath","selector":"//[@class="data-container"]"}

javaseleniumxpathweb-scrapingpseudo-element

Answer №1

When a like ::before is present, it indicates that the element has been:

  • Styled, or
  • Inserted before certain content.

This means the element is dynamic and to locate it, you need to use WebDriverWait to ensure visibilityOfElementLocated(), along with one of these Locator Strategies:

  • cssSelector:

    WebElement element = new WebDriverWait(driver, 20).until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("div.providers-list.clearfix div.data-container")));

  • xpath:

    WebElement element = new WebDriverWait(driver, 20).until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//div[@class='providers-list clearfix']//div[@class='data-container']")));

Answer №2

Here is the X-path to use: //*[contains (@class, 'clearfix')]

Answer №3

Have you attempted to retrieve data from the object that was returned by findElements? Here is an example of how it should be done:

List <WebElement> myList = driver.findElements(By.xpath("//*[@class=\"providers-list clearfix\"]"));
for(WebElement eachList:myList)
{
System.out.println(eachList.findElement(By.xpath("div[1]")).getText());
}

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Selenium - Tips for entering text in a dynamically generated text field using Javascript!

I'm fairly new to the world of web scraping and browser automation, so any guidance would be greatly appreciated! Using Python's Selenium package, my objective is: Navigate to Login using the provided username & password Complete my order thr ...

javascriptpythonseleniumselenium-webdriver

Python script that iterates through multiple pages to gather a list of elements

In my software, I have a database table containing information about users. However, due to the large number of users, this table is split into multiple pages. I am looking for a way to extract a complete list of all users across these pages using Selenium ...

pythonlistloopsselenium

Tips for converting a return value to a string and utilizing it in a different method:

Just starting out as a Ruby programmer, I'm currently working on an automation test using Ruby and Capybara. My system involves a webtable from which I need to extract certain cell values to be used in another method. This is the code I've come ...

rubystringseleniumcapybara

The @JsonAnyGetter annotation supersedes the PropertyNamingStrategy

Scenario: In my project using Jackson 2.4.5, I have a dynamic bean that needs to be serialized into JSON. This bean can store some of its state in 'optional' properties in an internal map and utilizes @JsonAnyGetter on an accessor method that re ...

javajsonjacksonspring-boot

What is preventing https://username:[email protected] from functioning properly?

I am encountering an issue with logging into a website that displays a popup asking for my username and password as soon as I land on the site. Even though I have tried using https://username:<a href="/cdn-cgi/l/email-protection" class="__cf_email__" d ...

javafirefoxselenium

Selenium Alert: An invalid or illegal selector has been specified, even though the selector is technically valid

Having trouble locating and clicking an element based on its class name using Selenium. After running the code, encountered this error: https://pastebin.com/1V2hHFGK. Confirmed that the element with the class name does exist: https://gyazo.com/ab0faa7f523a ...

pythonseleniumselenium-webdriver

Is there a way to customize the JSON-wrapped key for a class without being able to annotate the class with @JsonTypeInfo?

I have the following code snippet: import org.codehaus.jackson.map.*; public class MyPojo { int id; public int getId() { return this.id; } public void setId(int id) { this.id = id; } public static void main(String[] args) throws ...

javajsonjacksonwrapper

Unable to access the Selenium Proxy while configuring it with BrowserMob

Having successfully created a Selenium proxy using BrowserMob on my local PC, I encountered an error when attempting to run the same code on a server (Windows Server 2008 R2 Standard) which reads "cannot connect to tunnel". I have experimented with variou ...

seleniumproxyselenium-chromedriverbrowsermob

Having Trouble Logging into Instagram Using C# Selenium

Hey everyone, I recently helped my mom set up an Instagram account for her work. I wanted to create an auto follower bot for her using Selenium, but every time I try to test it, Instagram locks me out and asks me to try again in a few minutes. The proble ...

c#seleniumselenium-webdriverselenium-chromedriverinstagram

Tips on ensuring the text value is completely loaded in Selenium before proceeding with the next automation step

When experimenting with Selenium, I attempted the code snippet below to extract row values. public static void main(String[] args) throws InterruptedException { WebDriverManager.chromedriver().setup(); ChromeDriver driver = new ChromeDriver(); ...

javaseleniumcookiesautomationwebdriverwait

Choosing an Option Using Regular Expressions in a Dropdown Menu Using Selenium

I am encountering difficulties when trying to select values from a unique Dropdown Button species. This particular button behaves like a dropdown menu, but it is coded as a button in the website's HTML. Selenium keeps throwing an error whenever I atte ...

pythonregexselenium

Encountering difficulties launching a Springboot application via Docker Compose using Selenium RemoteWebDriver

I am currently working on containerizing my Spring Boot application which utilizes selenium/standalone-firefox-debug. I have created a Docker Compose file, but when I try to bring it up, I encounter an error: "Could not start a new session. Possible cau ...

seleniumdockerspring-bootdocker-compose

Eclipse is throwing a "Source not found" error during the debugging process

While debugging my code, I encountered the following error: When I set a breakpoint and try to debug, it throws a 'source not found' error. I have already clicked on "Edit Source Lookup Path" in Eclipse and added my project, but unfortunately, i ...

javaeclipseselenium-webdriver

Tips for launching a hyperlink in a new tab using Python and Selenium

When browsing a website, I am having trouble opening links in a new tab using Selenium with Python. Despite trying various solutions suggested here, the new tabs simply refuse to open as intended. (I've attempted different methods, but none seem to wo ...

pythonseleniumselenium-webdriver

Utilizing Selenium to extract information from a data table

I am facing a challenge with extracting data from a paginated table using Selenium. Despite having code that can successfully retrieve data, it only grabs the first 50 results out of the entire table. I believe utilizing Selenium to iterate through all p ...

pythoncsvselenium-webdriverweb-scraping

The system encountered an issue when attempting to convert a VALUE_STRING into an instance of java.util.ArrayList

I have developed a REST service using Jersey and deployed on AppEngine. This service implements the PUT verb, which consumes media type application/json. Data binding is handled by Jackson. The PUT verb accepts an enterprise-departments relation in JSON f ...

javajsonarraylistjacksonjson-deserialization

Can someone explain why the output of this code is not the expected value?

I've created a selenium program using Python: from selenium import webdriver i=0 driver = webdriver.Chrome(executable_path="/usr/local/bin/chromedriver") driver.get("https://problogger.com/jobs/") elems = driver.find_elements_by_t ...

pythonselenium

Issue encountered with Maven and Selendroid: Unable to find specified archiver for 'apk' files

Currently, I am in the process of creating a test case that utilizes selendroid-standalone for managing the connection to a mobile phone. The project is structured with Maven and includes multiple submodules. To incorporate this functionality, I have inclu ...

androidmavenseleniumselendroid

Activating a button using Selenium in Python

Mission: Utilize Selenium and Python to seek out a specific company name on LinkedIn by entering it into the search bar. Then navigate to the "Companies" section in order to find information about companies related to that keyword, not just individuals fro ...

python-3.xseleniumselenium-webdriverselenium-chromedriver

Error message encountered in Selenium WebDriver due to an element becoming stale

I've been attempting to automate a scenario in which I am required to select and click on all the dropdown options one by one. However, my code only clicks on the first option and displays an error message stating that the element is stale and not cli ...

javaeclipseseleniumselenium-webdriver