Finding the value within a div element with Selenium and Python

How can I extract the value inside a div using xpath or css_selector?

This is the HTML markup:

<div class="catalog-products view-tile" data-catalog-products="" data-slider-available="" data-primary-as-icon=""><div data-id="product" class="catalog-product ui-button-widget" data-product="5a7b4c6d-0bc3-11ec-a2b0-00155dfc8232" data-code="4862447" data-preview-slider-inited="1"><div class="catalog-product__image"><a class="catalog-product__image-link" href="/product/5a7b4c6d0bc3c823/videokarta-msi-geforce-210-n210-1gd3lp/" data-toggle-slider=""><picture><source type="image/webp" media="(min-width: 768px)" etc

I am trying to retrieve the data-code value 4862447

  1. Attempted to access via xpath:

    /html/body/div[1]/div/div[2]/div[2]/div[3]/div/div[1]/div[1]

    In the Chrome console, this part was highlighted:

    <div data-id="product" class="catalog-product ui-button-widget" data-product="5a7b4c6d-0bc3-11ec-a2b0-00155dfc8232" data-code="4862447" data-preview-slider-inited="1">

    Struggling to retrieve the data-code value.

  2. Tried using css_selector:

    div[data-id='product']

    Received the same line:

    <div data-id="product" class="catalog-product ui-button-widget" data-product="5a7b4c6d-0bc3-11ec-a2b0-00155dfc8232" data-code="4862447" data-preview-slider-inited="1">

    Still unable to extract the value.

pythonseleniumxpathcss-selectorswebdriverwait

Answer №1

In order to retrieve and print the value stored in the data-code attribute, you will need to utilize WebDriverWait for ensuring the visibility_of_element_located() function is successful. You can employ one of the provided locator strategies below:

  • Using CSS_SELECTOR:

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.catalog-products.view-tile > div.catalog-product.ui-button-widget[data-id='product']"))).get_attribute("data-code"))

  • Using XPATH:

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[@class='catalog-products view-tile']/div[@data-id='product' and @class='catalog-product ui-button-widget'][.//a[@class='catalog-product__image-link' and @href and @data-toggle-slider]]"))).get_attribute("data-code"))

  • Note : Don't forget to include the necessary imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

A unique sorting technique specifically designed for organizing elements within a three-dimensional numpy array

Imagine you have a 3D numpy array with p panels, each containing r rows and c columns. You want to sort only the columns of a specific panel, causing the corresponding elements on the other panels to rearrange accordingly. If you're unfamiliar with s ...

pythonsortingmultidimensional-array

I'm feeling a bit lost when it comes to understanding the rebuild/update_index process in Django-H

After deleting a record from my Django app, I initially ran the update_index command followed by the rebuild_index command. The record was still searchable after the first step, but appeared to be gone after the second step when I checked my app. Panicked, ...

pythondjangodjango-haystack

Leveraging NumPy for array index discovery

If I am working with the following array: array = ['KDI', 'KDI', 'KDU', 'KDA', 'ANU', 'AMU', 'BDU', 'CDU', 'CDU', 'DAI', 'DAH'], dtype='

pythonnumpyfor-loopnumpy-ndarray

How can we determine the selected web form element using the tab key in Selenium WebDriver with Java?

My current approach involves using the sendKeys(key,Keys.TAB) method to navigate a form. Actions action = new Actions(driver); CharSequence key = null; for(int i=0;i<42;i++) { action.sendKeys(key,Keys.TAB).build().perform(); } Aft ...

javaseleniumselenium-webdriver

Modify button appearance upon clicking in tkinter

I have a fun project idea to create buttons with names from a list. The goal is to make the button's relief change from groove to sunken when clicked, but there can only be one sunken button at a time. If another button is clicked while one is already ...

pythonbuttontkinter

Utilize Skimage to Divide Image

Is there a way to divide an image into fourths using skimage? I've only been able to find information on cropping images, but I specifically need to break the image into quarters for symmetry comparison purposes. Are there any functions or methods a ...

pythonsplitscikit-image

Python - how to extract distinct pairs from lists of varying lengths in a single operation?

I've encountered a complex scenario involving multi-line strings within my coding journey: mystr = """ ACTIVE 3/1.1 IDLE 3/1.1 IDLE 3/1.2 IDLE 3/1.2 ACTIVE 3/1.3 IDLE 3/1.3 ACTIVE 3/1.4 IDLE 3/1.4 ACT ...

pythonstringlistfor-loop

Create a hierarchical dictionary structure by nesting dictionaries containing child elements from an array

Let's say I have fetched an array of dicts from a web API. Each dict contains keys like name, description, 'parent', and children. The children key consists of an array of dicts. Here is a hypothetical example for better understanding: [ ...

pythonarraysdictionarytree

Authentication unsuccessful (pyodbc.InterfaceError) ('28000', "[28000] [Microsoft][ODBC SQL Server Driver][SQL Server] User 'xxxx' is unable to login

Having trouble connecting to the SQL server using Python 3.8. Should I consider downgrading to Python 3.7 or am I missing something? CHECK OUT MY CODE AND ERROR MESSAGE: params = urllib.parse.quote_plus \ ('Driver={SQL Server};Server=tcp:xxxxx.d ...

pythonazureconnection

Leverage the power of pandas for implementing data categorization

Click here to view the image.I am working with an Excel file that contains 365 rows representing each day and 2310 columns representing regions, with temperature data for each day. I need to categorize the temperatures into intervals of 1° and then dete ...

pythonpandasnumpy

When Python-Selenium triggers Firefox, the browser does not show the URL

I attempted to utilize the code provided on http://pypi.python.org/pypi/selenium to open the Firefox browser and enter a URL in the address bar. The Firefox browser successfully launches, but it fails to input the URL into the address bar, resulting in an ...

pythonfirefoxselenium

Issue encountered when attempting to pass a value from an Excel spreadsheet to a text box web element using Selenium in Python: TypeError occurs as object type 'float' does not support the len() function

Test Code: sheet = inputbook.sheet_by_name('Page1') for x in range(rowcount) : #formname = driver.find_element_by_id('walletRequest') #formname.click() value1 = sheet.cell_value(x,0) str(value1) #time.sleep(10) anam ...

pythonseleniumselenium-webdriverwebdriversendkeys

Working with multiple dropdowns dynamically in VBA and utilizing either webelements or selectelements

Recently, I have been working with Selenium and VBA and encountering an issue. When trying to work with dynamic drop-downs, I am unable to refresh my web element or select element and sometimes receive error messages. For instance, I am dealing with a web ...

excelvbaseleniumweb-scrapingdrop-down-menu

Steps to converting a list into a key for a dictionary

In my dataset, I have multiple lists each containing 4 elements. LoL=[[1,1,1,1],[4,2,3,[1,3]],[4,5,3,[0,4]]] The last element in some of the lists can be a sub-list like [0,4] in [4,5,3,[0,4]]. I am looking to convert these lists into keys for a dictio ...

pythonlistdictionary

Grafana Single Stat Panel made easy with a Simple JSON response

Currently, I am utilizing Grafana 3.0 beta 5 along with the Simple JSON datasource plugin to build out a backend in Python to extract business data from my ERP system. I understand that the Simple JSON plugin requires 4 REST methods: / query search ann ...

pythonjsongrafana

Error: 'WebElement' does not contain a property called 'below'

I am currently in search of the relative element to the login button located on the discord.com/login page. My goal is to identify the register button/element using a function I came across in the selenium documentation. Once found, I intend to click on ...

pythonseleniumselenium-webdriver

Use a boolean ndarray to mask out nans and replace them with specific values in

Looking to apply a boolean mask for row manipulation in a numpy array: isnan = np.isnan(self.X[:, AGE_COLUMN].astype(float)) self.X[isnan, AGE_COLUMN] = np.mean(self.X[:, AGE_COLUMN].astype(float)) Both isnan and X have the same dtype. Initially, identi ...

pythonnumpynanmasked-array

[Protractor][Scroll] I need assistance with scrolling my webpage using a while loop. Could someone please help me troubleshoot the code?

When this function is called, it initiates scrolling and then pauses the browser for a 2-second period. scrollToElement(webElement: any) { browser.executeScript('window.scrollTo(0,400);').then(()=>{ console.log("sleepin ...

angulartypescriptseleniumselenium-webdriverprotractor

To access the first element of the list in the instance, simply call instance(0, 0) in Python

Hey there! I'm trying to work with a class called Grid, which has an instance grid. My goal is to have grid(0, 0) return the value of grid.content[0][0]. The content property of the grid object is a list containing other lists. class Grid(): def __i ...

python

Preventing selenium from displaying WebDriver manager initialization logs

Every time I start a new selenium driver, I encounter the following message: ====== WebDriver manager ====== Current chromium version is 90.0.4430 Get LATEST chromedriver version for 90.0.4430 chromium Driver [/root/.wdm/drivers/chromedriver/linux64/90.0.4 ...

pythonpython-3.xseleniumloggingwebdrivermanager-python