Tips for grabbing the following element locator path with Selenium?

I am working on a piece of code to download a CSV file. Here is the code snippet:

chromedriver = "/usr/lib/chromium-browser/chromedriver"
driver = webdriver.Chrome(chromedriver)
driver.get('https://www.aemo.com.au/Electricity/National-Electricity-Market-NEM/Data-dashboard#aggregated-data')
time.sleep(5)
driver.find_element_by_xpath("//div[@class='dashboard-tab-content-options-state']//div[@class='dashboard-tab-content-options-item' and text()='SA']").click()
button = driver.find_element_by_xpath("//div[@class='dashboard-tab-content-download-btn' and text()= 'Download Historic Data as .csv']")
button.click()

Originally, the code selects the SA field using the first xpath shown here: https://i.stack.imgur.com/ieVPA.png

However, I would like to make the selection from the options below, like this: https://i.stack.imgur.com/AaUhQ.png

This is the HTML structure:

<div id="dashADF" class="dashboard-tab-content" style="display: block;">
            <div class="dashboard-tab-content-row">
    <div class="dashboard-tab-content-options">
        <div class="dashboard-tab-content-text-container">
            <div class="dashboard-tab-content-title">Aggregated Price and Demand Data - Current Month</div>
        </div>
        <div class="dashboard-tab-content-options-state">
            <div data-tabname="dashADF" data-current="true" class="dashboard-tab-content-options-item active">QLD</div>
            <div data-tabname="dashADF" data-current="true" class="dashboard-tab-content-options-item">NSW</div>
            <div data-tabname="dashADF" data-current="true" class="dashboard-tab-content-options-item">VIC</div>
            <div data-tabname="dashADF" data-current="true" class="dashboard-tab-content-options-item">TAS</div>
            <div data-tabname="dashADF" data-current="true" class="dashboard-tab-content-options-item">SA</div>
        </div>
    </div>
</div>
<div class="dashboard-tab-content-row">
    <div class="dashboard-tab-content-download">
        <div data-tabname="dashADF" data-current="true" class="dashboard-tab-content-download-btn">Download Current Month</div>
    </div>
</div><hr>
<div class="dashboard-tab-content-row">
    <div class="dashboard-tab-content-options">
        <div class="dashboard-tab-content-text-container">
        </div>
        <div>
            <div class="dashboard-tab-content-options-state">
                <div data-tabname="dashADF" data-current="false" class="dashboard-tab-content-options-item active">QLD</div>
                <div data-tabname="dashADF" data-current="false" class="dashboard-tab-content-options-item">NSW</div>
                <div data-tabname="dashADF" data-current="false" class="dashboard-tab-content-options-item">VIC</div>
                <div data-tabname="dashADF" data-current="false" class="dashboard-tab-content-options-item">TAS</div>
                <div data-tabname="dashADF" data-current="false" class="dashboard-tab-content-options-item">SA</div>
            </div>
            <div class="dashboard-tab-content-options-time">
            </div>
        </div>
    </div>
</div>
<div class="dashboard-tab-content-row">
    <div class="dashboard-tab-content-download">
        <div data-tabname="dashADF" data-current="false" class="dashboard-tab-content-download-btn">Download Historic Data as .csv</div>
    </div>
</div>
        </div>
python-3.xseleniumselenium-webdriverxpathwebdriver

Answer №1

Here is the XPath I would utilize

//div[.='Aggregated Price and Demand Data - Historical']/following::div[.='SA']

This approach will specifically retrieve the desired element. You could encapsulate this in a function, enabling you to input the section name and button text for any required combination.

def click_button_in_section(section_name, button_text)
    WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//div[.='{}']/following::div[.='{}']".format(section_name, button_text)))).click()

This function can be invoked like so:

click_button_in_section("Aggregated Price and Demand Data - Historical", "SA")
click_button_in_section("Aggregated Price and Demand Data - Historical", "NSW")
click_button_in_section("Aggregated Price and Demand Data - Current Month", "TAS")
click_button_in_section("Aggregated Price and Demand Data - Current Month", "VIC")

... and similarly for other cases

Answer №2

If you need to click() on the SA tab in the Aggregated Price and Demand Data - Historical section, here is a solution for you:

  • Code Snippet:

    from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("start-maximized")
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get('https://www.aemo.com.au/Electricity/National-Electricity-Market-NEM/Data-dashboard#aggregated-data')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[@class='dashboard-tab-content' and @id='dashADF']//following::div[@class='dashboard-tab-content-options-item' and text()='SA'][2]"))).click()

  • Browser Screenshot:

https://i.stack.imgur.com/eMwRI.png

Answer №3

These two elements appear identical with the same text enclosed in tags, I am optimistic that this solution will be successful, give it a try:

driver.find_element_by_xpath("//div[@data-tabname='dashADF' and @data-current='false' and  text()='SA']");

driver.find_element_by_xpath("(.//*[normalize-space(text()) and normalize-space(.)='TAS'])[2]/following::div[1]");

driver.find_element_by_xpath("(.//*[normalize-space(text()) and normalize-space(.)='Download Historic Data as .csv'])[1]/preceding::div[2]");

driver.find_element_by_xpath("//div[@id='dashADF']/div[3]/div/div[2]/div/div[5]");

driver.find_element_by_xpath("//*[@id='dashADF']/div[3]/div/div[2]/div[1]/div[5]");

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

I am experiencing difficulty selecting choices from a drop-down menu in FireFox 48 when using the marionette driver and selenium 3

When testing with the new geckodriver and marionette enabled for FF48 and selenium 3, I encountered an issue where the test was unable to select options from a dropdown list using any element. Strangely, the test worked perfectly in FF45 and all other brow ...

firefoxselenium-webdriverwebdriverselenium-firefoxdrivergeckodriver

The Chrome Binary Stacktrace is elusive to Selenium in the current environment

I'm currently utilizing Selenium for a web scraping script and everything runs smoothly in general. However, when attempting to run the script in a different environment, I encountered the following error: Traceback (most recent call last): at block ...

pythonseleniumselenium-webdriverselenium-chromedriver

The webpage freezes when attempting to run jQuery with Selenium

I'm currently facing an issue where my selenium script hangs the webpage whenever I try to find an element using jQuery. The script doesn't execute and a pop up appears in the browser with the message "A script on this page may be busy, or it may ...

javascriptjqueryhtmlselenium

Utilizing Selenium WebDriver in Python to locate and interact with WebElements

Currently, I am utilizing Selenium for real-time web scraping. However, I am encountering issues searching for a specific WebElement even though the documentation indicates that it is feasible. while True: try: member = self.pdriver.find_all(" ...

pythonselenium

Is there a way for me to select an option from the dropdown list in this specific code?

Apologies for any confusion caused by the code I shared. My goal is to automate the dropdown feature, but I am only familiar with using the Select class and unsure of how to proceed. Please refer to the image linked below and provide guidance on selecting ...

seleniumwebdriver

The error message "ValueError: Precision cannot be used in integer format specifier" is thrown when

As a beginner in Python, I am currently exploring the format method. This information is from a book where I learning Python programming: The format method in Python involves substituting each argument value into the specified place. The specifications c ...

pythonstringpython-3.x

Gradually increase the time in a dataframe column by the initial value of the column

I am facing a situation where I need to increment the timestamp of a particular column in my dataframe. Within the dataframe, there is a column that contains a series of area IDs along with a "waterDuration" column. My goal is to progressively add this d ...

pythonpython-3.xpandasdataframe

There is an issue with the attribute of the object, as it does not contain an 'assert

I am encountering an issue where I receive the error message 'object has no attribute 'assertEqual' when attempting to execute the following code snippet: self.assertEqual("IRELAND INSTITUTE OF PITTSBURGH", driver.find_element_by_id("cname" ...

pythonselenium

Selenium Webdriver | Element with 'href' attribute cannot be found

I have been attempting to find a specific link using Selenium Webdriver with Xpath and CSS. <table width="100%" cellspacing="0" cellpadding="0" border="0"> <tbody> <tr> <tr> <td class="cspbItmA"> <a class="cspbItm" target= ...

seleniumxpath

Identify and enumerate the actionable components within a web page

I need assistance developing a Java function that can identify and return the count of interactive objects on a webpage that trigger an action when clicked, but excluding hyperlinks. Examples include buttons, image buttons, etc. Does anyone have any sugge ...

javaseleniumhtml-parsing

Instructions on how to display a list of distinctive icons featured in article headlines along with their respective usage frequency

Visit this website for the latest news. Count the number of articles displayed on the page and identify all unique icons used in the articles. ...

selenium-webdriver

What exactly does the Test Explorer initiate?

Currently, I am interested in performing tests on a web page front end that interacts with an ASP.NET RESTful Web API using Selenium. My understanding is that when I select "Run All" in the test explorer, it initiates my web server, opens my web page (re ...

c#visual-studioseleniumtestingasp.net-web-api

What is the reason for the num.is_integer() function returning false?

class Item: pay_rate = 0.8 # The pay after %20 discount all = [] def __init__(self, name: str, price: float, quantity=0): #Run validations to the recieved arguments assert price >= 0, f"Price {price} is not greater than ...

pythonpython-3.xfunctionoop

Find the button for uploading files using Selenium

I am encountering an issue with trying to click the button displayed below (image and HTML). Despite attempting to locate it using both xpath and ID, Selenium is still unable to find it. https://i.stack.imgur.com/KMMku.png <input id="wsUpload1" type= ...

pythonseleniumselenium-webdriver

Encountering a TimeoutException while trying to scrape a blank Webelement with Python Selenium

I am currently working on a web scraping project to extract the names of Pet shops along with their addresses by iterating through different country states and cities. My goal is to export this data into an Excel file. However, I encountered a TimeoutExcep ...

pythonseleniumloopstimeoutexception

Unveiling the hidden secrets of HTML code through scraping

Looking to extract data from the cadastral records in Poland available at . Specifically, I am interested in retrieving information from the element tagged with id 'gfi_0'. The challenge is that this element is not immediately accessible; it only ...

pythonhtmlseleniumweb-scrapinghtml-table

Struggling to manage the cookies section

I'm encountering an issue with the cookies page on a website when I try to launch it. Specifically, I need assistance in clicking the "Allow Cookies" button within that frame. Can someone please provide guidance? driver = new FirefoxDriver(); driver. ...

selenium-webdriver

Python code encounters a Selenium error while trying to click the video button

Struggling to automate the playback of my math online video through Selenium, but encountering a problem. Other elements on the page load before the video with the play button appears. I need a way to automatically trigger that button without any delays. ...

pythonselenium

Using Python with Selenium, automate the process of clicking on each list item within

Hey there! I recently implemented the code below, and it worked perfectly. However, I'm now trying to figure out how to make it click on all the li items within a ul that have an empty class attribute. Any ideas on how to do this using Xpath? Xpath: ...

htmlpython-2.7python-3.xselenium-webdriveriteration

What are the steps to configure a Jenkins CI server for executing automated BDD selenium tests with a remote webdriver?

Just diving into the world of Jenkins and currently attempting to configure a server to execute selenium tests from a GitHub repository. Something seems off in my setup, but despite multiple attempts, I can't seem to pinpoint the issue. I have set up ...

pythonseleniumgithubjenkinsremote-server