How can I determine the specific quantity of XPATH links with unique identifiers in Selenium?

Question

How can I determine the specific quantity of XPATH links with unique identifiers in Selenium?

Seeking automation with Python3 and selenium to streamline searches on a public information site. The process involves entering a person's name, selecting the desired spelling (with or without accents), navigating through a list of lawsuits, and accessing individual case pages.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException, NoSuchElementException
from selenium.webdriver.common.keys import Keys
import time
import re

Specify the search name

name = 'JOSE ROBERTO ARRUDA'

Set up paths, starting link for search, and an empty list to store data

firefoxPath="/home/abraji/Documentos/Code/geckodriver"
link = 'https://ww2.stj.jus.br/processo/pesquisa/?aplicacao=processos.ea'
processos = []

Initialize driver and navigate to the initial search page

driver = webdriver.Firefox(executable_path=firefoxPath)
driver.get(link)

Input text and initiate the search

WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#idParteNome'))).click()
time.sleep(1)
driver.find_element_by_xpath('//*[@id="idParteNome"]').send_keys(name)
time.sleep(6)
WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#idBotaoPesquisarFormularioExtendido'))).click()

Select all possible spellings for the search

WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#idBotaoMarcarTodos'))).click()
WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#idBotaoPesquisarMarcados'))).click()
time.sleep(1)

... ... ...

python html selenium

Answer 1

Answer №1

Here is the code snippet for the "Capture routine":

wait = WebDriverWait(driver, 20)

#...    

while True:
    links = wait.until(EC.presence_of_all_elements_located((By.XPATH, "//span[contains(@class,'classSpanNumeroRegistro')]")))
    print("Number of links:", len(links))
    for i in range(1, len(links) + 1):
        # Access the direct link and click
        .until(EC.element_to_be_clickable((By.XPATH, f"(//span[contains(@class,'classSpanNumeroRegistro')])[{i}]//a"))).click()

        # Run tests to gather data
        try:
            unique_num = driver.find_element_by_xpath('//*[@id="idProcessoDetalhesBloco1"]/div[6]/span[2]/a').text
        except NoSuchElementException:
            unique_num = "no_unique_number_found"

        try:
            proc_name = driver.find_element_by_xpath('//*[@id="idSpanClasseDescricao"]').text
        except NoSuchElementException:
            proc_name = "name_not_found"

        try:
            aut_date = driver.find_element_by_xpath('//*[@id="idProcessoDetalhesBloco1"]/div[5]/span[2]').text
        except NoSuchElementException:
            aut_date = "date_not_found"

        # Fill dictionary and list
        dictionary = {"unique_num": unique_num,
                      "proc_name": proc_name,
                      "aut_date": aut_date
                      }
        processes.append(dictionary)

        # Go back a page to click on next process
        driver.execute_script("window.history.go(-1)")

    # wait.until(EC.presence_of_element_located((By.CLASS_NAME, "classSpanPaginacaoImagensDireita")))
    next_page = driver.find_elements_by_css_selector(".classSpanPaginacaoProximaPagina")
    if len(next_page) == 0:
        break
    next_page[0].click()

Answer 2

Here is the code snippet for the "Capture routine":

wait = WebDriverWait(driver, 20)

#...    

while True:
    links = wait.until(EC.presence_of_all_elements_located((By.XPATH, "//span[contains(@class,'classSpanNumeroRegistro')]")))
    print("Number of links:", len(links))
    for i in range(1, len(links) + 1):
        # Access the direct link and click
        .until(EC.element_to_be_clickable((By.XPATH, f"(//span[contains(@class,'classSpanNumeroRegistro')])[{i}]//a"))).click()

        # Run tests to gather data
        try:
            unique_num = driver.find_element_by_xpath('//*[@id="idProcessoDetalhesBloco1"]/div[6]/span[2]/a').text
        except NoSuchElementException:
            unique_num = "no_unique_number_found"

        try:
            proc_name = driver.find_element_by_xpath('//*[@id="idSpanClasseDescricao"]').text
        except NoSuchElementException:
            proc_name = "name_not_found"

        try:
            aut_date = driver.find_element_by_xpath('//*[@id="idProcessoDetalhesBloco1"]/div[5]/span[2]').text
        except NoSuchElementException:
            aut_date = "date_not_found"

        # Fill dictionary and list
        dictionary = {"unique_num": unique_num,
                      "proc_name": proc_name,
                      "aut_date": aut_date
                      }
        processes.append(dictionary)

        # Go back a page to click on next process
        driver.execute_script("window.history.go(-1)")

    # wait.until(EC.presence_of_element_located((By.CLASS_NAME, "classSpanPaginacaoImagensDireita")))
    next_page = driver.find_elements_by_css_selector(".classSpanPaginacaoProximaPagina")
    if len(next_page) == 0:
        break
    next_page[0].click()

Answer 3

Answer №2

If you want to continue running the loop until the next button appears on the screen, you can use this logic:

try:
    next_button = driver.find_element_by_class_name('classSpanPaginacaoProximaPagina')
    if(next_button.is_displayed()):
        next_button.click()

except NoSuchElementException:
     print('Next page button not found')

Answer 4

If you want to continue running the loop until the next button appears on the screen, you can use this logic:

try:
    next_button = driver.find_element_by_class_name('classSpanPaginacaoProximaPagina')
    if(next_button.is_displayed()):
        next_button.click()

except NoSuchElementException:
     print('Next page button not found')

How can I determine the specific quantity of XPATH links with unique identifiers in Selenium?

Answer №1

Answer №2

Similar questions

Python is the way to go for clicking on a link

What is the best way to serialize data when a writer subclass of iobase.write is responsible for writing records to a local server, and the writer process is distributed

What could be causing my div elements to not appear on the page?

What is the best way to dynamically change the JSON-LD Script for the Schema?

Setting a fixed data value within a div for subsequent retrieval through a function

The asyncio add_signal_handler function is failing to capture the sigint and sigterm signals

Issue with requests due to MissingSchema exception

What is the reason behind the change in style hierarchy when using ':last-child'?

What is the best way to eliminate the gap between header, content, and footer sections when in mobile view?

When trying to load the JSON data saved in a JSON file, I encounter an error: JSONDecodeError: Expecting value at line 1, column 1

Keeping information saved locally until an internet connection is established

Searching for a hyperlink WebElement with Java and Selenium: Finding the way to locate it

Is there a way to extract the unicode/hex representation of a symbol from HTML using JavaScript or jQuery?

How to override an event in JavaScript when a specific value is entered

Handling alert, confirm, and popup events in C# using Selenium

Tensorflow encountered an error in Python indicating that input_1:0 is being both fed and fetched, resulting in an InvalidArgumentError

Combining Python, Neo4j, and JSON to uniquely create a new NODE for each user

The Selenium Internet Explorer driver is having difficulty locating the accurate webpage source

Troubleshooting issue with setting browser version using Selenium Add Additional Capability in .NET Framework not resolving

The enchanting dance of words ceased as Poetry ran off with Worker.py, only to realize that the file b'/snap/bin/worker.py' was nowhere to be found in the directory