Utilizing Python with Selenium for Web Scraping

https://i.stack.imgur.com/klhCL.png I need to retrieve numbers like 14,401. I have attempted the following code:

WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[@id='wiz-iframe-intent']")))
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="highcharts-5tawcvw-0"]/svg/g[8]/g[1]/text/tspan[1]'))).text

This is resulting in a TimeOut Exception.

Answer №1

Here is the code snippet you can use to interact with an element in SVG using Selenium:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC 
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.select import Select
import time

driver = webdriver.Chrome(executable_path = r'D:/Automation/chromedriver.exe')
driver.maximize_window()
driver.get("https://eu1.dashboard.clevertap.com/login.html")

wait = WebDriverWait(driver, 20)
action = ActionChains(driver)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.ID,"wiz-iframe-intent")))
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[@class='CT_Interstitial']//span[@class='CT_InterstitialClose']"))).click()

driver.switch_to.default_content()

wait.until(EC.element_to_be_clickable((By.NAME, "email"))).send_keys("username")

wait.until(EC.element_to_be_clickable((By.NAME,"password"))).send_keys("****")  

wait.until(EC.element_to_be_clickable((By.ID,"submitBtn"))).click()

wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, 'div.ct-breadcrumb')))

try:
    wait.until(EC.frame_to_be_available_and_switch_to_it((By.ID,"wiz-iframe-intent")))
    wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.close-btn.js-close-popup>button"))).click()
except:
    pass

driver.switch_to.default_content()  
action.move_to_element(driver.find_element_by_css_selector("div.sidebar__brand+ul>li:first-child>a")).perform()

wait.until(EC.visibility_of_element_located((By.LINK_TEXT, "All Boards")))  

wait.until(EC.element_to_be_clickable((By.LINK_TEXT,"All Boards"))).click()

wait.until(EC.element_to_be_clickable((By.LINK_TEXT,"pe-funnel"))).click()

wait.until(EC.visibility_of_element_located((By.XPATH, "//*[name()='tspan' and contains(@stroke-linejoin,'round')]"))) 

all_data = driver.find_elements_by_xpath("//*[name()='tspan' and contains(@stroke-linejoin,'round')]")

print(len(all_data))

for data in all_data:
 print(data.text)

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Using lxml in Python: Extracting text displays only English characters while others appear scrambled

Below is the code snippet I'm working with: import requests from lxml.etree import HTML title_req = requests.get("https://www.youtube.com/watch?v=VK3QWm7jvZs") title_main = HTML(title_req.content) title = title_main.xpath("//span[@id='eow-title& ...

Learn how to streamline the task of verifying Twitter profiles by creating an automated system that can also send messages if desired

Is there a way to use Selenium with Python and Requests to check if a specific profile has the message function? https://i.stack.imgur.com/0Q4RB.png ...

Utilizing selenium to input text values in robot framework

Using Robot Framework with Selenium 2.0, I have created a test that fills out a form and saves it. However, the issue I am facing is that when entering numeric values into text boxes, the default value of 0.0 is not being cleared. This results in the new v ...

Tips for disguising Python Selenium to access doordash accounts

Attempting to utilize Python Selenium for logging into doordash. Here's the code snippet: user_agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0.3 Safari/605.1.15" proxy = '5 ...

Choose a specific ID to view all potential links

The data table below displays identifiers (customer IDs) in columns ColumnA and TempB: ColumnA TempB value 0 2149712 1431291 7.7 1 2149712 1421222 6.3 4 2149712 5212 ...

Comparing WinAppDriver and LeanFT

Currently, I am utilizing LeanFT for automating a windows-based application and considering using WinAppDriver to avoid the licensing costs associated with the former tool. Are there any potential drawbacks to switching to WinAppDriver at a high level? M ...

Require remote access to a Firefox browser on a connected machine through an open connection in a robot test case

Issue at hand: Attempting to run a Robothost Testcase which involves using the Open Connection keyword to connect to machineX and then utilizing the Open Browser keyword to open Firefox on machineX. However, instead of opening the Firefox browser on machin ...

The error that is being encountered is: 'float' data type cannot be read as an integer in Python version 3.4

I am encountering an issue while attempting to play a video file and the error message reads as follows: $ /usr/bin/python3.4 /home/ramakrishna/PycharmProjects/Lanedect/driving-lane-departure-warning-master/main.py Traceback (most recent call last): ...

Python allows for the color coding of numbers using a method that is comparable to the conditional

Looking for a Python module or workaround to color code numbers from 0 to 100, transitioning from red to green. I want to visually represent a score by changing the font color based on the number without starting from scratch. Considering creating a dictio ...

Tips for implementing Bitwise Exclusive OR on a nested list:

Consider having a list of lists like [[1,2,3],[1,2,3,4]] How would one go about finding the xor of all the elements in the list of lists? For example, the xor of [1^2^3] is 0, and the xor of [1^2^3^4] is 4. Therefore, the resulting list will be [0,4] Th ...

Creating a Dockerfile for a Ruby Capybara scraper: Step-by-step guide

I'm in the process of creating a Dockerfile to run a Ruby Capybara scraper within a docker container. The code I've tested works fine on my host OS, but it's throwing an error when executed within the docker container. Dockerfile FROM ruby: ...

Is it necessary to create a setter method for testing purposes in order to access a class variable?

In my code, I have a method called request_ticker_data which uses an input ticker string to retrieve data from Yahoo Finance. If the data cannot be accessed, it adds the ticker to a list named self.invalid. Additionally, there is a method called reset_inva ...

Obtain some content using Java along with Selenium WebDriver

I only want to extract the text "Invitation sent to xxxxx". I do not need the content inside the button tag. <button class="action" data-ember-action="" data-ember-action-3995="3995"> Visualizar profile </button> This is how I am c ...

Is the Selenium WebDriver disregarding the setTimeout command?

Has anyone experienced the issue where the selenium.setTimeout() command is being ignored when using WebDriverBackedSelenium? Any insights or solutions? ...

WebdriverIO - string manipulation issue removing characters

I've encountered a strange issue while using the WebdriverIO functions setValue and addValue. It seems that whenever my string contains the number 3, it is mysteriously removed and not entered into the input boxes. I can manually type 3 into these inp ...

Verify if the condition is met and generate a DataFrame containing the results

I'm trying to check if the value in a column is greater than 1 and then create a new column that displays 1 or 0 based on whether the condition is true or not. However, I am encountering a value error and I don't understand why. x = df[df['C ...

Personalized Jasmine matchers for a Protractor/WebDriverJS component

Greetings fellow developers, I'm looking to create a custom matcher for Protractor/WebDriverJS element. Can anyone assist in improving my current code? Here is what I aim to include in the specs files: var button = element(by.tagName('button&a ...

Tips for waiting for a page to load in Selenium using Java

I am currently seeking a method to delay page execution until all elements have been fully updated. An example scenario would be waiting for a screen value to change after clicking a button, thus ensuring that the new value has been displayed. My objectiv ...

Is there a way to execute a TestNG/Selenium test suite concurrently in Internet Explorer across various servers?

Currently, I am working with a Selenium test suite that utilizes the IEDriver and is executed using TestNG. The main objective is to use this test suite for regression testing. My aim is to run this test suite against all of our production servers. Present ...

Guide on building a FastAPI endpoint that can handle both Form and JSON body in one request

Is it possible to design an endpoint in FastAPI that can handle both (multipart) Form data and JSON body? How can I make the endpoint recognize the type of data it is receiving? ...