Extract initial odds data from Oddsportal webpage

I am currently attempting to scrape the initial odds from a specific page on Oddsportal website: https://www.oddsportal.com/soccer/brazil/serie-b/gremio-cruzeiro-WhMMpc7f/

My approach involves utilizing the following code:

from selenium.webdriver.common.by import By  
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
import pandas as pd
from selenium.webdriver import ActionChains
import warnings
warnings.filterwarnings("ignore")
from time import sleep
from tqdm import tqdm_notebook

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
wd = webdriver.Chrome('chromedriver',chrome_options=chrome_options)
driver =webdriver.Chrome('chromedriver',chrome_options=chrome_options)

driver.get('https://www.oddsportal.com/soccer/brazil/serie-b/gremio-cruzeiro-WhMMpc7f/')
sleep(2)

element=driver.find_element(By.ID,'odds-data-table').find_element(By.CLASS_NAME,'table-container').find_element(By.XPATH,"//table[@class='table-main detail-odds sortable']").find_element(By.XPATH,"//tbody/tr[1]/td[2]')

hover=ActionChains(driver).move_to_element(element)
hover.perform()
sleep(2)
driver.find_element(By.ID,'tooltipdiv')

Despite successfully creating the ActionChains object, the driver is encountering difficulty in locating the element 'tooltipdiv' post-action.

For added insight, the 'tooltipdiv' element only becomes visible on the page when the cursor interacts with the element in the ActionChains object.

Click here for a screenshot reference

Attempts using 'click_and_hold', 'click' methods have been made, however, none have proven effective. I'm uncertain about what altered, but this identical script functioned correctly several months ago.

What steps can I take to address this issue?

Answer №1

By making some fine tuning adjustments, using explicit wait with proper xpath and expected conditions will lead to a smooth functioning. Here are the 3-4 problems encountered:

  1. The cookies policy popup needs to be handled.
  2. The find_element(*) function chain was breaking due to an extra space in the class name as 'table-container'
  3. Instead of using ActionChains(), simply using click() worked for me.
  4. After clicking, waiting for the tooltip element to be clickable is necessary since it alters the DOM based on mouse actions

Below is the updated code after fixing the issues -

Updated selenium script...

Output:

List of outputs...

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Using Selenium in Python to extract distinct list

I'm currently working on a script to scrape hotel.com using selenium import time import os from selenium import webdriver from selenium.webdriver.chrome.service import Service from webdriver_manager.chrome import ChromeDriverManager from selenium.webd ...

Python error: No element found using driver.find_element_by_class_name method - Element not found in DOM

As of now, I am engaged in a project with Selenium where I am trying to accurately identify elements using the 'Inspect' feature on Chrome. There are instances where some of my buttons to click are only defined by class, but certain classes fail ...

The issue persists with FastAPI CORS when trying to use wildcard in allow origins

A simplified representation of my code utilizing two different approaches. from fastapi import FastAPI middleware = [ Middleware( CORSMiddleware, allow_origins=["*"], allow_credentials=False, allow_methods=["*"], ...

Python regular expressions for identifying multiple instances of double characters within a word

Seeking assistance in identifying words with 2 sets of double letters within a dictionary structure. Although I am new to Python / regex, I have attempted to create code based on similar questions from various sources. However, my code does not produce th ...

Using C# Selenium WebDriver to efficiently locate hyperlinks based on several specific strings, all while incorporating a reference within the string

I am trying to click a link using two different strings. However, the code I have tried does not seem to be working. Can anyone provide any guidance? var xPathString = String.Format("//a[contains(text(), 'Enforcement') and contains(text(), ...

Error encountered in the main thread: Could not find class definition for org/openqa/selenium/remote/codec/w3c/W3CHttpCommandCodec

An issue occurred in the main thread with the message: java.lang.NoClassDefFoundError: org/openqa/selenium/remote/codec/w3c/W3CHttpCommandCodec Could this error be related to dependencies? I tried searching for solutions on Stack Overflow but couldn' ...

Can you find a method to tally characters on a webpage viewed in a browser app using R?

I have a collection of text files (.txt) that contain saved web pages, specifically public profile pages from a social media platform. I am interested in measuring the amount of content on these profiles, but simply converting the files to .html and viewin ...

Is it possible to send an AJAX request to a Django view that might result in a redirect?

Query I encountered an issue while attempting to access a specific Django view through AJAX. This particular view redirects users if they haven't authorized the site with Google. I suspect the problem arises from redirecting "within" a view requested ...

Python: The specified index is not within the range of the list

Here's the code I'm working with: def my_sort(list): for _ in list: if list[0] > list[1]: list[0], list[1] = list[1], list[0] return my_sort(list[1:2]) Unfortunately, I keep encountering this error message: IndexEr ...

What is the process for importing a .sql file into Selenium in order to run various scripts simultaneously in the Selenium environment?

Is there a way to run multiple commands in selenium, such as inserting, updating, and selecting * from employee? How can I accomplish this task? ...

Tips for effectively testing GWT and SmartGWT together in unit tests?

Currently, I am conducting unit testing on the client side of a GWT+SmartGWT application. Initially, I utilized GwtTestCase for testing, but found it to be too time-consuming for such a large application. Even when using GwtTestSuite, the execution time wa ...

Error encountered: Element cannot be clicked on at specified coordinates - Angular/Protractor

Recently, I have been testing out CRUD functionality in an Angular app using Protractor. One recurring issue I've encountered is with the create/edit buttons, which all open the same modal regardless of the page you're on. The frustrating part i ...

Leverage environment variables within SFTP paths when using Paramiko

Currently, I am executing a script on my server and attempting to retrieve some files from my Firewall. However, when I use an environment variable (signified by the $ symbol) to reference the file, I encounter a "file not found" error. Surprisingly, I hav ...

Python tkinter infinite loop causing program to become unresponsive

Currently, I am developing a tkinter application that automatically backs up files by monitoring the last modification time of the file. The code for this app is provided below. from tkinter import * from tkinter import filedialog import shutil import os f ...

Having trouble displaying text within a span element after using execute_script with Selenium?

After numerous attempts to insert text into an input box that functions as a span element, I decided to seek guidance on the proper approach. In my quest, I turned to utilizing the execute_script() method, but disappointingly, no progress was made. Below ...

In Python, the shutil.move function is designed to move directories along with their

So I was coding in Python and encountered a problem that I can't seem to resolve. Here's the code snippet: import shutil import pathlib import os source_folder =(r'C:\Users\Acer\Desktop\New') destination_folder =(r ...

Tackling the task of identifying elements using @FindBy when they possess multiple id values

Is it possible to declare page elements (mobile elements, using @FindBy or @AndroidFindBy) with two potential ids that will vary based on the app version being tested? One id is for the staging version and another for production, each slightly different - ...

Prevent the opening of tabs in selenium using Node.js

Currently, I am using the selenium webdriver for node.js and loading an extension. The loading of the extension goes smoothly; however, when I run my project, it directs to the desired page but immediately the extension opens a new tab with a message (Than ...

Finish the n-Digit Multiplicand

I've been working on a problem, but the time complexity of my code is very high. Is there a way to reduce it? Below is the question along with my current code implementation. A number is considered a complete 'n' digit factor if it can divid ...

Tips for converting the 'numericals' in the input provided by the user into the initial point of a loop

I am in the process of developing a program to analyze the game Baccarat, and while I have a good grasp of the basics, I require assistance in enabling users to paste multiple games at once. Below is an example: games = input('Enter the games you wis ...