Code Proposal:
To simplify the process of gathering links to all games on a given day from the page , I am looking to create a script that allows me to dynamically change the date, such as 2021/08/01 or any other date. This way, I can easily loop through and gather information from multiple days with a single code execution.
Despite being a slow approach without using Headless, the current model successfully clicks all buttons, expands the data, and imports all 465 listed match links:
for btn in driver.find_elements_by_xpath("//tr[contains(@class,'group-head clickable')]"):
btn.click()
Full Code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time
options = Options()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-logging"])
driver = webdriver.Chrome(r"C:\Users\Computador\Desktop\Python\chromedriver.exe", options=options)
url = "https://int.soccerway.com/matches/2021/07/28/"
driver.get(url)
driver.find_element_by_xpath("//div[@class='language-picker-trigger']").click()
driver.find_element_by_xpath("//a[@href='https://int.soccerway.com']").click()
time.sleep(10)
for btn in driver.find_elements_by_xpath("//tr[contains(@class,'group-head clickable')]"):
btn.click()
time.sleep(10)
jogos = driver.find_elements_by_xpath("//td[contains(@class,'score-time')]//a")
for jogo in jogos:
resultado = jogo.get_attribute("href")
print(resultado)
driver.quit()
However, when adding
options.add_argument("headless")Message: element click intercepted
In order to address this issue, further analysis led to the use of WebDriverWait (). An attempt was made to resolve it by applying the following syntax:
for btn in WebDriverWait(driver, 1).until(EC.element_to_be_clickable((By.XPATH, "//tr[contains(@class,'group-head clickable')]"))):
btn.click()
Full Code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")
options.add_argument("headless")
options.add_experimental_option("excludeSwitches", ["enable-logging"])
driver = webdriver.Chrome(r"C:\Users\Computador\Desktop\Python\chromedriver.exe", options=options)
url = "https://int.soccerway.com/matches/2021/07/28/"
driver.get(url)
driver.find_element_by_xpath("//div[@class='language-picker-trigger']").click()
driver.find_element_by_xpath("//a[@href='https://int.soccerway.com']").click()
time.sleep(10)
for btn in WebDriverWait(driver, 1).until(EC.element_to_be_clickable((By.XPATH, "//tr[contains(@class,'group-head clickable')]"))):
btn.click()
time.sleep(10)
jogos = driver.find_elements_by_xpath("//td[contains(@class,'score-time')]//a")
for jogo in jogos:
resultado = jogo.get_attribute("href")
print(resultado)
driver.quit()
Unfortunately, this implementation returns an error due to 'NoneType' object not being iterable.
'NoneType' object is not iterable
Reason for the Option:
1 - The need for running automation in an online terminal without displaying the browser window on screen, optimizing processing speed within terminal constraints.
2 - Flexibility to input different dates instead of hardcoded values like 2021/07/28 in the URL:
url = "https://int.soccerway.com/matches/2021/07/28/"
Future integration of a variable for date selection:
today = date.today().strftime("%Y/%m/%d")
In search of faster alternatives, referenced methods may present improved efficiency while avoiding the requirement of WebDriver. However, challenges persist when attempting to scale beyond the default date provided.
The anticipated outcome (with limited representation due to character constraints) includes 465 game links:
https://int.soccerway.com/matches/2021/07/28/europe/uefa-champions-league/fc-sheriff-tiraspol/alashkert-fc/3517568/
https://int.soccerway.com/matches/2021/07/28/europe/uefa-champions-league/fk-neftchi/olympiakos-cfp/3517569/
...
https://int.soccerway.com/matches/2021/07/28/africa/cecafa-senior-challenge-cup/uganda-under-23/eritrea-under-23/3567664/
Note 1: Various types of score-time classes exist, including score-time status and score-time score, thus necessitating the usage of contains in
"//td[contains(@class,'score-time')]//a"Update
In addition to addressing the current challenge, exploring enhanced and swifter options for improving the existing method would be greatly valued as part of my ongoing learning process. (Continuously seeking advancements beyond dated methodologies).