I've been attempting to extract the ticker symbol, stock name, price, sector, and market cap columns from this website. I'm facing challenges in identifying the correct HTML elements using the appropriate code. Although I've used Selector Gadget to locate the XPaths, my confidence in understanding HTML trees and rules isn't very high. Notably, the first three columns appear to be grouped within a single td on the webpage. Below is the current code snippet which pulls entire rows. Any assistance would be greatly appreciated.
from selenium import webdriver
from selenium.webdriver.common.by import By
import re
import pandas as pd
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
import time
from selenium.common.exceptions import NoSuchElementException
driver = webdriver.Chrome()
website = 'https://www.tradingview.com/markets/stocks-turkey/market-movers-all-stocks/'
driver.get(website)
while True:
try:
loadMoreButton = driver.find_element(By.XPATH,'//*[contains(concat( " ", @class, " " ), concat( " ", "content-D4RPB3ZC", " " ))]')
time.sleep(2)
loadMoreButton.click()
time.sleep(5)
except Exception as e:
print (e)
break
print ("Complete")
time.sleep(10)
matches = driver.find_elements(By.TAG_NAME,'tr')
ticker_symbol = []
ticker_name = []
ticker_price =[]
ticker_sector =[]
ticker_marketcap =[]
for match in matches:
print(match.text)
driver.quit()