In my Python Google Colab project, I am attempting to access a CSV file from the following link:
After scrolling down slightly on the page, there is a download button visible. My goal is to extract the link using Selenium or BeautifulSoup in order to read the CSV file. The code snippet I am working with looks like this:
# Installing necessary packages
!pip install selenium
!apt-get update # Update Ubuntu for proper apt installation
!apt install chromium-chromedriver
!cp /usr/lib/chromium-browser/chromedriver /usr/bin
# Import required libraries
import pandas as pd
from selenium import webdriver
import sys
# Using Selenium to fetch and read the CSV file
sys.path.insert(0,'/usr/lib/chromium-browser/chromedriver')
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome('chromedriver',chrome_options=chrome_options)
driver.get('https://www.macrotrends.net/stocks/charts/AAPL/apple/stock-price-history')# Enter the URL of the desired page here
btn = driver.find_element_by_tag_name('button')
btn.click()
df = pd.read_csv('##.csv')
Everything seems to be functioning properly up to the btn.click() step, but I encounter an error afterwards because I'm not able to locate the download button's link or the file name. Can anyone provide guidance on how to do this successfully? Any help would be greatly appreciated.