Scenario:
The code snippet below utilizes Selenium to locate a series of links from the Simply Recipe Index URL, then saves them in a list named linklist. The script proceeds to loop through each link in the linklist, fetching recipe text and storing it in a collection called recipe_list
from bs4 import BeautifulSoup
import requests
from splinter import Browser
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import selenium
import time
import csv
#set up chromedriver for WINDOWS
driver=webdriver.Chrome('chromedriver.exe')
url = "https://www.simplyrecipes.com/index/"
driver.get(url)
response=requests.get(url)
soup=BeautifulSoup(response.text,'html.parser')
#set up chromedriver for MAC
driver=webdriver.Chrome("/Users/williamforsyth/Documents/uc_davis/Homework_Repos/group-project-2/Kathryn/chromedriver")
url = "https://www.simplyrecipes.com/index/"
driver.get(url)
response=requests.get(url)
soup=BeautifulSoup(response.text,'html.parser')
linklist=[]
links=soup.find_all('a')
for link in links:
linklist.append(link)
linklist_text=[]
for i in range(164,1068):
linklist_text.append(linklist[i].text)
recipe_list=[]
for link in linklist_text:
time.sleep(0.3)
target=driver.find_element_by_partial_link_text(link)
target.click()
time.sleep(0.1)
cards = driver.find_elements_by_class_name("grd-title-link")
for i in range(0,len(cards)):
try:
newcards = driver.find_elements_by_class_name("grd-title-link")
time.sleep(0.3)
newcards[i].click()
time.sleep(0.3)
recipe=driver.find_element_by_id("sr-recipe-callout")
recipe_list.append(recipe.text)
driver.back()
time.sleep(0.3)
except:
continue
driver.get(url)
Dilemma:
I now wish to include a functionality that appends every instance of recipe.text from the loops to a CSV file. Is there an easy way to achieve this without having to overhaul the entire code?
The intended code snippet to append to a CSV alongside the current action of appending to recipe_list:
recipe_list.append(recipe.text)