My goal is to monitor a page for updates while maintaining the same session and cookies, without sending a completely new request.
How can I determine if there are HTML updates within my current request? The page will redirect but keep the same URL.
This is the code I am currently using:
import requests
url = 'xxx'
headers = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'
}
response = requests.get(url, headers=headers, allow_redirects=True, config={'keep_alive': True})
def get_status():
html = response.text # this should be the current HTML, not the HTML when I made the initial request
if x in html:
status = "exists"
else:
status = "null"
return status
print(get_status())
UPDATE: I plan to use a while loop to execute this function every 5 seconds to check if the status equals "exists".
UPDATE2: Although I attempted to implement this using requests_html, I am not receiving all the necessary cookies:
import requests_html
from requests_html import HTMLSession
session = HTMLSession()
session.headers.update({'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36'})
r = session.get('x')
r.html.render(reload=False)
print(r.cookies.get_dict())