Recently starting to use Python, I am currently using Python's seleniumbase library to scrape a website and need to periodically run this fetch script. While experimenting, I encountered a script timeout error when the response time exceeded around 950 milliseconds. However, I could still see the fetch response in the browser's network tab. The code runs smoothly if the response delay is under 950 ms.
script = """
return new Promise((resolve, reject) => {
var csrf = document.querySelector('meta[name="csrf-token"]').content;
var obj;
fetch("MyUrl", {
'headers': {
'accept': '/',
'accept-language': 'en-US,en;q=0.9',
'content-type': 'application/x-www-form-urlencoded; charset=UTF-8',
'sec-ch-ua': '"Chromium";v="116", "Not)A;Brand";v="24", "Google Chrome";v="116"',
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"Windows"',
'sec-fetch-dest': 'empty',
'sec-fetch-mode': 'cors',
'sec-fetch-site': 'same-origin',
'x-csrf-token': csrf,
'x-requested-with': 'XMLHttpRequest'
},
'referrer': 'MyUrl',
'referrerPolicy': 'strict-origin-when-cross-origin',
'body': 'consularid=3&exitid=1&servicetypeid=1&calendarType=2&totalperson=1',
'method': 'POST',
'mode': 'cors',
'credentials': 'include'
})
.then(res => res.text())
.then(data => {
obj = data;
resolve(obj);
})
.catch(error => {
reject(error);
});
});
"""
obj_variable = sb.execute_script(script)
An example of the error displayed is as follows:
Message: script timeout
(Session info: chrome=116.0.5845.141)
Stacktrace:
GetHandleVerifier [0x00007FF61E0352A2+57122]
(No symbol) [0x00007FF61DFAEA92]
(No symbol) [0x00007FF61DE7E25D]
(No symbol) [0x00007FF61DEEF314]
(No symbol) [0x00007FF61DED6FDA]
(No symbol) [0x00007FF61DEEEB82]
(No symbol) [0x00007FF61DED6DB3]
(No symbol) [0x00007FF61DEAD2B1]
(No symbol) [0x00007FF61DEAE494]
GetHandleVerifier [0x00007FF61E2DEF82+2849794]
GetHandleVerifier [0x00007FF61E331D24+3189156]
GetHandleVerifier [0x00007FF61E32ACAF+3160367]
GetHandleVerifier [0x00007FF61E0C6D06+653702]
(No symbol) [0x00007FF61DFBA208]
(No symbol) [0x00007FF61DFB62C4]
(No symbol) [0x00007FF61DFB63F6]
(No symbol) [0x00007FF61DFA67A3]
BaseThreadInitThunk [0x00007FFAFEF07614+20]
RtlUserThreadStart [0x00007FFB007C26B1+33]
I attempted to add a timeout to the script code without success; it is possible that my implementation was incorrect.