I've developed a selenium script using python 3. The script needs to run daily at 7AM for approximately 5 minutes. However, I don't want to keep my laptop running all the time just for this task. Can anyone suggest an alternative that is convenient and efficient?
I explored the option of using Lambda AWS, but encountered difficulties with installing chrome (or its headless version) on that platform. Additionally, I have limited experience working with servers.
AWS Lambda is a great option for achieving your objective, especially if the script will run for a short duration. The maximum execution time has been extended to 15 minutes, which should meet your needs.
Implementing headless Chrome in Lambda (specifically in Python; there are plenty of JS/node.js solutions available) is possible - I have personally successfully utilized this project in the past - https://github.com/21Buttons/pychromeless
If you want to schedule the execution at a specific time, you can utilize Amazon's CloudWatch.
If you want to run a Selenium script using Python 3 without having to install a browser, you can utilize the GhostDriver.
Ghost Driver works as an implementation of the Remote WebDriver protocol with PhantomJS at its core. Its main purpose is to be an essential component of PhantomJS itself, providing a JavaScript API.
phantomjs.page.settings.SETTING = VALUEphantomjs.page.customHeaders.HEADER = VALUEphantomjs.page.whitelistphantomjs.page.blacklistunhandledPromptBehaviorloggingPrefsphantomjs.binary.pathphantomjs.ghostdriver.pathphantomjs.cli.argsphantomjs.ghostdriver.cli.argsCode Snippet :
from selenium import webdriver
driver = webdriver.PhantomJS(executable_path='/path/to/phantomjs')
driver.get('https://www.google.com/')
print(driver.title)
driver.quit()
Output on Console:
Google
For my Selenium Java automation of a web application, there is an angularJS ng-repeat table with checkboxes in each row that I need to select based on certain conditions for testing. I have tried the following code snippet to locate and interact with thes ...
I'm currently tackling a web scraping project focusing on a popular ecommerce platform. I want to keep the browser hidden during this process, and usually, using the "--headless" option would do the trick. However, the specific page I'm trying to ...
After reading the post on changing IP addresses in Python using Selenium, I realized it didn't provide much help for my current situation. I am working on a test website and attempting to implement an anti-brute force script. My aim is to block an IP ...
I am currently utilizing phantomjs with mink: default: extensions: Behat\MinkExtension\Extension: goutte: ~ selenium2: browser: phantomjs wd_host: http://localhost:8643/wd/hub capabilities: we ...
Struggling to choose a radio button with Python Selenium here. I've exhausted all possible solutions found online, but none seem to work for the specific website I'm dealing with. Here's the complete code snippet: import time from selenium ...
Currently, I am working in a Python 2.7 environment on my Mac using the Terminal. I have taken all necessary steps to ensure that my geckodriver is readable, writable, and executable: -rwxrwxrwx@ 1 Esi staff 3600548 31 Oct 19:55 geckodriver The geckod ...
When using selenium to scrape data within a span element, my code retrieves the entire content of the span item, such as "280 [-49]". How can I clear the extra text inside the span and only get the value 280? warrant_bar = driver.find_element_by_xpath(&apo ...
In an unusual twist, my organization has blocked access to Github.com, leaving me unable to download the geckodriver.exe file. I've searched for alternative sources for downloading the file, but they all seem to lead back to Github. Does anyone have ...
I need help with extracting specific data from a datagrid using XPath in Selenium. After gathering all the rows into an IList<IWebElement> with this XPath: .//*[@class='datagrid']/tbody/tr, it seems to be working fine. Now, I want to extr ...
As I navigate through a webpage and attempt to submit a form, I encounter an issue. The submit button is located at the bottom of the form but is obstructed by a popunder advertisement. Despite trying to execute element.click(), an error is returned: Sy ...
I am experiencing a particular issue with the sendKeys() function. It seems to be malfunctioning only in one specific field within my form. Despite filling in the field, the value remains unchanged and the test passes without any error message. In a diffe ...
After installing Robot Framework and setting up some test cases, I utilized the keyword Maximize Browser Window to enlarge the browser. However, now I am in need of a way to minimize it. Despite checking the official documentation, I could not find a speci ...
I'm currently encountering some challenges when trying to scrape a website that utilizes react for certain parts of its content, and I'm unsure about the reason behind my inability to extract the data. Below is the HTML structure of the website: ...
Currently, I am utilizing selenium-4.1.0 and my quest is to find the most lightweight webdriver for optimal speed. I came across HtmlUnitDriver, however, when working with python, it is a necessity to initiate a selenium server before attempting to use t ...
I recently encountered a problem while working on my Python Selenium project. It seems that Python is having trouble recognizing the code snippet EC.presence_of_element_located. I am not sure why this error is occurring. Below is the segment of code where ...
Currently, I am attempting to utilize Python Selenium to choose specific files. dir = os.chdir("C:\\Users\\adam\\OneDrive - Wheelers Lane Technology College\\Pictures\\ama") for file in glob.glob( ...
Currently, I am utilizing Splinter and Python to automate a specific website. While I have successfully employed the send_keys() method in Selenium, I have encountered an issue when attempting to send multiple keys simultaneously. For instance, the code sn ...
The Webdriver protocol specification discusses the buttonDown action (emphasis added): Press and hold the left mouse button (at the coordinates specified by the last moveto command). It is important to note that the next mouse-related command should be ...
I've written a script (https://pastebin.com/dWLFvirn) that I'm trying to run through the terminal, but it's giving me an error. Here's a screenshot of the error (https://ibb.co/NSfLgL8). I would like to use "" as the link to make the s ...
I've encountered a challenge while attempting to scrape data from the website: . The issue lies in the fact that the page I'm trying to extract information from is dynamic, making it more difficult than usual. My goal with Selenium is to interact ...