The PyQt progress soars to 100% as soon as it begins

Running the code in the doWork method and clicking on button1 makes the progress bar function correctly.

However, if I pass the list to the doWork method from other methods like btn2 or btn3, the progress bar suddenly jumps to 100% as soon as it starts.

from PyQt5 import QtCore, QtGui, QtWidgets
from PyQt5.QtWidgets import *
from PyQt5.QtCore import *
from PyQt5.QtGui import *
import sys
from selenium import webdriver

class SeleniumWorker(QtCore.QObject):
    progressChanged = QtCore.pyqtSignal(int)
    def doWork(self, lst=['http://www.somesite.com/',
        'http://www.somesite.com/page2',
        'http://www.somesite.com/page3']):
        progress = 0
        browser = webdriver.Firefox()
        links = lst
        for link in links:
            browser.get(link)
            progress += 100 / len(links)
            self.progressChanged.emit(progress)
        browser.close()

class Widget(QtWidgets.QWidget):
    def __init__(self, *args, **kwargs):
        QtWidgets.QWidget.__init__(self, *args, **kwargs)
        lay = QtWidgets.QHBoxLayout(self)
        progressBar = QtWidgets.QProgressBar()
        progressBar.setRange(0, 100)
        button1 = QtWidgets.QPushButton("Start1")
        button2 = QtWidgets.QPushButton("Start2")
        button3 = QtWidgets.QPushButton("Start3")
        lay.addWidget(progressBar)
        lay.addWidget(button1)
        lay.addWidget(button2)
        lay.addWidget(button3)
        self.thread = QtCore.QThread()
        self.worker = SeleniumWorker()
        self.worker.moveToThread(self.thread)
        self.thread.started.connect(self.worker.doWork)
        button1.clicked.connect(self.thread.start)
        button2.clicked.connect(self.btn2)
        button3.clicked.connect(self.btn3)
        self.worker.progressChanged.connect(progressBar.setValue)


    def btn2(self):
        self.lst2 = ['http://www.somesite.com/page4',
        'http://www.somesite.com/page5',
        'http://www.somesite.com/page6']
        self.worker.doWork(self.lst2)

    def btn3(self):
        self.lst3 = ['http://www.somesite.com/page7',
        'http://www.somesite.com/page8',
        'http://www.somesite.com/page9']
        self.worker.doWork(self.lst3)


if __name__ == '__main__':
    app = QtWidgets.QApplication(sys.argv)
    w = Widget()
    w.show()
    sys.exit(app.exec_())

Answer №1

It appears that the concept of my previous solution is not clear to you, so let me provide a detailed explanation:

[1] self.thread = QtCore.QThread()
[2] self.worker = SeleniumWorker()
[3] self.worker.moveToThread(self.thread)
[4] self.worker.progressChanged.connect(progressBar.setValue, QtCore.Qt.QueuedConnection)
[5] self.thread.started.connect(self.worker.doWork)
  1. QThread represents a thread handler which allows tasks to be executed on a separate thread from the main GUI thread.

  2. You are instantiating a SeleniumWorker object with the doWork method, intended for tasks that should not run on the main GUI thread, hence the use of QThread.

  3. For the method to be executed in a different thread, the object containing it must be moved to that particular thread.

  4. The signal from the object in the separate thread is connected to the QProgressBar in the GUI thread using QtCore.Qt.QueuedConnection.

  5. Upon starting the thread, the doWork function will be invoked, and since the self.worker object resides in another thread, the function will also run in that designated thread.


In your code snippet later on, you invoke the doWork function before the thread has started, causing it to execute in the main thread.

def btn2(self):
    ...
    # main thread
    self.worker.doWork(self.lst2)

One approach to passing URLs is by utilizing a setter method and then commencing the thread. Upon initiation of the thread, doWork is called leading to emission of the progressChanged signal.

Summarizing the above discussion yields:

(Code snippet provided below)
(Additional functionality: Buttons are disabled while the thread is active to prevent re-triggering.)

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

An issue with Selenium web scraping arises: WebDriverException occurs after the initial iteration of the loop

Currently, I am executing a Selenium web-scraping loop on Chrome using MacOS arm64. The goal is to iterate through a list of keywords as inputs in an input box, search for each one, and retrieve the text of an attribute from the output. A few months ago, I ...

Python does not recognize the Json-like response

After receiving a response using urllib3 (also tested with requests), I decoded it and then used json.loads on it. The data, post-decoding with decode('utf-8'), that I would like to convert into JSON is as follows: json_thing = b'{\n& ...

Forecasting with a combination of various input variables in a time series analysis

Imagine we are working with a time-series dataset that shows the daily count of orders over the past two years: https://i.stack.imgur.com/vI2AA.png To predict future orders, Python's statsmodels library can be used: fit = statsmodels.api.tsa.statesp ...

Attempting to select an input field

I'm having trouble clicking on the Select Files button that is nested within a SPAN and Input Tag. Despite using Xpath, Id, and Name, I am unable to successfully click on the button. <span> Select <u>a</u> Files... </span> & ...

Differentiating between the components and contents of a webpage during the process of web scraping

import mechanize from bs4 import BeautifulSoup import urllib2 import cookielib cj = cookielib.CookieJar() br = mechanize.Browser() br.set_handle_robots(False) br.set_cookiejar(cj) br.open("*******") br.select_form(nr=0) br.form['ctl00$BodyConten ...

Appium is reporting a console error stating, "The specified search parameters were unable to locate an element on the page."

Having difficulty finding the element in my mobile app. ` public class Ovex { private static AndroidDriver driver; public static void main(String[] args) throws MalformedURLException, InterruptedException { DesiredCapabilities capabilities = n ...

Error encountered in Colab when importing keras.utils: "to_categorical" name cannot be imported

Currently utilizing Google's Colab to execute the Deep Learning scripts from François Chollet's book "Deep Learning with python." The initial exercise involves using the mnist dataset, but encountering an error: ImportError: cannot import name & ...

Could not find the button or link using any of the following methods: xpath, id, name, or css selector

Element not found using specified id, name, xpath or CSS selector Attempts were made with the following codes, but both failed to yield a response wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//*[@id=\'form\']/p/ ...

streamlining the deletion of Yahoo emails

Currently, I am utilizing Selenium IDE within Firefox to remove unread emails from Yahoo mail. With approximately 30,000 unread emails in our inbox, it seems that Yahoo only deletes less than 100 at a time. I have recorded the necessary steps for this pro ...

What could cause pandas to return a sum of 0 when using .sum(axis=1) with numpy datetime64 values in one row?

My data includes a mixture of floating numbers and numpy datetime64 values in different rows within a pandas dataframe. df2 = pd.DataFrame( [[np.datetime64('2021-01-01'), np.datetime64('2021-01-01')], [2, 3]], columns=['A&a ...

Maximizing the Benefits of Scrapy's Concurrency Using Non-Selenium Requests

Currently facing an intriguing challenge while developing a Scrapy web scraper for extracting products from a website. The obstacle lies in the fact that the catalog pages on the site utilize lazy-loading, restricting me to gather only the initial 12 items ...

What is the best method for identifying distinctive patterns within a csv dataset?

I have a CSV file that contains log data. There are only two columns I am interested in, namely 'case:concept:name' and 'concept:name'. My goal is to identify unique traces and determine how many times these unique traces repeat in the ...

Utilizing Python with Selenium to automate clicking on a button depending on its type, value, and class attributes

Here is the structure of the HTML code: <button type="submit" name="page" value="2" class="_btn _btng">Next →</button> <button type="submit" name="page" value="1" class="_btn">← Back</button> I am trying to click on the "Next ...

Developing a Node/Express API that initiates an asynchronous Python subprocess

Essentially, it involves spawning a Python subprocess which in turn spawns another asynchronous Python subprocess. However, due to the lengthy title, Node/ExpressJS is supposed to wait for the initial Python subprocess to ensure its successful execution, b ...

werkzeug.exceptions.BadRequestKeyError: 400 Bad Request: The server is unable to process the request sent by the browser or proxy. This error occurred in a Flask web application

Can someone guide me on troubleshooting this issue in Flask? I am still learning. Server running at (Press CTRL+C to exit) 127.0.0.1 - - [26/Jul/2020 11:19:45] "GET /predict HTTP/1.1" 500 - Traceback (most recent call last): raise exceptions. ...

Issue with setting capabilities in Firefox Selenium causing user agent to remain unchanged

This code was functioning smoothly, however, upon transitioning from Chrome to Firefox, it has started throwing an error. I am seeking assistance in resolving this issue FirefoxOptions options = new FirefoxOptions(); options.addArguments("--incog ...

Having trouble choosing an option from the dropdown menu with Puppeteer Js

I need help with Puppeteer JS to select the initial element in a dropdown. Any suggestions? Once I input the city name in the text field, I want to choose the first option from the dropdown menu. const puppeteer = require('puppeteer'); (async ...

Can we enhance the optimization of curve fitting when one variable depends on another?

Is it feasible to perform curve fitting on a dual exponential function where one parameter is strictly greater than another? I'm unsure of how to enforce this constraint in my code. Below is a simple example: import numpy as np import matplotlib.pyp ...

Error: Attempting to append to a `dict` object in JSON is not a valid operation

Why isn't the append method working on dictionary objects? Here is my code snippet: with open (file) as f: for line in f: review = json.loads(line) review.append((review['reviewText'],review['overall']))#append ...

Can anybody provide information on clearing the CACHE Memory of a browser programmatically with Selenium WebDriver?

Is there a way to clear the browser's CACHE Memory without having to delete Cookies and risk logging out of my account? ...