Questions tagged [python]

Python, a versatile and dynamic programming language, offers a multitude of functionalities. Known for its easy comprehensibility and efficient syntax enforcement, Python has gained popularity among developers. However, it's important to acknowledge that Python 2 is no longer supported since January 1, 2020. If you have questions regarding specific versions of Python, kindly include the [python-2.7] or [python-3.x] tag. Additionally, when utilizing alternate Python variations such as Jython or PyPy, or incorporating libraries like Pandas or NumPy, please remember to include relevant tags.

Is it expected behavior for PyCharm to be unable to refactor a Python enum member decorated with @unique?

PyCharm 2022.3.1, Build #PY-223.8214.51, assembled on December 20, 2022 Python 3.10.6 When an enum is marked with the @unique decorator and is defined in a separate file, PyCharm seems to have difficulty finding usages for refactoring or renaming. It als ...

pythonenumspycharm

Is there a way to add a pause between typed characters when using .send_keys()?

I've been working on automating an online application and I'm looking for a way to make the ".send_keys()" function more authentic. Instead of inputting "[email protected]" rapidly into the text field, I want to introduce a slight delay betw ...

pythonselenium

Saving output to a document. [Programming in Python]

I have successfully completed a task given by my teacher to generate attributes for a user based on their name. I used Json to write the data to a file and managed to do it with just 'math' and 'random', as per the requirements. However ...

pythonjsonfileattributes

What could be causing the increase in file size after running PCA on the image?

I am currently working on developing an image classification model to identify different species of deer in the United States. As part of this process, I am utilizing Principal Component Analysis (PCA) to reduce the memory size of the images and optimize t ...

pythonimagemachine-learningimage-processingpca

What methods can be used to retrieve specific components from a given string?

https://i.stack.imgur.com/rbxYv.png I have a specific string that contains information about teams named 'a' and 'b'. My goal is to extract various elements from this string, such as 'City', 'Team Name', 'Sport ...

python

Deducing characteristics of Azure virtual machines from text with Python Software Development Kit

Looking to automatically determine the available memory and number of vCPUs based on an Azure VM size provided as a string (e.g. "STANDARD_A4_v2"). I've searched through azure-mgmt-compute without success. I came across this post which uses a ComputeManag ...

pythonazureazure-sdk-python

Is there a Python framework that specializes in serving images?

My iOS app project requires a backend server capable of efficiently handling image files and dynamic operations like interacting with a data store such as Redis. Python is my preferred language for the backend development. After exploring various Python w ...

pythonimageframeworkswebserverstatic-files

Python Selenium: Typing speed is too slow

When attempting to fill out a form quickly using element.send_keys("Anything"), I am finding that it takes significantly longer than expected. I have experimented with various Chromedriver versions, but the issue persists. Are there any possible reasons wh ...

pythonperformanceseleniumkeysend

Troubleshooting Matplotlib and Pyplot Import Problem in Python with Tkinter

Having an issue here where the 'pyplot' element of 'matplotlib' cannot be called. In the given code snippet, I had to include "TkAgg" for the matplotlib element to function properly, as it is a common problem. import matplotlib matplotlib.use("TkAgg") Bu ...

pythonpython-3.xmatplotlibtkinterpython-import

In Python, generating a fresh dataframe with a list of distinct dates as the index

I am working with a dataframe where the index is set as df.index = 2016-08-01 06:45:00 2016-08-01 07:00:00 2016-08-01 07:15:00 . . 2018-03-28 11:30:00 2018-03-28 11:45:00 2018-03-28 12:00:00 My goal is to create a new dataframe that ...

python

Using Python Webdriver to Execute JavaScript File and Passing Arguments to Functions

How can I execute a JavaScript function and pass arguments to it? value = driver.execute_script(open("path/file.js").read()) I have successfully executed the file, but I am unsure of how to pass arguments to the function within it. Any suggestions would ...

javascriptpythonselenium-webdriver

Best method for retrieving JSON information from the internet using an API

Looking to work with a URL like this: http://site.com/source.json?s= My goal is to develop a Python class that can take in the "s" query, send it to the specified site, and then extract the JSON results. I've made attempts at importing json and setting ...

pythonjson

Unable to assign a default value of an object variable in an object method - Python

Using Windows 10, x64, and Python 2.7.8. I'm trying to implement the code below: class TrabGraph: def __init__(self): self.radius = 1.0 def circle(self, rad=self.radius): print(rad) test = TrabGraph() test.circ ...

pythonpython-2.7object

Differences between Local and Global importing in Python

Is there a way to force my interpreter (2.7) to import a module from site packages in case of a conflict? Let's say you are working with a directory structure like this: top_level ----cool_mod ----init.py ----sweet_module.py You already have swee ...

pythonpython-2.7import

Establishing a URL for the Django ImageField model attribute using a custom storage solution designed specifically for Azure Cloud Storage

I am encountering an issue with my Django web application. Users are able to upload images successfully, but for some reason the URLs of these images are not being set properly when stored in Azure Cloud Storage. As a result, when I try to display these im ...

pythondjangoazureazure-cloud-servicesdjango-storage

Swipe down to view all the links

Out of 3821 links, only 103 were provided to me. I tried applying the condition `window.scroll` to retrieve all the links, but unfortunately, it did not work as expected. from selenium.webdriver.common.by import By from selenium.webdriver.common. ...

pythonselenium-webdriverweb-scrapingbeautifulsoup

Guide to randomizing a collection of URLs and implementing it with the Webdriver

Looking to extract Href links from a website and shuffle them, then iterate through each line in the list to scrape each webpage using Selenium in Python. I have come across information on how to do this with a notepad file, but I am looking for guidance o ...

pythonpython-3.xseleniumselenium-webdriver

Tips for displaying only the decimal component in heatmap annotation floats

I have decimal floats and I'm only concerned with the numbers after the decimal point, as the integer part is irrelevant for my problem. When displaying these numbers on seaborn plots, I need to use string formatting. Currently, I am using '0.2f' to show ...

pythonpandasseabornstring-formattingplot-annotations

Utilizing Python with Selenium WebDriver to choose options from a concealed dropdown menu using jQuery

I am having trouble selecting options from a hidden dropdown box. The website I am trying to automate is www.geforce.com/drivers. Specifically, I need to automate the 'manual driver search' feature on that page. I have attempted to select option ...

jquerypythonselenium-webdriver

Having trouble converting a Jupyter notebook into a PDF file

Encountering an error while attempting to convert my Jupyter notebook to PDF. https://i.stack.imgur.com/LHHpj.png Any assistance would be greatly appreciated. Thank you. ...

pythonjupyter-notebook

What is the best way to retrieve a random selection of strings from a given input?

import random def generate_random_strings(characters): return [random.choice(characters) for _ in range(4)] if __name__ == '__main__': characters = 'ygobpr' print(generate_random_strings(characters)) What is the best way to obtain a list of 4 ...

pythonstringrandom

Leveraging Selenium to dismiss a browser pop-up

While scraping data from Investing.com, I encountered a pop-up on the website. Despite searching for a clickable button within the elements, I couldn't locate anything suitable. On the element page, all I could find related to the 'X' to cl ...

javascriptpythonseleniumweb-scrapingbeautifulsoup

Python: ArgumentError - this function requires 6 arguments, but you've provided 8

During my attempt to implement a gradient descent algorithm, I encountered an intriguing issue related to the ineffective use of **kwargs. The function in question is as follows: def gradient_descent(g,x,y,alpha,max_its,w,**kwargs): # switch for v ...

pythonfunctionmachine-learninggradient-descent

Analyzing the column titles within a Pandas Dataframe for comparison

Is there a way to compare the column names of two separate Pandas data frames? Specifically, I am interested in comparing the columns between my train and test data frames. There are some columns missing in the test data frame that I need to identify. ...

pythonpandasnumpymachine-learningdata-science

Inspecting contents of browser using Python for web scraping

Currently, I am utilizing Python along with Selenium and Chrome web drivers to conduct web scraping within Visual Studio Code. Upon sending a GET request like this: driver.get('https://my_test_website/customerRest/show/?id=123') I am curious ab ...

pythonseleniumselenium-webdriverweb-scrapingselenium-chromedriver

Encountering Keyerror while trying to parse JSON in Python

Recently, I developed a program for extracting data from an API that returns information in JSON format. However, when attempting to parse the data, I encountered a key error. Traceback (most recent call last): File "test.py", line 20, in <module> ...

pythonjsonstringparsing

Using Python and Selenium, receiving the error message "element is not attached to the page document"

I am facing an issue with a python function that is supposed to cycle through all options of a product: submit_button = driver.find_element_by_id('quantityactionbox') elementList = submit_button.find_elements_by_tag_name("option") for x in elementList: ...

pythonselenium

Is it possible for a dictionary to be altered in an indirect manner?

Forgive my simple question, but I have yet to find a thorough solution. Consider the following: a_dict = {"A":0, "B":0} def GetElementRef(index): return a_dict[index] would_like_to_modify = GetElementRef("A") would_like_to_modify = 2 In this scenari ...

pythonlist

Unable to close Asynchronous Web.py Web Server

Referring to this example, I am curious why it's not possible to terminate this program using Ctrl+C: #!/usr/bin/env python import web import threading class MyWebserver(threading.Thread): def run (self): urls = ('/', 'MyWebse ...

pythonmultithreadinghttpserver

Unknown error occurred when changing the default login to email

There seems to be an issue with the specified field (username) for the Profile model. Please review the fields/fieldsets/exclude attributes in the CustomUserAdmin class. I encountered the above error, which I believe is related to changing the default log ...

pythondjango

Encountering a problem while executing a Python script with Selenium on GCP Cloud Run

I have a Python script that requires logging in and retrieving an access_token from an authentication server. The process involves navigating to the authentication server's URL, entering a username and password, clicking 'login', waiting for ...

pythonselenium-webdrivergoogle-cloud-platformselenium-chromedrivergoogle-cloud-run

Navigating through a loop with uncertain iterability of your object

There are instances where I have multiple axes in axes[0], while other times there is just one. To iterate over them, I use the following approach: for ax,_x in [(axes[0], X[0])] if len(X)==1 else zip(axes[0],X): Is there a preferred or more common way t ...

pythonpython-3.x

Extracting timestamped text data from a simulated chat interface

I am looking to gather chat data from Twitch clips. These are saved moments from livestreams where viewer reactions are captured. Take a look at this example clip: While I can scrape all the data by watching the video and utilizing query selectors, my goa ...

javascriptpythonnode.jsweb-scrapingtwitch

What is the best way to merge columns with dummy variables in Pandas while maintaining just one output?

How can we combine multiple columns from the data frame shown below to create a new column that contains either 1 if at least one of the columns has a value of 1, or 0 if none of the columns have a value of 1? data = {'cat_1': [1, 0, 1, 1, 0], &a ...

pythonpandasdummy-variable

"Selenium encounters a NoSuchElementException when trying to interact with a dropdown menu located inside an

I have been trying to access the voter list database through this link Unfortunately, I am unable to retrieve the first drop-down menu. I am currently using selenium version 3.14 This is the code that I have written: user_agent = "Mozilla/5.0 (X11; ...

pythonseleniumxpathiframewebdriverwait

Error encountered when attempting to resample time series data using Python's pandas module

I have a dataset with time-series data in the following format: ticker close created_at 2020-06-10 18:30:00+00:00 TSLA 1017.419312 2020-06-10 17:02:00+00:00 TSLA 1014.354980 2020-06-10 17:03:00 ...

pythonpandas

Locating the highest point in an extensive dataset using Python

Is there a method to filter out data points that are far from the peak? For instance, if my dataset contains 10 million points and the peak is approximately at 5 million, how can I remove points that are significantly distant from the peak in order to pin ...

pythonarrays

datetime displaying in an alternate format than the true value

I am attempting to reformat a datetime pattern within CSV files: Original date format: DAY/MONTH/YEAR Desired Outcome: YEAR/MONTH/DAY rows = df['clock_now'] contains the following data: 22/05/2022 12:16 22/05/2022 12:20 22/05/2022 12:21 22/05/2022 12:44 ...

pythonpandasdatetime

What is the best way to implement my function on every value in a JSON file?

I've got a JSON file that contains the following dataset: [{ "planet": "project pluto", "records": [ { "project": "project pluto", "plan": "paper" ...

pythonjson

The function 'read_json' in Pandas is not functioning properly as anticipated

Having trouble loading a JSON file with pandas as expected! I've checked various Stack Overflow answers but my issue doesn't seem to be there. The structure of the JSON file is shown below: View JSON File Code snippet used to load the file:- import panda ...

pythonjsonpandas

Using chromedriver-autoinstaller while restricted by a firewall

Can Chromedriver-autoinstaller be utilized behind a firewall? Alternatively, are there more effective solutions for managing ChromeDriver while operating behind a firewall? ...

pythonseleniumselenium-chromedriver

BeautifulSoup does not recognize circular HTML pages

Encountered an issue where the page parsing code consistently checks the same page every time, despite using it alongside selenium. Selenium has no problem opening new links, but the parsing only occurs on the initial page. The frustrating part is that si ...

pythonparsingselenium-webdriverweb-scrapingbeautifulsoup

Retrieve specific elements from a loop

I need help with a Python for loop that prints values ranging from 3 to 42.5. However, I am looking to store only the values between 1 and 8 in a variable. with requests.Session() as s: r = s.get(url, headers=headers) soup = BeautifulSoup(r.text, &apos ...

pythondatabaserequest

Discovering the smallest whole number from a selection (excluding any absent digits)

I have the beginning, final number, and a list of available values. start_number = 6001, end_number = 6190 List = [u'6001', u'6005', u'6008', u'6002'] Given this list, I am looking to find the smallest missing valu ...

python

Python 3.9.5: A possible bug where a single dictionary assignment is replacing multiple keys

Currently, I am parsing through a .csv file named courses. In this file, each row represents a different course containing an id, a name, and a teacher. My goal is to store this information in a dictionary. For instance: list_courses = { 1: {'id& ...

pythondictionarypython-3.7python-collections

Having difficulty printing a browser.find_element_by_xpath using Selenium in Python

I am trying to use the following code: print browser.find_element_by_xpath('/html/body/div[2]/div/div[3]/div/div/div[1]/div[2]/ul/li[4]/ul/li/span[3]/span[3]').text However, I keep getting an unexpected token error on the "browser.find_element_by_xpath" ...

pythonselenium

How to Send Java Packets to a Python Server?

I have recently started learning about sending UDP packets and I am encountering a challenge. I have successfully established communication between a Java client and server, as well as a Python client/server combo. However, I am struggling to figure out ho ...

javapythonclientudp

Using the ARIMA model with Python

Currently, I am utilizing ARIMA for forecasting in Python. Below is the code snippet I am working with: import numpy as np import pandas as pd import matplotlib.pyplot as plt from statsmodels.tsa.seasonal import seasonal_decompose from sklearn import d ...

python

Retrieve search results from Bing using Python

I am currently working on a project to develop a Python-based chatbot that can retrieve search results from Bing. However, my efforts have been hindered by the outdated Python 2 code and reliance on Google API in most available resources online. The catch ...

pythonweb-scrapingbing

Python's selenium fails to launch Chrome URL

Upon launching the browser, it remains stationary and unresponsive. The driver.get(url) command fails to initiate any action within the browser. from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.ch ...

pythonselenium-webdrivergoogle-forms

What is the best method for extracting all links from a webpage using selenium?

Currently, I am attempting to scrape a website using Python, Selenium, and Beautifulsoup. However, when trying to extract all the links from the site, I keep encountering an issue with invalid string returns. Below is my current code snippet - could some ...

pythonseleniumweb-scraping

Combining Flask with Celery: Maximizing Efficiency with Parallel Processes

My Flask Celery app instantiates the celery instance. I'm aware that I can add a normal Flask route to the same .py file, but would need to run the code twice: To run the worker: % celery worker -A app.celery ... To run the code as a normal ...

pythonflaskcelery

Solving puzzle captchas with Selenium while facing a maximum of 5 tries

I have been extracting data from this website: When using selenium to access the site, I sometimes encounter a puzzle captcha that needs to be solved after clicking on the "I am not a robot" captcha checkbox. The challenge is that the site only allows the ...

pythonselenium-webdriverrecaptcha

The Chrome Driver indicates compatibility with version 114, but it is actually intended to support version 113 - as mentioned in Selenium Python

from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.chrome.service import Service from webdriver_manager.chrome import ChromeDriverManager from time import sleep driver = webdriver.Chrome(service=Service(Chr ...

pythonselenium-webdriver

Finding a future variable in Python using Pandas based on time intervals in minutes

Apologies for the inadequate title, finding the right words to summarize my issue is challenging I am working with a dataset df1 that has the following index: [2014-01-02 10:00:02.644000, ..., 2014-01-02 15:59:58.630000] Length: 26761, Freq: None, Timezo ...

pythonpandas

If an <a> element is missing its href attribute, it will be non-clickable and unable to

I'm in the process of using Python/Selenium to automate the extraction of calendar appointment information from a specific website. When interacting with the webpage, triggering an action displays a popup containing detailed calendar appointment info ...

pythonselenium-webdriverclickhref

What is the best way to update values in a pandas data frame using values from a different data frame?

I have two data sets with varying sizes. I'm trying to substitute values in one dataset (df1) with values from another dataset (df2). I've tried looking for an answer on this platform, but I might not be articulating the question correctly. Any assistance ...

python

Python error: default variable is not initialized when importing the module

Imagine I have a module called foo.py that includes the following function: def bar(var=somedict): print(var) In my main program, main.py, I import this module, declare the variable somedict and then execute the function bar: from foo import * somed ...

pythonimportmodule

Display text by representing it as Unicode values

Is there a way to display a string as a series of unicode codes in Python? Input: "こんにちは" (in Japanese). Output: "u3053u3093u306bu307bu308cu307eu3057uf501" ...

pythonstringunicode

I encountered a problem extracting title URLs with Python during web scraping

I have encountered an issue while trying to scrape title URLs using my code. Can someone please help me troubleshoot it? Here is the code snippet: import requests from bs4 import BeautifulSoup # import pandas as pd # import pandas as pd import csv def ...

pythonweb-scrapingbeautifulsoup

"Python 3.5 successfully incorporates Html5Lib, while it is not supported in the 2.7

An error has been troubling me recently: Traceback (most recent call last): File "scrapeRecipe.py", line 29, in <module> br.select_form(name="aspnetForm") File "build/bdist.macosx-10.11-intel/egg/mechanize/_mechanize.py", line 619, in select ...

pythonpython-2.7systraceback

What is the best way to annotate specific portions of the cumulative total in matplotlib?

I am working on creating a basic histogram using matplotlib in Python. The histogram will display the distribution of comment lengths based on several thousand comments. Here is the code I have so far: x = [60, 55, 2, 30, ..., 190] plt.hist(x, bins=100) ...

pythonmatplotlibnlpdistribution

Scrapy spider malfunctioning when trying to crawl the homepage

I'm currently using a scrapy scrawler I wrote to collect data from from scrapy.contrib.spiders import CrawlSpider, Rule from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor from scrapy.selector import Selector from .. import items clas ...

pythonhtmlweb-scrapingweb-crawlerscrapy

What is the method to specify nu=4.0 (or any desired value) when utilizing the StudentsT() distribution in arch_model within Python?

How can I set nu=4.0 in this situation? model = arch_model(data) from arch.univariate import StudentsT model.distribution = StudentsT() ...

pythonmodeldistributionarch

The function process_request() was called with 2 arguments but only 1 was provided - A Tutorial on Basic HTTP Authentication Decorator

My current task involves creating a simple decorator that will allow us to apply basic HTTP Authentication to specific django views. This is the decorator I have implemented: def basic_auth(func): def process_request(self, request): if reque ...

pythondjango

Retrieve all href links using the Python selenium module

Exploring Selenium with Python has been my recent interest, and I was eager to extract all the links present on a particular web page using Selenium. Specifically, I wanted to retrieve all the links specified in the href= attribute of every <a> tag ...

pythonseleniumselenium-webdriverweb-scraping

Showing the heading again after a new page starts

Currently, I am using WeasyPrint to generate a document and facing an issue with long sections that may span multiple pages. When a section breaks across pages, the name of the section does not display after the page break as intended. The following examp ...

pythonhtmlcssweasyprint

Finding elements using CSS selectors in Selenium can be done by following these steps

Attempting to find an element in selenium using a css selector, I utilized the feature "copy css selector" and obtained: div>button[type ="submit"] Is this the accurate method? submit_button = driver.find_element_by_css_selector("input[t ...

pythonseleniumselenium-webdrivercss-selectorswebdriver

Generate a fresh jpg file upon user triggering the route in Flask using Ajax

app = Flask(__name__) @app.route('/' , methods = ['GET', 'POST']) def index(): print("hello") if request.method == 'GET': text = request.args.get("image") print(text) base64_img_bytes = text.en ...

pythonajaxflask

Converting strings in rows of a Pandas Dataframe into characters separated by commas

I am dealing with a dataframe that has sequence data in each row, formatted like this. MKEYGEDLK My goal is to process the sequence strings in each row to have them presented in this format: [M, K, E, Y, G, E, D, L, K] I attempted the following code: get ...

pythonpython-3.xpandas

Locating elements in Selenium using Python by matching only a portion of its id

I am currently working with Selenium in Python and I need to locate an element using only part of its id name. What would be the best approach for this? For instance, let's say I have already located an item with the id name coption5 like so: sixth_item ...

pythonseleniumselenium-webdriverxpathcss-selectors

Sending a file with an apostrophe in its name via scp in python

I am currently working on a Python script that is meant to copy files from a remote server to a local directory using scp. Due to the restrictions of running this script on an OpenELEC distribution (a minimal HTPC Linux distro with a read-only filesystem ...

pythonbashscp

Running multiple processes simultaneously is not supported on Windows when using Jupyter Notebook

Currently, I'm running into issues with multiprocessing on Windows using Jupyter notebook. Instead of running all my async's in parallel, they are being executed serially one at a time. Can someone offer guidance on what might be going wrong? I need to s ...

pythonpython-2.7jupyter-notebookpython-multiprocessing

Find Instagram posts between specified dates using the Instagram Graph API

I am facing an issue while trying to fetch media posts from last month on an Instagram Business profile that I manage. Despite using 'since' and 'until' parameters, the API is returning posts outside of the specified time range. The AP ...

pythonpython-3.xfacebook-graph-apiinstagraminstagram-api

Can the client's Python version impact the driver Python version when using Spark with PySpark?

Currently, I am utilizing python with pyspark for my projects. For testing purposes, I operate a standalone cluster on docker. I found this repository of code to be very useful. It is important to note that before running the code, you must execute this ...

pythonapache-sparkpyspark