Questions tagged [python-requests]

Exclusively designed for the Python Requests library, this tool offers a comprehensive set of features and a user-friendly interface for handling HTTP requests in Python.

Python requests no longer retrieves HTML content

I am trying to extract a name from a public Linkedin URL using Python requests (2.7). Previously, the code was functioning correctly. import requests from bs4 import BeautifulSoup url = "https://www.linkedin.com/in/linustorvalds" html = requests.get(url ...

Python is struggling to scrape dynamically loaded elements from a webpage

Currently facing an issue with scraping a specific webpage. The link to the page is provided here. Within this webpage, there is a crucial Cross Reference section that I am trying to scrape. However, when attempting to collect the content using Python requ ...

What is the best way to extract a table from a website that has stored the table data separately from the HTML?

Attempting to extract table data from the following URL: In a previous scraping attempt, the Python packages utilized were: from bs4 import BeautifulSoup import requests import mysql.connector import pandas as pd from sqlalchemy import create_engine Howe ...

Python's Smartsheet API: A Guide to Retrieving Response Data from Clients

I'm currently tackling the issue of understanding and planning for rate limits, particularly concerning the get_cell_history method in the Python API. Upon executing the following code snippet, I observe the subsequent response being displayed in my conso ...

Enhancing JSON.dumps function in Python's Requests Package for a GET Request with additional ampersands

My goal is to make a basic get request to a public API using the python requests library. I am working with requests version 2.25.1 and Python 3.6. The issue I'm facing is that there is an extra & appearing in the URL parameters, and I can't ...

Collect information using Python's request module

import requests from pprint import pprint headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36', } params = ( ('LeagueID', '00'), ('Season', '2017-18'), ...

Obtaining the byte representation of an HTML response, such as using the 'response.content' feature in Python's 'requests' library, can be achieved by following a few

When using the Python requests library and receiving a response, what is the precise representation of response.content with UTF encoding? How can I convert a string or text (such as response.text) to the exact format of response.content? For example: r ...

Tips on sorting through a JSON response for particular keywords

When utilizing an API for a software, my main goal is to extract the ID of a specific item. Below is the code I am using to make the GET request: import requests import json url = "apiurl" payload = "" headers = { 'Authorization&a ...

Why is the raise_for_status() function not being triggered when using pytest for requests?

Within this block of code, there is a class called StsAPI with a method named _get_authorize_token. class StsAPI: def _get_authorize_token(self) -> str: try: resp = requests.post(settings.AUTHORIZATION_SERVIC ...

Using Python to iterate through various pages on an API and extracting specific data before printing it

Hey there! I'm new to programming and practicing my skills. I've been exploring the Star Wars API, which contains data on characters, films, planets, and more from the Star Wars universe. Right now, I'm working on looping through all the pages of the API t ...

python requests timeout isn't functioning as expected

Seeking to enhance my comprehension of the python request timeout, I decided to conduct a brief test. Based on my understanding, the timeout parameter is measured in seconds; hence, a timeout value of 1 signifies that if the connection or read time exceeds ...

werkzeug.exceptions.BadRequestKeyError: 400 Bad Request: The server is unable to process the request sent by the browser or proxy. This error occurred in a Flask web application

Can someone guide me on troubleshooting this issue in Flask? I am still learning. Server running at (Press CTRL+C to exit) 127.0.0.1 - - [26/Jul/2020 11:19:45] "GET /predict HTTP/1.1" 500 - Traceback (most recent call last): raise exceptions. ...

Python Requests encountered an error with too many redirects, surpassing the limit of 30 redirects

I attempted to scrape data from this webpage using the python-requests library. import requests from lxml import etree,html url = 'http://www.amazon.in/b/ref=sa_menu_mobile_elec_all?ie=UTF8&node=976419031' r = requests.get(url) tree = etree.HTML(r.te ...

Python may not always accurately detect function loop conditions

I have the following Python code snippet: def numTest(): getNum = "https://sms-activate.ru/stubs/handler_api.php?api_key=" + api + "&action=getNumber&service=go&country=3" numReq = requests.get(getNum) smsNum = n ...

Can anyone tell me why the file content disappears whenever I try to save my modifications in Python?

In my file test_final.txt, I have a collection of simple words and sentences that I want to shuffle using the shuffle method. However, when I implement the code, it empties out my test_final.txt. Additionally, I am struggling to understand why the Shuffle ...

Troubles encountered when trying to save an Excel.Writer document to a different directory

A timed backup system is being developed for an Excel document using Python, since there will be multiple users accessing it. The goal is to alter the file path away from the local directory. Below is the code snippet; import pandas as pd import datetime ...