Python program that runs a loop through a file, sends API requests using the `requests` library, and saves the results

Currently working on a program to loop through movie titles in a text file and make API calls to store the responses.

The text file contains a single title on each line, for example:

Titanic
Avatar
A Star Is Born

The API being used is from a website called www.odmpapi.com

Below is the code that correctly constructs the URL:

import requests
import sys

prefixURL = 'http://www.omdbapi.com/?t='
suffixURL = '&apikey=xxx4s23'

text_file = open("url.txt", "w")

with open('print.txt', 'r') as f:
    for i in f:
        uri = prefixURL + i.rstrip(' \n\t') + suffixURL
        print uri
        text_file.write(url)
        text_file.write('\n')

text_file.close()

text_file = open("responses.txt", "w")


with open('url.txt', 'r') as f2:
    for i in f2:
        url = i.strip(' \n\t')
        batch = requests.get(i.rstrip(' \n\t'))
        data = batch.text
        print data
    text_file.write(data)
    text_file.write('\n')

text_file.close()

However, there seems to be an issue where only the response for the last title in the list is written to responses.txt.

Answer №1

Consider implementing this method:

import requests
import sys

base_url = 'http://www.omdbapi.com/?t={}&apikey=xxx4s23'

with open('input.txt', 'r') as f_input, open('results.txt', 'w') as f_output:
    for line in f_input:
        search_term = line.strip(' \n\t')
        url = base_url.format(search_term)
        print(url)
        response = requests.get(url)
        f_output.write("{},{}\n".format(url, response.text))

This script writes the URL and its response into a separate file. Python's .format() function allows you to insert your search term directly into the base_url without needing string concatenation. Simply substitute each placeholder {} with an argument.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

What is the best way to load all Arraylists from the model function?

I am integrating the FlexibleAdapter library to retrieve data from a JSON API in my android studio project. public void createHolderSectionsDatabase(int size, int headers) { databaseType = DatabaseType.MODEL_HOLDERS; HeaderHolder header = null; ...

The Significance of Strides in Tensorflow

I am currently delving into the concept of the strides argument within tf.nn.avg_pool, tf.nn.max_pool, and tf.nn.conv2d. The documentation emphasizes that strides: A list of integers with a length of at least 4. This denotes the stride of the sliding ...

Can one effectively retrieve data from Android SQLite database by querying JSON information?

Can Android SQLite databases be queried for JSON data, similar to how XML data can be stored and queried in sql-server using xpath? Is it possible to do something like this with sqlite? ...

Is there a way to initiate a JSON POST request from an iOS gadget and direct it towards an ASP.NET URL?

I am currently developing an application that requires data sharing with a windows server. After researching various examples online, I have come up with the following implementation below. Although both the iOS code and web-service are functioning properl ...

Issues with Drag and Drop functionality in Selenium using Python

I am trying to incorporate a drag and drop functionality using selenium with Python. I have set up the Chrome WebDriver and written the following code, but it doesn't seem to be working as expected. Any assistance would be greatly appreciated. from s ...

What is the best way to organize and calculate the average of data in a dataframe?

I am encountering an issue with my dataframe, which is structured like the image in the link below. https://i.stack.imgur.com/nKfiO.png My goal is to calculate the mean of the 'polarity' field, but I keep running into errors. grouped = df.group ...

Creating new variables in Pandas with multiple conditional statements

Struggling with Pandas multiple IF-ELSE statements and need assistance. The dataset I am working with is the Titanic dataset, displayed as follows: ID Survived Pclass Name Sex Age 1 0 3 Braund male 22 2 1 1 Cumings, Mrs. female ...

Is it possible for the colorbar to assign only one value as a particular value?

I have an image with a donut shape and I want to apply different colors only on that donut shape, while keeping the rest of the area black. How can I set the background to be black? fig, axe = plt.subplots() im2 = axe.imshow(data1, vmin=-30, vmax=5 ...

I encountered a RangeError with code [ERR_HTTP_INVALID_STATUS_CODE] due to an invalid status code being undefined

I have encountered a specific error while running my note-taking app. The error seems to be related to the following piece of code - app.use((err,req,res,next)=>{ res.status(err.status).json({ error : { message : err.message ...

Tips for understanding nested JSON or array data structure?

My data is stored in a constant called translations and it looks like this: { "item-0": { "_quote-translation": "translation Data", "_quote-translation-language": "English", "_quote-trans ...

color of the foreground/background input in a dropdown menu that

I am attempting to modify the foreground or background color utilized by a dash searchable dropdown when text is entered for searching in the list. The input text is currently black, which makes it extremely difficult to read when using a dark theme. Wit ...

Python code for generating a festive holiday tree

Looking for a way to represent a Christmas tree using just a sentence. Here's the code I've come up with so far, but it's not quite right. sentence = 'The whole Christmas tree.' for word in sentence.split(): for i, char in enu ...

The concept of setting a value is not defined in JavaScript when compared to

Within my primary python script, the following code snippet is present. @app.route('/accounts/test/learn/medium') def medium(): word = random.choice(os.listdir("characters/")) return render_template('accounts/test/medium.html', word=w ...

What is the best method to generate a new table in pandas that is built from an existing table?

After reading this post on reordering indexed rows in a Pandas data frame based on a list, I tried the following code: import pandas as pd df = pd.DataFrame({'name' : ['A', 'Z','C'], 'company ...

Encountering Issues While Installing TA-Lib on Debian Linux Dockerfile

Struggling with installing TA-Lib in a Dockerfile based on python:3.10.13-slim-bookworm. Despite following various examples, I continue to encounter errors: https://dev.to/lazypro/build-a-small-ta-lib-container-image-4ca1 https://github.com/deepnox-io/do ...

Python Problem: Frustrating StaleElementReferenceException

I am facing issues while using Selenium for a simple scraping task. The code is throwing StaleElementReferenceException randomly during execution, resulting in incomplete data retrieval. Despite experimenting with various waits in Selenium, I have not bee ...

Converting Laravel Custom Error Validation JSON Response to an Array

I am currently working on developing an API for a registration form and running into an issue with the error format. While the validator correctly displays errors in an object format, I require the JSON response in an array format. $validator = Validato ...

Getting the stack with Python selenium webdriver

I tried running this code snippet to test Selenium, but encountered an issue where the Firefox window did not open and no error messages were displayed. The print statement was also not executed. from selenium import webdriver driver = webdriver.Firefox ...

Error encountered while parsing JSON data with Gson: MalformedJsonException

Hello, I am working on a project that involves parsing JSON data. However, I keep encountering a Malformed exception while trying to do so. Below is the code I'm currently using: Object listOfGroups = sthService.getSthList(); Gson gson = new ...

Using Selenium with Python: Overcoming StaleElementReferenceException when selecting by class

Attempting to create a basic Python script using Selenium, I encountered a StaleElementReferenceException after the loop runs once. Check out the code snippet being executed: from selenium import webdriver browser = webdriver.Firefox() type(browser) bro ...