Filtering rows by the time difference between two datetime64 columns

I have a dataset that has the following structure

trip_id     start_date  start_station_id    end_date    end_station_id  subscription_type   journey_duration    weekday
0   913460  2019-08-31 23:26:00     50  2019-08-31 23:39:00     70  Subscriber  0 days 00:13:00     Sat
1   913459  2019-08-31 23:11:00     31  2019-08-31 23:28:00     27  Subscriber  0 days 00:17:00     Sat
2   913455  2019-08-31 23:13:00     47  2019-08-31 23:18:00     64  Subscriber  0 days 00:05:00     Sat
3   913454  2019-08-31 23:10:00     10  2019-08-31 23:17:00     8   Subscriber  0 days 00:07:00     Sat
4   913453  2019-08-31 23:09:00     51  2019-08-31 23:22:00     60  Customer    0 days 00:13:00     Sat

After calculating the journey duration using the formula:

trip_data['journey_duration'] = trip_data['end_date'] - trip_data['start_date']

I now want to filter out rows where the journey duration is more than, for instance, 36 hours.

I attempted the following method but it did not work as expected:

trip_data2 = trip_data[(trip_data['journey_duration'] < 1days 12:00:00) ].copy()

If anyone has any suggestions on how to achieve this, I would greatly appreciate it.

Thank you

pythondatetime

Answer №1

Here's a solution to your problem:

# Converting start_date and end_date to datetime objects:
df["start_date"] = pd.to_datetime(df["start_date"])
df["end_date"] = pd.to_datetime(df["end_date"])

# Filtering rows based on time difference less than 36 hours:
df_filtered = df[(df["end_date"] - df["start_date"]).dt.total_seconds() < 36 * 60 * 60]
print(df_filtered)

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Python tutorial: utilizing the join() and sort() methods

My goal is to take an input string and produce a list of unique lowercase letters from that string, sorted in alphabetical order without any punctuation marks. For instance, if the input is "happy!", the output should be ['a','h',' ...

pythonfunctionsortingjoin

Retrieving data in JSON format from an API and converting it into a more usable JSON format using Flask

How can I convert data from JSON format into a web application using Flask to populate values in HTML? Here is the code in spec_player.html: {% for p in posts: %} <p>{{p.first_name}}</p> {% endfor %} This method works (main.py): posts = ...

pythonjsonapiflask

I am unable to execute this Python script

Having an issue with a syntax error while running my Python program. age = int(input('how old are you?22') Next_year_age = age + 1 print(f'on my next birthday, I will be {next_year_age}.') ...

python

What is the best way to use __repr__ with more than one argument?

(Python 3.5.2) In my code, I have implemented the __repr__ method for a specific class like this: class d(): def __init__(self): self._values = [] return def __pos__(self): return self._values[0] def __repr__(self, val ...

pythonpython-3.x

Ways to merge two dataframes of varying lengths when both have datetime indexes

I am dealing with two different dataframes as shown below: a = pd.DataFrame( { 'Date': ['01-01-1990', '01-01-1991', '01-01-1993'], 'A': [1,2,3] } ) a = a.set_index('Date') ------------- ...

pythonpandasdataframeconcatenation

Finding the "send item" button in Outlook Office with the help of Chromedriver

Hey there, I'm having some trouble clicking on the "New Group" option in the office365 menu using chromedriver with selenium. Unfortunately, when I try to run the script I created, it doesn't seem to be working as expected. Here is a screenshot s ...

pythonseleniumselenium-chromedriver

How can we group data by minute, hour, day, month, and year?

I've been trying to find a resolution for my current issue, but I seem to be stuck. I'm really hoping that you can assist me. The Issue: My goal is to determine the number of tweets per minute. Data Set: time sentime ...

pythonpandasdataframetime

Can a unique and optional attribute be defined for a class model in (Flask) SQLAlchemy?

I am looking to define a unique and optional attribute in Python Sql Alchemy with Flask and Flask-SQLAlchemy. Is this achievable? membership_number = db.Column(db.String(120), unique=True) ...

pythonflasksqlalchemyflask-sqlalchemy

`Inconsistent data retrieval encountered during web scraping across multiple URLs`

I found a collection of approximately 170 URLs, all originating from the same website www.blackoffer.com, where articles on various topics are published. My objective is to extract the Article Title and corresponding Paragraphs from each page. To accompl ...

pythonselenium

Unable to add several objects to a dictionary value that is of type list

Struggling to move forward, feeling overwhelmed. I have a certain arrangement (referred to as event_dict): { 'ABC123':[], 'ABC234':[], 'ABC444':[] } Along with something like this (known as event_list) : [ {'c ...

pythonstructure

Using Python to validate JSON keys and values

I'm currently working on developing a Python code that takes a JSON file as input. This JSON file can contain various data structures such as dictionaries and lists. The main goal of my program is to print out the keys present in the JSON file. Howeve ...

pythonjsonlistdictionarykey-value

Launching a blank webpage by employing Selenium and a web driver

I'm trying to create a webpage using chromedriver and here is the code I have so far: from selenium import webdriver url = "wow.com" driver = webdriver.Chrome("/Users/macbook/Desktop/chromedriver") driver.get(url) But when I run it, this is what h ...

pythonpython-3.xseleniumwebdriver

Replace HTML elements with AJAX and JavaScript

Working with MySQL data in pyramid presents a challenge as I need to dynamically change an HTML if statement based on the results from JS ajax calls. The main page receives data from views.py and passes it to the .mak script. The key views here are the ma ...

javascriptpythonhtmlajax

Is there an excess memory consumption associated with objects encapsulated by pybind11?

I'm curious about the potential memory implications of using C++ classes or structs wrapped by pybind11. Let's explore a simple scenario: struct Person { std::string name; int age; } // Demonstrating basic bindings pybind11::class_<Perso ...

pythonc++pybind11

Discovering an element within the identical class as another element using Selenium in Python

If I have already located an element, is there a method to find another in the same category? For instance: <div class="day"> <span class="day_number">4</span> <span class="day_item_time" data-day-total-time="day-total-time">1 ...

pythonseleniumselenium-webdriverwebdriver

Creating spinboxes in Python and retrieving integer data from them instead of strings

I'm currently working on developing a calculator application that provides an estimate of annual gas expenses. In order to execute the program, I need to use an algorithm that retrieves input data from spinboxes as integers. However, when attempting t ...

pythoncanvastkinterinttkinter-canvas

Typing in a tkinter entry box effortlessly without the need to first click on it

Every time I launch my tkinter program, I find myself having to click on the entry box before I can start typing. Is there a way to configure it so that I can type without the need to click on the entry box? Thanks in advance. ...

pythontkinter

Python application for flattening and mapping nested JSON keys

I am completely new to JSON and struggling with understanding the structure of a JSON file. Here is an example of a JSON file I have: {"employeeId":{"0":02100, "1":02101, "2":02102,... "1000000":021000000}, "employeeName":{"0":"Smith", "1":"John", "2":" ...

pythonjsondictionarykeyflat

Incorporating a color-coded legend onto a Folium map for

When creating a map in Folium with multiple layers, each containing shaded areas (utilizing GeoJSON) colored by a colormap, I encountered an issue with adding legends to my layers. An initial solution was found here, but it posed problems as the legend rem ...

pythonvisualizationgeospatialcolormapfolium

What is the best way to assign a unique number to every div id that is generated?

I am currently using a django for loop to retrieve data from a query set. As the information is displayed, I would like to have each item wrapped in a div tag with a unique id that increments by 1 for every instance. Is there a way to achieve this directly ...

javascriptjquerypythondjango