What is the best way to calculate the average of 2-dimensional arrays that contain nan values?

I have a collection of 12 .grd files located in the same directory, all formatted as 2-dimensional arrays.

To calculate their average values, I initially used a simple loop method

ave_value = np.zeros_like(test_array) # creating an array with zeroes
for i in range(0,12,1):
    file_path = %(i+1)+'.grd' #1.grd, 2.grd,...
    # read_grd is a custom function
    value, nodata_value = read_grd(file_path)
    value[value == nodata_value] = np.nan    
    ave_value_+=value
ave_value = ave_value/12.0 

However, I soon realized that this approach did not account for positions containing np.nan values.

For instance, while position [2,2] in one grid file was np.nan, it contained actual values in other datasets. In the averaging process mentioned above, the resulting average value in ave-value also turned out to be np.nan.

Is there a more effective solution to average different 2-dimensional arrays considering the presence of np.nan values? For example, if the specific position contains np.nan values in 3 files, we should only average the corresponding values from the remaining 9 files.

Any advice or suggestions you can provide would be greatly appreciated.

Answer №1

To calculate the mean while ignoring NaN values, you can utilize the np.nanmean() function. Here is a helpful link for more information: https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.nanmean.html

data = np.array([np.nan, 3.4, 5.2, np.nan, 1.2])
result = np.nanmean(data)
print(result)

>>> 3.266666666667

If you are working with 2D arrays or n-dimensional arrays, you can adjust the calculation of the mean by using the axis parameter to specify your desired axis.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Automated Menu Selection using Selenium

I'm currently facing a challenge in writing a Python script using Selenium to interact with a webpage. I am struggling to use the .click() method to select an expandable list on the page. Despite successfully logging in and navigating to the desired p ...

Convert .docx documents using python-docx to adjust font styles and sizes. Looking to restructure paragraphs within the designated files

In the process of transcribing .docx files, the goal is to change the font and font sizes while preserving attributes like bold, underline, and italic. Additionally, headers and graphics will be incorporated into the target.docx files that are created. Th ...

What happens when you return an HTTP 404 response without specifying a content-type?

Can Django return a 404 response without specifying a content-type? I attempted to use return HttpResponseNotFound() in my view, but it always includes the header Content-Type: text/html; charset=utf-8. ...

How many objects have been collected per initial URL?

With the help of scrapy, I have been able to crawl through 1000 different URLs and store the scraped items in a mongodb database. However, I am interested in knowing how many items have been found for each URL individually. Currently, from the scrapy stats ...

Tips on assigning html actions/variables to a flask submit button

I am facing an issue with my Flask app where the form submission process takes a while for the backend to process. In order to prevent users from refreshing or resubmitting, I want to display a loading gif during this time. The current HTML code for my fl ...

Performing matrix summation and multiplication using the numpy library

It has been quite a while since I last delved into the world of matrix math, almost two decades to be exact. A specific point exists in space: point1 = (x, y) Alongside this point is a scaler value: scaler = 0.5 An intricate transformational matrix ...

pygame zero - graphics in motion going downwards

One of the challenges in my game is implementing falling objects that appear at random positions on the screen. However, I keep encountering an error message that says: "empty range for randrange() (0,-799, -799)". It seems like the object must have a ne ...

I am looking to switch out existing HTML content with new content of my own

I have an html code snippet that I need to modify by replacing the existing content with "This is html code". Even though the code runs without errors, the content does not change. The specific text that needs to be replaced is ""A full stack develop ...

What is the best way to include a quotation within an f-string?

Currently in the process of conducting a PR review and stumbled upon this code snippet: value = "" for i in uuids: value = value + "'" + i + "'" + "," where_clause = fiel ...

How can I ensure that chromedriver in my Test Automation Suite directory is always kept up to date automatically?

Currently, my Automation framework utilizes Selenium and Python to operate. The Chrome browser is run using the chrome driver stored within a specific directory in the automation framework. However, the issue arises when the chrome driver undergoes updates ...

The error message "No module named Image" is indicating that there is a missing module in the specified file path build/exe.win-amd64-3.6/scipy/misc

I am working on a Python application for image processing that will be compiled into an EXE file. My tools of choice are tkinter and cx_Freeze. However, I have encountered an error during the process. https://i.stack.imgur.com/ThRs4.jpg Below is the con ...

I attempted to access the website through selenium using this code, but to my surprise, the site shut down immediately upon opening

Upon running this code, the webpage quickly opens and then just as swiftly closes. Despite using the most up-to-date version, I am unable to access the page. It is worth noting that I am a Windows user. The intention behind running this code was to access ...

Streamline the OAuth process for Dropbox API by automating the login with just a click of a button

Currently, I am facing an issue with logging into Dropbox using the official Dropbox API in order to upload a file. The problem lies in the code not clicking on the submit button for login authentication. Rather than producing an error, the code simply han ...

Tips on setting up virtualenv for Python 3 on a Mac

I have multiple versions of Python installed on my Mac and I want to set python3 as the default version while also using virtualenv and virtualenvwrapper. To achieve this, I created an alias in my ~/.zshrc alias python='python3' I also added t ...

issue with accessing global variable within class block

While the Python code within the 'outer' function runs smoothly on its own, placing it inside a class seems to cause issues that are difficult for me to comprehend. (I am aware that I could simply pass data as a variable inside the function or d ...

To resolve the SyntaxError in matplotlib when trying to swap axes in a 3D wireframe plot, make sure to specify named arguments instead of using expressions

Using Python 3.4.3 and unable to update the system, I encountered an issue while trying to create a 3D wireframe plot using matplotlib and mpl_toolkits.mplot3d. ax.plot_wireframe(*a,b, rstride=1, cstride=2) >> SyntaxError: only named arguments may f ...

The concept and notation of universal functions in numpy: understanding the basics

Currently, I am studying the numpy package and have come across this code snippet: import numpy as np a = np.array([[1,2,3], [4,5,6]]) np.add.reduce(a) My confusion lies in the dot notation: np.add.reduce(a) This seems different from the more straightf ...

Ways to save information to a .csv file chosen by the user using the file_uploader tool in Streamlit

I am currently working on a Web Application using Streamlit and have implemented a file uploader for users to select a CSV file in order to save data. However, I encountered an error while trying to write data from AgGrid to the selected CSV file using the ...

Extracting Features: 0% complete, still in progress indefinitely... using Python for a deep learning project comparing dogs vs. cats

import cv2 import numpy as np import os from random import shuffle from tqdm import tqdm ​ TRAIN_DIR = '/home/ajmal/Dogs vs cat/train' TEST_DIR = '/home/ajmal/Dogs vs cat/test' IMG_SIZE = 50 LR = 1e-3 CNN = 'dogsvscat ...

Is there a way to move all the elements in a nested python list in a downward direction?

In our scenario, we have a 4x4 grid with empty spaces and 1s grid = [ [" "," ","1"," "], [" "," ","1"," "], ["1","1"," "," "], ["1& ...