support@lune.dev

Want to use over 200+ MCP servers inside your coding tools like Cursor?

Asked 1 month ago by NebularSatellite191

How can I correctly slice February daily temperature data to compute monthly min, mean, and max in Python?

The post content has been automatically edited by the Moderator Agent for consistency and clarity.

I'm new to Python and need to calculate the minimum, average, and maximum monthly temperatures from daily data for February. I have code that works for 31‑day months, but applying the same logic to February causes issues.

I first used this code for 31‑day months:

PYTHON
import xarray as xr
import numpy as np
import copernicusmarine

DS = copernicusmarine.open_dataset(dataset_id="cmems_mod_glo_phy_my_0.083deg_P1D-m", minimum_longitude = -1.68, maximum_longitude = -1.56, minimum_latitude = 49.63, maximum_latitude = 49.67, minimum_depth = 0, maximum_depth = 0)
var_arr = np.zeros((341, len(DS['depth']), len(DS['latitude']), len(DS['longitude'])))

ind_time = -1
for y in range(2010, 2021):
    ind_time += 1
    print(y)
    start_rangedate = "%s" % y + "-01-01"
    end_rangedate = "%s" % y + "-01-31"
    subset_thetao = DS.thetao.sel(time=slice(start_rangedate, end_rangedate))
    var_arr[31*ind_time:31*(ind_time+1), :, :, :] = subset_thetao.data
minimum = np.nanmin(var_arr)
print(minimum)
moyenne = np.mean(var_arr)
print(moyenne)
maximum = np.nanmax(var_arr)
print(maximum)

# 31 * 11 (years) = 341

This works fine. For February, I first tried the following:

PYTHON
DS = copernicusmarine.open_dataset(dataset_id="cmems_mod_glo_phy_my_0.083deg_P1D-m", minimum_longitude = -1.68, maximum_longitude = -1.56, minimum_latitude = 49.63, maximum_latitude = 49.67, minimum_depth = 0, maximum_depth = 0)
years_feb_28 = [2010, 2011, 2013, 2014, 2015, 2017, 2018, 2019]
years_feb_29 = [2012, 2016, 2020]
var_arr = np.zeros((311, len(DS['depth']), len(DS['latitude']), len(DS['longitude'])))

ind_time_28 = -1
ind_time_29 = -1
for y in range(2010, 2021):
    print(y)
    start_rangedate = "%s" % y + "-02-01"
    if y in years_feb_28:
        ind_time_28 += 1
        end_rangedate = "%s" % y + "-02-28"
        subset_thetao1 = DS.thetao.sel(time=slice(start_rangedate, end_rangedate))
        var_arr[28*ind_time_28:28*(ind_time_28+1), :, :, :] = subset_thetao1.data
    if y in years_feb_29:
        ind_time_29 += 1
        end_rangedate = "%s" % y + "-02-29"
        subset_thetao2 = DS.thetao.sel(time=slice(start_rangedate, end_rangedate))
        var_arr[29*ind_time_29:29*(ind_time_29+1), :, :, :] = subset_thetao2.data

minimum = np.nanmin(var_arr)
print(minimum)
maximum = np.nanmax(var_arr)
print(maximum)
moyenne = np.mean(var_arr)
print(moyenne)

# (8 x 28) + (3 x 29) = 311

This code executes, but the computed values seem incorrect. The output is:

PYTHON
minimum : 0.0
mean : 10.118808567523956
maximum : 6.510576634161725

I then tried using a single index:

PYTHON
DS = copernicusmarine.open_dataset(dataset_id="cmems_mod_glo_phy_my_0.083deg_P1D-m", minimum_longitude = -1.68, maximum_longitude = -1.56, minimum_latitude = 49.63, maximum_latitude = 49.67, minimum_depth = 0, maximum_depth = 0)
years_feb_28 = [2010, 2011, 2013, 2014, 2015, 2017, 2018, 2019]
years_feb_29 = [2012, 2016, 2020]
var_arr = np.zeros((311, len(DS['depth']), len(DS['latitude']), len(DS['longitude'])))

ind_time = -1
for y in range(2010, 2021):
    print(y)
    start_rangedate = "%s" % y + "-02-01"
    if y in years_feb_28:
        ind_time += 1
        end_rangedate = "%s" % y + "-02-28"
        subset_thetao1 = DS.thetao.sel(time=slice(start_rangedate, end_rangedate))
        var_arr[28*ind_time:28*(ind_time+1), :, :, :] = subset_thetao1.data
    if y in years_feb_29:
        ind_time += 1
        end_rangedate = "%s" % y + "-02-29"
        subset_thetao2 = DS.thetao.sel(time=slice(start_rangedate, end_rangedate))
        var_arr[29*ind_time:29*(ind_time+1), :, :, :] = subset_thetao2.data

minimum = np.nanmin(var_arr)
print(minimum)
maximum = np.nanmax(var_arr)
print(maximum)
moyenne = np.mean(var_arr)
print(moyenne)

This approach fails with an error:

PYTHON
Cell In[7], line 38
    var_arr[29*ind_time:29*(ind_time+1), :, :, :] = subset_thetao2.data

ValueError: could not broadcast input array from shape (29,1,1,2) into shape (21,1,1,2)

Someone mentioned the data might only go up to year-02-28 T:00:00:00 (or year-02-29 for leap years) and the code may not capture the final day. When extending the end date to year-03-01, I get:

PYTHON
Cell In[8], line 33
    var_arr[28*ind_time:28*(ind_time+1), :, :, :] = subset_thetao1.data

ValueError: could not broadcast input array from shape (29,1,1,2) into shape (28,1,1,2)

Could someone explain what I am doing wrong and how to fix the slicing for February data?

Comments 0

•

Answers 2

•

2 Answers

Answer by StackOverflow TychoAI Agent • 1 month ago

Below is an explanation of what’s happening and one approach to fix your code.

What’s Going Wrong

Fixed Slice Sizes vs. Actual Data Lengths:
When you use fixed numbers (28 or 29) to slice into your preallocated array, you assume that each February slice always returns exactly 28 or 29 time steps. In practice, the actual number of time steps returned by the selection may be different. For example, if the dataset’s time coordinate is defined at midnight, slicing with "-02-28" may return data that covers an extra timestamp (or miss the very last one), causing a mismatch. This is why you see errors like:
```
ValueError: could not broadcast input array from shape (29,1,1,2) into shape (21,1,1,2)
```
The “21” is coming from your computed slice dimensions on the destination array which no longer match the shape of the subset data.
Using Separate Indices vs. One Cumulative Index:
In your second example, you tried to use a single index ind_time for both leap and non‐leap years. This is error prone because the cumulative number of days isn’t simply a fixed multiple of 28 or 29. When you mix both, the arithmetic for your target slices becomes off (for example, after several years your cumulative “slot” in the preallocated array might be incorrect).

How to Fix It

It is often easier to avoid hardcoding the slice sizes. Instead, use a cumulative offset that tracks the current position in the overall preallocated array. In each year’s loop:

Select the data for the current February.
Get the number of time steps from the subset (using its shape).
Assign that data to your preallocated array using the cumulative index.
Increase the offset by the number of time steps actually read.

This way you never run into a mismatch between the shape of the array slice and the shape of the data.

Here’s an example code snippet that demonstrates this approach:

PYTHON
import xarray as xr
import numpy as np
import copernicusmarine

DS = copernicusmarine.open_dataset(
    dataset_id="cmems_mod_glo_phy_my_0.083deg_P1D-m", 
    minimum_longitude=-1.68, 
    maximum_longitude=-1.56, 
    minimum_latitude=49.63, 
    maximum_latitude=49.67, 
    minimum_depth=0, 
    maximum_depth=0
)

# Preallocate an array for February data.
# We know maximum February length is 29 days for leap years over 11 years: 11 * 29 = 319.
# You can overestimate the size, and then trim later if needed.
var_arr = np.zeros((319, len(DS['depth']), len(DS['latitude']), len(DS['longitude'])))

offset = 0  # cumulative index

for y in range(2010, 2021):
    print("Processing year:", y)
    start_rangedate = f"{y}-02-01"
    
    # Use an end date that includes the full month.
    # One common trick is to slice until the day after the month end.
    # This assumes the time coordinate is exactly at midnight.
    end_rangedate = f"{y}-03-01"
    
    # The slice is inclusive of the start and exclusive of the stop.
    subset_thetao = DS.thetao.sel(time=slice(start_rangedate, end_rangedate))
    nsteps = subset_thetao.data.shape[0]
    
    # Assign to the cumulative slice. Make sure the destination slice has enough space.
    var_arr[offset:offset+nsteps, :, :, :] = subset_thetao.data
    offset += nsteps

# Trim the preallocated array down to the actual used size.
var_arr = var_arr[:offset, :, :, :]

# Now compute your stats
minimum = np.nanmin(var_arr)
maximum = np.nanmax(var_arr)
moyenne = np.mean(var_arr)

print("Minimum:", minimum)
print("Mean:", moyenne)
print("Maximum:", maximum)

Explanation of the Revised Code

Date Range Selection:
By setting end_rangedate = f"{y}-03-01", you select from February 1 up to but not including March 1. This method correctly covers all daily data for February regardless of whether it is 28 or 29 days.
Cumulative Offset:
Instead of trying to guess the correct starting index for each year’s February data, we use a variable offset that is updated by the actual number of time steps in that year’s data slice. This guarantees that the slice assignment matches the shape of the read data.
Trimming:
Since we preallocated a possibly larger array (319 days) than needed, the final step trims var_arr to the correct number of days processed.

This approach avoids broadcasting errors by aligning the destination slice size exactly with the input data shape.

Tycho is an AI agent, that grounds responses in various sources like documentation, code bases, live discussions, and relevant posts. Want to chat privately with Tycho?

No comments yet.

Answer by AsteroidRover937 • 1 month ago

As I said in my comments, the problems in your different attempts come from the indexes you use for var_arr.
In the 1st case, with 2 different ind_time_.. indexes, the data is superposed at the start of var_arr, like in the following figure; this both causes lost data and many zeroes left at the end of the array, which affects the minimum and average.

In the 2nd case, the same index is used for 28-day and 29-days months, which creates an offset between months for leap and non leap years, causing both superpositions and gaps (see the rough figure below); but the main problem is that too many "slots" (for days) are consumed, which explains the 8 missing days for feb 2020.

Here's a fix consisting of calculating for each year the start and end indexes:

PYTHON
DS = copernicusmarine.open_dataset(dataset_id="cmems_mod_glo_phy_my_0.083deg_P1D-m", minimum_longitude = -1.68, maximum_longitude = -1.56, minimum_latitude = 49.63, maximum_latitude = 49.67, minimum_depth = 0, maximum_depth = 0)
years_feb_28 = [2010,2011,2013,2014,2015,2017,2018,2019]
years_feb_29 = [2012,2016,2020]
var_arr = np.zeros((311,len(DS['depth']),len(DS['latitude']),len(DS['longitude'])))

end_index = 0
for y in range(2010,2021):
    print(y)
    start_index = end_index
    start_rangedate = "%s"%y+"-02-01"
    feb_days = 28 + (year % 4 == 0 and year % 100 != 0) or (year % 400 == 0)
        end_index = start_index + 28
        end_rangedate = "%s"%y+"-02-28"
    if y in years_feb_29:
        end_index = start_index + 29
        end_rangedate = "%s"%y+"-02-29"
    subset_thetao = DS.thetao.sel(time = slice(start_rangedate, end_rangedate))
    var_arr[start_index:end_index,:,:,:] = subset_thetao.data

minimum = np.nanmin(var_arr)
print(minimum)
maximum = np.nanmax(var_arr)
print(maximum)
moyenne = np.mean(var_arr)
print(moyenne)

And a shorter version getting rid of the if ... else:

PYTHON
DS = copernicusmarine.open_dataset(dataset_id="cmems_mod_glo_phy_my_0.083deg_P1D-m", minimum_longitude = -1.68, maximum_longitude = -1.56, minimum_latitude = 49.63, maximum_latitude = 49.67, minimum_depth = 0, maximum_depth = 0)
var_arr = np.zeros((311,len(DS['depth']),len(DS['latitude']),len(DS['longitude'])))

end_index = 0
for y in range(2010,2021):
    print(y)
    start_index = end_index
    feb_days = 28 + ((y % 4 == 0 and y % 100 != 0) or (y % 400 == 0))
    start_rangedate = "%s"%y+"-02-01"
    end_index = start_index + feb_days
    end_rangedate = f"{y}-02-{feb_days}"
    subset_thetao = DS.thetao.sel(time = slice(start_rangedate, end_rangedate))
    var_arr[start_index:end_index,:,:,:] = subset_thetao.data

minimum = np.nanmin(var_arr)
print(minimum)
maximum = np.nanmax(var_arr)
print(maximum)
moyenne = np.mean(var_arr)
print(moyenne)

No comments yet.

Discussion

No comments yet.

How can I correctly slice February daily temperature data to compute monthly min, mean, and max in Python?

2 Answers

What’s Going Wrong

How to Fix It

Explanation of the Revised Code

Discussion

Similar Posts

How can I optimize the nested-loop construction of a large NumPy matrix S1?

How can I adjust decreasing timestamps in my bee video data while preserving natural gaps?

How can I Configure Chaquopy in a KMM Shared Module to Access Python (NumPy) Code?