Add SeasonGrouper, SeasonResampler

[dcherian]

These two groupers allow defining custom seasons, and dropping incomplete seasons from the output. Both cases are treated by adjusting the factorization -- conversion from group labels to integer codes -- appropriately.

Docs are here: https://xray--9524.org.readthedocs.build/en/9524/user-guide/time-series.html#handling-seasons

The last piece from #8509

• Closes Ordered Groupby Keys #757
• Closes Computing 'seasonal means' spanning 4 months (with resample or groupy) #6180
• Closes xarray_ Group by season #6664
• Closes How to change or reorder grouping by 'time.season'? #5134
• Closes Error: Changing months combination in xarray season for India Region #6012
• Closes Calculating ONDJFM mean for each season #6865
• StackOverflow:
  • https://stackoverflow.com/questions/61139226/how-to-get-season-as-jjas-instead-of-jja-in-xarray
  • https://stackoverflow.com/questions/70658388/how-to-select-djfm-season-instead-of-xarray-groupby
  • https://stackoverflow.com/questions/69021082/december-january-seasonal-mean
  • https://stackoverflow.com/questions/64976340/keeping-time-series-while-grouping-by-season-in-xarray
  • https://stackoverflow.com/questions/68455725/how-to-enable-season-selection-as-jjas-instead-of-jja-in-xarray
  • https://stackoverflow.com/questions/60342881/efficient-workaround-to-group-by-multiple-time-coordinates-in-xarray
• Tests added
• User visible changes (including notable bug fixes) are documented in whats-new.rst
• New functions/methods are listed in api.rst

Example:

import xarray as xr
from xarray.groupers import SeasonGrouper, SeasonResampler

ds = xr.tutorial.open_dataset("air_temperature")

# custom seasons! 
ds.air.groupby(time=SeasonGrouper(["JF", "MAM", "JJAS", "OND"])).mean()

ds.air.resample(time=SeasonResampler(["DJF", "MAM", "JJAS", "ON"])).count()

TODO:

• Needs a boatload of tests
• narrative docs
• support drop_incomplete in SeasonGrouper
• more cftime calendar support

cc @tomvothecoder do you have time to contribute some tests? I bet we'll simplify a bunch of xcdat this way, and you probably already have tests :)

[oliviermarti]

First comment, but I have performed only quick test

1 -

In your short example, it's probably :
from xarray.groupers import SeasonGrouper, SeasonResampler
and not :
from xarray.core.groupers import SeasonGrouper, SeasonResampler

Or my Github knowledge is too limited, and I'm not testing the right branch.

2 - Season grouper

Seems OK for all I have tested. In particular I can :

• Use it for one season only (subsampling):
  ds.air.groupby(time=SeasonGrouper(["MAM"]))['MAM']

• Use it for 4 months season, as we often use in paleoclimate with calendar shifting (oversampling)
  ds.air.groupby(time=SeasonGrouper(["DJFM", "MAMJ", "JJAS", "SOND"])).mean()

3 - Season resampler

Works as expected from the example.
I would really appreciate to have access to subsampling :
ds.air.resample(time=SeasonResampler(["DJF", "MAM", "JJA", "ON"]))
or :
ds.air.resample(time=SeasonResampler(["JJAS"]))
and to oversampling :
ds.air.resample(time=SeasonResampler(["DJFM", "MAMJ", "JJAS", "SOND"]))

It could be useful to have a NaN value for an incomplete season : the first DJF cannot not be computed, and is not. This mean that the first value is not a DJF one, but a MAM value. Could be a bit misleading.

4 - cftime

I have tested it with cftime calendars instead of datetime. It works with the traditional calendar (gregorian, standard). But not with others like 360_day, 365_day, julian., proleptic_gregorian :
TypeError: cannot compute the time difference between dates with different calendars

5 - Simple data

I've build a dataset with the number ot the month as a variable. So I'm sure that the computation is correct.

Thanks' for these features. They are quit easy and straigthforward to use.

In particular, it allows to work on variables, as xcdat features work on Dataset only, which yields a more complicated syntax.

I'm gonna try to imagine further tests.

Olivier

[dcherian]

Thanks @oliviermarti ! this is incredibly helpful

In your short example, it's probably : from xarray.groupers import SeasonGrouper, SeasonResampler

Yes, my mistake. I fixed the snippet.

ds.air.groupby(time=SeasonGrouper(["DJFM", "MAMJ", "JJAS", "SOND"])).mean()

This should not work, did you really get correct results.

It could be useful to have a NaN value for an incomplete season

The drop_incomplete option should let you control this.

[oliviermarti]

ds.air.groupby(time=SeasonGrouper(["DJFM", "MAMJ", "JJAS", "SOND"])).mean()

This should not work, did you really get correct results.

In fact not ! Only the first value is correct. A bit dangerous that it returns a result and not an error.

It could be useful to have a NaN value for an incomplete season
The drop_incomplete option should let you control this.

drop_incomplete compute a value, using less month. This is the behaviour of xcdat. It would like to get a nan (I should ask that to xcdat too).

Olivier

[dcherian]

ds.air.groupby(time=SeasonGrouper(["DJFM", "MAMJ", "JJAS", "SOND"])).mean()

I went back to my dev notebook, turns out I figured this out already!

[tomvothecoder]

cc @tomvothecoder do you have time to contribute some tests? I bet we'll simplify a bunch of xcdat this way, and you probably already have tests :)

Hi @dcherian, thank you for this PR! I've been looking forward to having this feature in Xarray. No guarantees on a timeline, but I plan to start looking at this PR this week. I'll experiment with this feature and see how I can leverage it to simplify xCDAT PR #423 for custom seasons. I'll also try to contribute any useful tests.

[tomvothecoder]

Hey @dcherian, quick question. Will this PR add support for using SeasonGrouper along with Datetime components (e.g., ds.time.dt.year)?

For example, if I wanted to perform grouped averaging on year and custom seasons it might look like:

ds.air.groupby(time=[ds.time.dt.year, SeasonGrouper(["JF", "MAM", "JJAS", "OND"])]).mean()

[dcherian]

Yes, I think so this is conceptually similar to groupby(["time.month", "time.year"]) but we require dict inputs to be explicit

ds.air.groupby(
    {"time.year": UniqueGrouper(), "time": SeasonGrouper(["JF", "MAM", "JJAS", "OND"])}
).mean()

Here's an example, that is hopefully easy to interpret

Is this right?

One trouble I'm having here is defining good tests for arbitrary seasons. Do you have suggestions or existing ones in xcdat that i can generalize?

[tomvothecoder]

Yes, I think so this is conceptually similar to groupby(["time.month", "time.year"]) but we require dict inputs to be explicit

ds.coords["year"] = ds.time.dt.year
ds.air.groupby(year=UniqueGrouper, time=SeasonGrouper(["JF", "MAM", "JJAS", "OND"])).mean()

Here's an example, that is hopefully easy to interpret

Is this right?

Yeah that's exactly what I was looking for. Thanks!

One trouble I'm having here is defining good tests for arbitrary seasons. Do you have suggestions or existing ones in xcdat that i can generalize?

Sure I have some in xcdat that you can adapt for this PR. I will share them with you shortly.

[tomvothecoder]

Another question: If we're defining custom seasons with months that span the calendar year, those months are from the previous year correct?

For example for "NDJFM", "ND" should be from the previous year.

air.groupby(year=UniqueGrouper(), time=SeasonGrouper(["NDJFM"])) 

[dcherian]

Yes it tried to be that smart

[dcherian]

@tomvothecoder @oliviermarti i fixed the existing tests now, please try it out!

FWIW the need to support seasons=["JJAS"] is adding quite some complexity. We should consider not supporting it.

[tomvothecoder]

I'm writing a few tests right now. How do you want me to add them to your fork branch?

Another question: If we're defining custom seasons with months that span the calendar year, those months are from the previous year correct?

For example for "NDJFM", "ND" should be from the previous year.

air.groupby(year=UniqueGrouper(), time=SeasonGrouper(["NDJFM"])) 

I noticed in a test I'm writing for the above code that "ND" is being taken from the same year, not the previous year. I think we expect the previous year "ND" to be used instead. I will show a clear example once I add the test.

[dcherian]

Ah nice find. A PR to this branch should be the easiest

[dcherian]

@oliviermarti & @tomvothecoder are you able to do one more test run here? It's basically complete though could use more tests as always.

[oliviermarti]

Deepak, I wish I could try the new Groupers which are promising (as seen at https://discourse.pangeo.io/t/pangeo-showcase-xarrays-groupby-oh-my/4666) My question is : how can I get the correct xarray branch for that ? Is it possible ? I’ve try to run 2024-pangeo-showcase.ipynb with a fresh extract of xarray (main branch), and I get : from xarray.groupers import SeasonGrouper --------------------------------------------------------------------------- ImportError Traceback (most recent call last) Cell In[17], line 1 ----> 1 from xarray.groupers import SeasonGrouper ImportError: cannot import name 'SeasonGrouper' from 'xarray.groupers' (/Users/marti/Unix/GitHub/xarray/xarray/groupers.py <http://localhost:60151/Users/marti/Unix/GitHub/xarray/xarray/groupers.py>) Thanks for your help, Olivier

Le 19 mars 2025 à 05:07, Deepak Cherian ***@***.***> a écrit : dcherian left a comment (pydata/xarray#9524) @oliviermarti & @tomvothecoder are you able to do one more test run here? It's basically complete though could use more tests as always. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned. <https://github.com/oliviermarti> <https://github.com/tomvothecoder> <#9524 (comment)> <https://github.com/notifications/unsubscribe-auth/AAVYYTTQVRO5SKOBXZCFOCT2VDUQZAVCNFSM6AAAAABOREL7ECVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZVGI3DSOBQGY> dcherian left a comment (pydata/xarray#9524) <#9524 (comment)> @oliviermarti <https://github.com/oliviermarti> & @tomvothecoder <https://github.com/tomvothecoder> are you able to do one more test run here? It's basically complete though could use more tests as always. — Reply to this email directly, view it on GitHub <#9524 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVYYTTQVRO5SKOBXZCFOCT2VDUQZAVCNFSM6AAAAABOREL7ECVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZVGI3DSOBQGY>. You are receiving this because you were mentioned.

— Olivier Marti LSCE Bât 714 p. 1049 MERMAID Team Office : +33 1 69 08 77 27 Mobile : +33 6 45 36 43 74 ***@***.***

[dcherian]

This PR branch is the branch: https://github.com/dcherian/xarray/tree/custom-groupers

[dcherian]

If you like this PR/API please voice support here: #10198 (a 👍 on the OP would be enough)

[oliviermarti]

This PR branch is the branch: https://github.com/dcherian/xarray/tree/custom-groupers

If I go there and search for SeasonGrouper, it is found in the design_notes/grouper_objects.md, but not in the code.
See : https://github.com/search?q=repo%3Adcherian%2Fxarray+SeasonGrouper&type=code

I would be pleased to test SeasonGrouper. Or should I wait a litlle more ?

Olivier

[oliviermarti]

@oliviermarti & @tomvothecoder are you able to do one more test run here? It's basically complete though could use more tests as always.

Sure, I would like to. I just need more precision to get the right xarray version.

[dcherian]

I see it here: https://github.com/dcherian/xarray/blob/0d8210a6a5ffda32b80406ac077fec40a2a4fa15/xarray/groupers.py#L729

if you have the Github CLI it should be

git clone git@github.com:pydata/xarray.git
gh pr checkout 9524

or

git clone git@github.com:dcherian/xarray.git
git switch custom-groupers

[oliviermarti]

Hi,

My recent tests are really positive. The new Grouper syntax is simple and convenient. All this works with cftime calendars which is nice.

Now I would like to ask for two important features :

• Weighted season averages.

Within a season, months have different lengths. So I need averages that reflect that, by weighting by the length of the months. When computing seasons from monthly means, it is not the case. My data (like model model outputs) have time_bounds that can be used to compute the time delta that could be used as weights.

• Improved time axis.

I have data with time values at the middle of the time period (often middle of month), and time bounds at beginning/end of each period. This is very common in the CMIP database for instance. Model outputs are mainly averages on a given period (from the model time step to hour, day, month, etc ...). Period bounds are an important information.

I would like to have seasonnal averages that keep this kind of time axis : value at the middle of the period, and bounds. This will allows further computation with the right weighting by periods length. And plots will better reflect the averaging period.

Olivier

[dcherian]

Thanks for testing!

Yes, these are both important features, but we'll need to make progress on #8005 first, which is finally happening in #10137, #9671

[tomvothecoder]

I'll try testing if I can soon! Thanks again for this PR @dcherian.