correct simulated daily precipitation using monthly observed precipitation about python-cmethods

[btschwertfeger]

Hey @zhongwang,

thank you for being interested in this tool! Unfortunately I‘m on vacation until the middle of October and have no access to my personal computer - so I‘ll try to describe it and will provide an example as soon as I‘m back.

Of course you can correct simulated data with a daily time scale using monthly observed/reanalyzed data. For this you can create a data set with daily values and just put in the values of the respective month of your montly (observational) data set.

This can be done for example by just copying the sinulated data of the control period and then replace the values as described before.

The correction/adjustment will work as long as the grid of all input data sets have the same spatial resolution and of course you have to take a look at the method-specific requirements.

The application of cmethods or BiasAdjustCXX stays the same - as described in the documentation.

[btschwertfeger]

Thank you very much for the great tool. I have simulated the daily precipitation dataset and observed precipitation in monthly time scale. Is it possible to correct the daily data using monthly observation? If possible, can you please give an example to show how to do it using cmethods or BiasAdjustCXX ?

General

In order to correct/adjust a modeled/simulated time series (or multiple time series), three data sets are required for both the scaling and distribution-based methods.

These three data sets must include modeled data for the reference/control period (e.g., years 1991-2020) and modeled data that needs to be corrected/adjusted (period of adjustment/scenario period; e.g., years 2021-2050). In addition, observed or reanalysis data from the reference period are needed to maintain a connection to reality.

Deviations from these conditions will "result in results" that are not scientifically adequate.

Requirements

Both cmethods and BiasAdjustCXX iterate over individual time series. I assume that the mathematical foundations of the implemented methods, as well as their requirements, have been understood see cmethods docs or BiasAdjustCXX docs.

Answer

To return to the original question: Yes, it is possible to correct daily values of modeled data using the two mentioned tools.

As previously mentioned, this requires three datasets. Let's assume you have modeled and observed data for the reference period 1991-2020, but the observed data is only available as monthly averages. Therefore, you have 12*30=360 values.

You want to adjust modeled data for the period 2021-2050, which is available as daily (mean) values.

To achieve this, you can "stretch" the modeled and observed data of the reference period, by setting the monthly average for each day. This is necessary to ensure that the time series of the reference period have the same temporal intervals as the data you want to adjust.

Here's an example of how this can be done in Python:

import numpy as np
import xarray as xr


# Create some time ranges
historical_time_monthly = xr.cftime_range("1991-01-01", "2020-12-31", freq="M", calendar="noleap")
historical_time_daily= xr.cftime_range("1991-01-01", "2020-12-31", freq="D", calendar="noleap")
future_time_daily = xr.cftime_range("2021-01-01", "2050-12-31", freq="D", calendar="noleap")

# Generate fake observation data; monthly scale (reference period)
obsh = xr.Dataset(
    {
        "your_variable": xr.DataArray(
            data=np.random.uniform(low=0, high=1, size=(len(historical_time_monthly),)), # 360 because 30 years à 12 months
            dims=("time",),
            coords={"time": historical_time_monthly}
        )
    }
)
# Generate fake modeled data; daily values (reference period)
simh = xr.Dataset(
    {
        "your_variable": xr.DataArray(
            data=np.random.uniform(low=0, high=1, size=(len(historical_time_daily),)), # 10950 values (30 years, daily data /wo leap)
            dims=("time",),
            coords={"time": historical_time_daily}
        )
    }
)
# Generate fake modeled data; daily values (data to adjust)
simp = xr.Dataset(
    {
        "your_variable": xr.DataArray(
            data=np.random.uniform(low=0, high=1, size=(len(historical_time_daily),)), # same as above
            dims=("time",),
            coords={"time": future_time_daily}
        )
    }
)

So now we have the observational data of the control period in monthly scale, and the modeled data of both periods on a daily basis. We now need to adjust the observational data to transform it to daily data. For this we create a new dataset for the observational data with a daily scale.

obsh_daily = xr.Dataset(
    {
        "your_variable": xr.DataArray(dims=("time",), coords={"time": historical_time_daily})
    }
)

… and fill in the monthly values, so that the mean value for example for the January 1991, every day will have the value of that monthly mean of the source data set.

# Loop through years and months to assign the monthly values to the daily dataset
index = 0
for year in range(1991, 2021):
    for month in range(1, 13):

        # Create a boolean mask
        mask = (obsh_daily["time.year"] == year) & (obsh_daily["time.month"] == month)

        # … and assign the monthly value to each day
        obsh_daily["your_variable"][mask] = obsh["your_variable"][index]
        index += 1

Finally you can apply cmethods like:

from cmethods import CMethods as cm


result = cm.linear_scaling(
    obs=obsh_daily["your_variable"],
    simh=simh["your_variable"],
    simp=simp["your_variable"],
    group="time.month",
    kind="*"
)

You can also save the transformed data set (obsh_daily) to netcdf and apply BiasAdjustCXX as described in the docs and its repository.

Please keep in mind that this procedure may be artificial and modifying the reference data which is the base of your all that adjustment procedures, is not a good practice.

Let me know if there is still anything unclear.

[zhongwangwei]

Thank you very much for your answer and example. I found this tool is very useful. By the way, I have another question. Again, if the adjustment dataset is the same as the control dataset, the results are still reasonable?

[btschwertfeger]

As of my experience during my work at the Alfred-Wegener-Institute, adjustments of modeled data based on modeled and observed/reanalyzed data was a practice to tune model outputs to use them as inputs again. AFAIK this is not done anymore.

The significance of data used for analysis corrected like this may be biased, since it only relies on data of a single time frame.

So there were reasons for doing that but it just depends on your data, expected output and how others interpret them.

Please don't see my answers as single source of truth.