Description of the numerical weather prediction products at MET Norway
Check out the wiki page for more information.
Description of the numerical weather prediction products at MET Norway
License: GNU General Public License v3.0
Description of the numerical weather prediction products at MET Norway
Check out the wiki page for more information.
Dear developers,
I checked mean average error of metno historical predictions for year 2021 and compared to fmi-station data. One would expect prediction to get worse for a longer prediction time but that seems not to be the case for temperature and wind speed (for pressure it seems ok).
Do I have a bug in my code or can this be explained some other way, a typical plot of MAE with respect to time (0-54h) is below (here for temperature).
Hi,
First of all, thank you for all of your great products!
I'm downloading data from the MET post-processed products Rerun version 2 product for October 23rd, 2016 for Denmark. I find that some hours have precipitation that seems unreasonable high leading to daily precipitation above 400 mm (see the figure below for the sum of the hourly precipitation).
If one looks at the hour to hour precipitation for the coordinate (in lambert conformal conic the coordinate is (-291442.2, -764322)) with the highest precipitation they seem to be just consistently high, no specific hour stands out. Do any of you know why one would observe such high values?
Thanks!
Jens
Dear developers,
I have to change the order of lat lon in
https://github.com/metno/NWPdocs/wiki/Examples#plotting-timeseries-for-the-nearest-neighbour-of-a-lonlat-point
to get it work for Helsinki (MET Nordic dataset version3)
With code below helsinki position is ok (lat, lon order changed in transform compared to example above)
ds=xr.open_mfdataset("met_analysis_1_0km_nordic_20200912T12Z.nc")
lat=60.205791,
lon=24.929186,
crs= "+proj=lcc +lat_0=63 +lon_0=15 +lat_1=63 +lat_2=63 +no_defs +R=6.371e+06",
xname='x',
yname='y',
proj = pyproj.Proj.from_crs("4326", crs)
X, Y = proj.transform(lat, lon)
# Find nearest neighbour
x = ds.variables[xname].to_numpy()
y = ds.variables[yname].to_numpy()
Ix = np.argmin(np.abs(x - X))
Iy = np.argmin(np.abs(y - Y))
Hello metno,
I have noticed that the MEPS deterministic files from meps_det_2_5km_20231205T00Z.nc
to meps_det_2_5km_20240305T21Z.nc
are missing from https://thredds.met.no/thredds/catalog/meps25epsarchive/catalog.html.
Is it possible to know the reason of the recall and when they will be available again? Many thanks in advance.
In the documentation in
https://github.com/metno/NWPdocs/wiki/MET-Nordic-dataset#files
it says:
"The Archive/Rerun version 3 folder contains the most recent rerun of analyses and forecasts. "
Questions: where are the forecasts? I only find analyses.
For instance
wget https://thredds.met.no/thredds/fileServer/metpparchivev3/2020/06/01/met_analysis_1_0km_nordic_20200601T12Z.nc
gives me data
but
wget https://thredds.met.no/thredds/fileServer/metpparchivev3/2020/06/01/met_forecast_1_0km_nordic_20200601T12Z.nc
gives
ERROR 404:
If one browses manually analysis files are there but not the historical forecasts
Hi!
I extracted data from the operational dataset (https://thredds.met.no/thredds/catalog/metpparchive/2022/catalog.html) for the period 01/01/2021-01/07/2022. For the last few months (since about mid-March) the integral_of_surface_downwelling_shortwave_flux_in_air_wrt_time file has unexpected values.
Some files (e.g. https://thredds.met.no/thredds/catalog/metpparchive/2019/03/04/catalog.html?dataset=metpparchive/2019/03/04/met_analysis_1_0km_nordic_20190304T10Z.nc) have x_wind/y_wind variables, while others have wind_speed/wind_direction
"wind_direction_10m.standard_name" states "wind_from_direction"
is x_wind/y_wind intended as the "wind_from_direction" as well?
We are removing the proj4 attribute from the projection variable. This must be reflected in the wiki examples.
There seems to be many outliers in the radiation variable for the analysis products (http://thredds.met.no/thredds/catalog/metpparchivev1/catalog.html) . Here I made the assumption that the radiation variable represents deaccumulated values and that they represent deaccumulated values over a period of 1 hour. So I just divided the values by 3600s to get W/m^2. But I see that some values are too high. Look at the plots below. I just took one point inside one of Statkrafts catchments and plotted the radiation for the whole period (2013.09.01-2018.09.01). I have also included a plot where only the values from one of the years with the outliers are plotted.
@tnipen , @cristianlussana do you know the cause for this?
Missing variables in file MET Nordic analysis > Archive > Rerun_version_1 > http://thredds.met.no/thredds/dodsC/metpparchivev1/2016/11/08/met_analysis_1_0km_nordic_20161108T02Z.nc.html
Hei,
I'd like to get 3d data for Finland. Tried from "https://thredds.met.no/thredds/dodsC/mepslatest/meps_det_vc_2_5km_latest.nc"
In principle this data is good, but lat lon grid is pretty sparse in Finland (about 100km from one point to another...).
Is there any 3d data from metno that would be usable in Finland?
Or could you point out where in (Finland?) to get this data for free?
terveisin, Markus
Hello metno,
Since 2023-01-30 09:00, the MEPS files have a slightly different spatial grid values. They no longer are exactly spaced in increments of 2500 m.
import xarray as xr
import pandas as pd
T06 = xr.open_dataset(
'https://thredds.met.no/thredds/dodsC/meps25epsarchive/2023/01/30/meps_det_2_5km_20230130T06Z.nc'
)
T09 = xr.open_dataset(
'https://thredds.met.no/thredds/dodsC/meps25epsarchive/2023/01/30/meps_det_2_5km_20230130T09Z.nc'
)
diff_stats = pd.concat(
objs=[
T06['x'].diff(dim='x').to_series().rename('T06_x').describe(),
T06['y'].diff(dim='y').to_series().rename('T06_y').describe(),
T09['x'].diff(dim='x').to_series().rename('T09_x').describe(),
T09['y'].diff(dim='y').to_series().rename('T09_y').describe(),
],
axis='columns'
)
print(diff_stats)
Is this intended or an error? Thanks in advance
Dear Developers,
Could not find oreography info in MEPS dataset nc files (info that near Galdhøpiggen it data should show about 2400m).
Is this information available somewhere in the same 2.5km MEPS grid?
Terveisin, Markus
Under https://github.com/metno/NWPdocs/wiki/MEPS-dataset#meps-parameters we should add description of the various precipitation parameters and which we recomment to use. Possibly some metadata are wrong as well
Dear developers,
I try to run (copy pasted from https://github.com/metno/NWPdocs/wiki/Examples#extracting-and-plotting-a-variable )
import netCDF4
filename = "https://thredds.met.no/thredds/dodsC/metpparchive/"+\
"2021/08/01/met_analysis_1_0km_nordic_20210801T12Z.nc"
ncfile = netCDF4.Dataset(filename)
This results to
Error:curl error: Problem with the SSL CA cert (path? access rights?)
curl error details:
Warning:oc_open: Could not read url
Traceback (most recent call last):
File "/home/hu-mka/git/linux-weatherstation/machinelearning/dev/metno_demo1.py", line 7, in <module>
ncfile = netCDF4.Dataset(filename)
File "src/netCDF4/_netCDF4.pyx", line 2353, in netCDF4._netCDF4.Dataset.__init__
File "src/netCDF4/_netCDF4.pyx", line 1963, in netCDF4._netCDF4._ensure_nc_success
OSError: [Errno -68] NetCDF: I/O failure: b'https://thredds.met.no/thredds/dodsC/metpparchive/2021/08/01/met_analysis_1_0km_nordic_20210801T12Z.nc'
pip command given (as instructed in https://github.com/metno/NWPdocs/wiki/Examples#extracting-and-plotting-a-variable)
pip install matplotlib netCDF4 numpy pyproj xarray
pip freeze
shows
certifi==2022.6.15
cftime==1.6.1
cycler==0.11.0
fonttools==4.37.1
kiwisolver==1.4.4
matplotlib==3.5.3
netCDF4==1.6.0
numpy==1.23.2
packaging==21.3
pandas==1.4.3
Pillow==9.2.0
pyparsing==3.0.9
pyproj==3.3.1
python-dateutil==2.8.2
pytz==2022.2.1
six==1.16.0
xarray==2022.6.0
I have ubuntu 22.04
Are there some special versions of the libraries one should use? I remember this did work earlier (but with different versions of the libraries).
Terveisin, Markus
geospatial_lon_min: 18.1
, should be -18.1
e.g. https://thredds.met.no/thredds/dodsC/meps25epsarchive/2021/01/01/meps_det_2_5km_20210101T12Z.nc.html
Some of the files in the post-processed forecast dataset (http://thredds.met.no/thredds/catalog/metpparchivev1/catalog.html) have unit for the radiation which does not match with the magnitude of the values in this variable. In the example below the unit in the metadata is W m^-2 while the variable seem to represent aggregated (integral) values. @tnipen, @cristianlussana, @ivarsei can this be fixed?
f="met_forecast_1_0km_nordic_20140701T06Z.nc"
ds=nc.Dataset(f,'r')
rad = ds.variables['integral_of_surface_downwelling_shortwave_flux_in_air_wrt_time']
rad
Out[32]:
<class 'netCDF4._netCDF4.Variable'>
float32 integral_of_surface_downwelling_shortwave_flux_in_air_wrt_time(time, y, x)
coordinates: longitude latitude
units: W/m^2
standard_name: surface_downwelling_shortwave_flux_in_air
grid_mapping: projection_lcc
unlimited dimensions: time
current shape = (61, 1544, 1270)
filling on, default _FillValue of 9.969209968386869e+36 used
rad[:,:,0].max()
Out[33]: 75324344.0
I try to read data, one day at the time, and I get the following issues. I do not know which of 24 files in each file that ails
Day: 0
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/10/17/met_analysis_1_0km_nordic_20171017T00Z.nc
Day: 1
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/10/18/met_analysis_1_0km_nordic_20171018T00Z.nc
Read failed: 'air_pressure_at_sea_level' <class 'KeyError'>
Day: 2
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/10/19/met_analysis_1_0km_nordic_20171019T00Z.nc
Read failed: NetCDF: Not a valid ID <class 'RuntimeError'>
etc.
Day: 1
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/03/01/met_analysis_1_0km_nordic_20190301T00Z.nc
Read failed: 'time' <class 'KeyError'>
Does anyone know where these errors come from? The "Not a Valid ID" message seems to coma and go unpredictable.
Trying to read all dates in the period 20161127 up to but not including 20190501 gives this list of errors:
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/02/03/met_analysis_1_0km_nordic_20170203T00Z.nc
Read failed: master dataset http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/04/06/met_analysis_1_0km_nordic_20170406T00Z.nc does not have any variables to aggregate
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/04/06/met_analysis_1_0km_nordic_20170406T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/08/16/met_analysis_1_0km_nordic_20170816T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/08/17/met_analysis_1_0km_nordic_20170817T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/08/18/met_analysis_1_0km_nordic_20170818T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/09/17/met_analysis_1_0km_nordic_20170917T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/09/18/met_analysis_1_0km_nordic_20170918T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/09/25/met_analysis_1_0km_nordic_20170925T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/10/18/met_analysis_1_0km_nordic_20171018T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/10/19/met_analysis_1_0km_nordic_20171019T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/10/20/met_analysis_1_0km_nordic_20171020T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/10/21/met_analysis_1_0km_nordic_20171021T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/11/07/met_analysis_1_0km_nordic_20171107T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/11/08/met_analysis_1_0km_nordic_20171108T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/11/14/met_analysis_1_0km_nordic_20171114T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/12/25/met_analysis_1_0km_nordic_20171225T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/12/26/met_analysis_1_0km_nordic_20171226T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2017/12/31/met_analysis_1_0km_nordic_20171231T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/01/13/met_analysis_1_0km_nordic_20180113T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/01/16/met_analysis_1_0km_nordic_20180116T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/01/17/met_analysis_1_0km_nordic_20180117T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/01/25/met_analysis_1_0km_nordic_20180125T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/02/27/met_analysis_1_0km_nordic_20180227T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/02/28/met_analysis_1_0km_nordic_20180228T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/03/01/met_analysis_1_0km_nordic_20180301T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/03/02/met_analysis_1_0km_nordic_20180302T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/03/04/met_analysis_1_0km_nordic_20180304T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/03/05/met_analysis_1_0km_nordic_20180305T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/03/09/met_analysis_1_0km_nordic_20180309T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/03/12/met_analysis_1_0km_nordic_20180312T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/03/18/met_analysis_1_0km_nordic_20180318T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/03/22/met_analysis_1_0km_nordic_20180322T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/05/30/met_analysis_1_0km_nordic_20180530T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/06/01/met_analysis_1_0km_nordic_20180601T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/06/25/met_analysis_1_0km_nordic_20180625T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/06/28/met_analysis_1_0km_nordic_20180628T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/07/09/met_analysis_1_0km_nordic_20180709T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/07/10/met_analysis_1_0km_nordic_20180710T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/07/13/met_analysis_1_0km_nordic_20180713T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/07/14/met_analysis_1_0km_nordic_20180714T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/07/15/met_analysis_1_0km_nordic_20180715T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/07/16/met_analysis_1_0km_nordic_20180716T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/07/17/met_analysis_1_0km_nordic_20180717T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/07/23/met_analysis_1_0km_nordic_20180723T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/09/08/met_analysis_1_0km_nordic_20180908T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/09/26/met_analysis_1_0km_nordic_20180926T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/09/27/met_analysis_1_0km_nordic_20180927T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/10/02/met_analysis_1_0km_nordic_20181002T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/10/10/met_analysis_1_0km_nordic_20181010T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/10/11/met_analysis_1_0km_nordic_20181011T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/10/15/met_analysis_1_0km_nordic_20181015T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/10/16/met_analysis_1_0km_nordic_20181016T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/10/17/met_analysis_1_0km_nordic_20181017T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/10/18/met_analysis_1_0km_nordic_20181018T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/10/28/met_analysis_1_0km_nordic_20181028T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/02/met_analysis_1_0km_nordic_20181102T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/03/met_analysis_1_0km_nordic_20181103T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/08/met_analysis_1_0km_nordic_20181108T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/09/met_analysis_1_0km_nordic_20181109T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/10/met_analysis_1_0km_nordic_20181110T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/12/met_analysis_1_0km_nordic_20181112T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/13/met_analysis_1_0km_nordic_20181113T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/14/met_analysis_1_0km_nordic_20181114T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/15/met_analysis_1_0km_nordic_20181115T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/19/met_analysis_1_0km_nordic_20181119T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/20/met_analysis_1_0km_nordic_20181120T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/21/met_analysis_1_0km_nordic_20181121T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/22/met_analysis_1_0km_nordic_20181122T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/23/met_analysis_1_0km_nordic_20181123T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/24/met_analysis_1_0km_nordic_20181124T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/11/27/met_analysis_1_0km_nordic_20181127T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/12/04/met_analysis_1_0km_nordic_20181204T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/12/05/met_analysis_1_0km_nordic_20181205T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/12/07/met_analysis_1_0km_nordic_20181207T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/12/08/met_analysis_1_0km_nordic_20181208T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/12/09/met_analysis_1_0km_nordic_20181209T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/12/10/met_analysis_1_0km_nordic_20181210T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/12/11/met_analysis_1_0km_nordic_20181211T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/12/12/met_analysis_1_0km_nordic_20181212T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/12/13/met_analysis_1_0km_nordic_20181213T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/12/23/met_analysis_1_0km_nordic_20181223T00Z.nc
Read failed: 'air_pressure_at_sea_level'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/12/24/met_analysis_1_0km_nordic_20181224T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/12/25/met_analysis_1_0km_nordic_20181225T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2018/12/26/met_analysis_1_0km_nordic_20181226T00Z.nc
Read failed: [Errno -70] NetCDF: DAP server error: b'http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/01/05/met_analysis_1_0km_nordic_20190105T02Z.nc'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/01/05/met_analysis_1_0km_nordic_20190105T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/01/09/met_analysis_1_0km_nordic_20190109T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/01/13/met_analysis_1_0km_nordic_20190113T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/01/16/met_analysis_1_0km_nordic_20190116T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/01/24/met_analysis_1_0km_nordic_20190124T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/01/26/met_analysis_1_0km_nordic_20190126T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/01/31/met_analysis_1_0km_nordic_20190131T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/02/06/met_analysis_1_0km_nordic_20190206T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/02/08/met_analysis_1_0km_nordic_20190208T00Z.nc
Read failed: master dataset http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/02/12/met_analysis_1_0km_nordic_20190212T00Z.nc does not have a aggregation dimension
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/02/12/met_analysis_1_0km_nordic_20190212T00Z.nc
Read failed: master dataset http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/02/14/met_analysis_1_0km_nordic_20190214T00Z.nc does not have a aggregation dimension
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/02/14/met_analysis_1_0km_nordic_20190214T00Z.nc
Read failed: master dataset http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/02/16/met_analysis_1_0km_nordic_20190216T00Z.nc does not have a aggregation dimension
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/02/16/met_analysis_1_0km_nordic_20190216T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/02/23/met_analysis_1_0km_nordic_20190223T00Z.nc/home/jarroy/.local/lib/python3.7/site-packages/numpy/core/_methods.py:36: RuntimeWarning: overflow encountered in reduce
return umr_sum(a, axis, dtype, out, keepdims, initial)
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/02/26/met_analysis_1_0km_nordic_20190226T00Z.nc
Read failed: 'time'
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/03/01/met_analysis_1_0km_nordic_20190301T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/03/13/met_analysis_1_0km_nordic_20190313T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/04/03/met_analysis_1_0km_nordic_20190403T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/04/06/met_analysis_1_0km_nordic_20190406T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/04/20/met_analysis_1_0km_nordic_20190420T00Z.nc
Read failed: NetCDF: Not a valid ID
http://thredds.met.no/thredds/dodsC/metpparchivev1/2019/04/26/met_analysis_1_0km_nordic_20190426T00Z.nc
Missing variables in file MET Nordic analysis > Archive > Rerun_version_1 >
Dear All,
Are there many files missing in the archive of operational forecasts,
could not find at least these:
https://thredds.met.no/thredds/dodsC/metpparchive/2018/03/07/met_forecast_1_0km_nordic_20180307T15Z.nc
https://thredds.met.no/thredds/dodsC/metpparchive/2018/03/09/met_forecast_1_0km_nordic_20180309T16Z.nc
Terveisin, Markus
Dear Developers,
I download file https://thredds.met.no/thredds/dodsC/metpparchive/2022/06/15/met_forecast_1_0km_nordic_20220615T00Z.nc
And cut coordinates to Helsinki region
The other values below look ok, but the precipation amount (sorry its off in the copy paste below) at the first time stamp is 9.96921E+036
Whats going on? ( I read into xarray doing cut in projected coordinates and taking nanmean over spatial coordinates, then to pandas and to csv shown below)
Here is minimal example code to reproduce the result:
import xarray as xr
import pyproj
ds = xr.open_dataset(
"https://thredds.met.no/thredds/dodsC/metpparchive/2022/06/15/met_forecast_1_0km_nordic_20220615T00Z.nc"
)
proj = pyproj.Proj(ds.projection_lcc.proj4)
X, Y = proj(longitude=25.1, latitude=60.2)
dm = 1500
east = X + dm
west = X - dm
north = Y + dm
south = Y - dm
ds = ds.sel(x=slice(west, east), y=slice(south, north))
ds = ds.mean(dim=("x", "y"), skipna=True)
df = ds.to_dataframe()
df.to_csv("helsinki_20220615_00.csv")
Terveisin, Markus
time,projection_lcc,forecast_reference_time,altitude,land_area_fraction,air_temperature_2m,air_pressure_at_sea_level,relative_humidity_2m,cloud_area_fraction,wind_speed_10m,integral_of_surface_downwelling_longwave_flux_in_air_wrt_time,integral_of_surface_downwelling_shortwave_flux_in_air_wrt_time,precipitation_amount,wind_speed_of_gust,wind_direction_10m
2022-06-15 00:00:00,-2147483647.0,2022-06-15,5.0,0.7633333,287.91003,100622.59,0.7065686,0.9068469,5.9370203,7261794.0,172984.89,9.96921e+36,10.124577,322.83487
2022-06-15 01:00:00,-2147483647.0,2022-06-15,5.0,0.7633333,287.61575,100620.016,0.74328095,0.780403,5.9827876,8463072.0,172984.89,0.0,10.053935,322.14633
2022-06-15 02:00:00,-2147483647.0,2022-06-15,5.0,0.7633333,287.3064,100627.5,0.73399603,0.6613028,5.8526263,9633541.0,230882.69,0.0,10.06389,320.86154
2
I've found some periods with missing data in the latest rerun.
For 2019 there is one period of 30 hours missing from thredds:
For 2020 the files that I could not read are:
Was trying to run copy pasted example from metno examples:
import xarray
import numpy as np
opendap_url = "https://thredds.met.no/thredds/dodsC/aromearcticlatest/"+\
"archive/arome_arctic_det_2_5km_latest.nc"
#Ask for time=0 to make the calculation smaller, height0=0 to remove the 4th dimension
ds = xarray.open_dataset(opendap_url).isel(time=0,height0=0)
# Formula to calculate pressure from hybrid: p(n,k,j,i) = ap(k) + b(k)*ps(n,j,i)"
ap = ds.ap
b = ds.b
surface_pressure = ds.surface_air_pressure
# Note that k = 0 is top of atmosphere (ToA), and k = 64 is surface
pressure_at_k = (ap + (b*surface_pressure))
temperature_at_k = ds.air_temperature_ml
R = 287.058
g = 9.81
height_at_k = xarray.full_like(pressure_at_k, fill_value=0)
max_k = len(height_at_k.hybrid) -1
# Compute the height of the lowest model level
height_at_k.values[max_k,:,:] = (R*temperature_at_k.isel(hybrid=max_k))/g
height_at_k.values[max_k,:,:] *= np.log(surface_pressure / (pressure_at_k.isel(hybrid = max_k)))
# Loop over the rest of the model levels
for hybrid in range(max_k - 1,-1,-1):
height_at_k.values[hybrid,:,:] = (R*temperature_at_k.isel(hybrid=hybrid))/g
height_at_k.values[hybrid,:,:] *= np.log((pressure_at_k.isel(hybrid = hybrid + 1)) / \
(pressure_at_k.isel(hybrid = hybrid)))
height_at_k.values[hybrid,:,:] += height_at_k.isel(hybrid=hybrid + 1)
but got error
Error:curl error: Problem with the SSL CA cert (path? access rights?)
google says this is server side error due to
outdated SSL certificate(s) for cURL installed on the server.
terveisin, Markus
There are several parameters with accumulated values (accumulated total precipitation, accumulated rainfall at surface, etc.) in the deterministic MEPS control data files. Over which time frame is the accumulated values calculated? We are interested in rain intensity per hour, how do we relate the accumulated rainfall to rain intensity in mm/hr?
First off all, thanks alot for providing the MEPS data free of charge. It has been a great playground for me lately :)
I am a enthusiastic paraglider pilot, and have developed a small service (see https://flysogn.streamlit.app/ and select "forecast") that shows wind and thermal activity forecasted by METS so that we can use it for planning and expectations. This seems to work quite well.
However, after some investigations, I have realized that you output a quite sparse grid in your meps_lagged_6_h_vc_2_5km
and meps_det_vc_2_5km
files.
For example, in my local area (sogndal), the two closest points are 10 and 16 km away, respectively. This would of course be better if I got proper output of the 2.5km grid (I assume an interpolation here would not be as good).
Is there a possibility to add this as model outputs? Or am I thinking wrongly here? The way I load the forecast can be seen [here].(https://github.com/simeneide/flysogn/blob/main/meps.py#L21).
Secondly, is it possible to add lat lon dimensions to the dataset? The way I can filter down a position now is through the following code (which is slow):
subset = dataset.where(
(np.abs(dataset.latitude - lat) < tol) & (np.abs(dataset.longitude - lon) < tol),
drop=True
)
Are the parameters cloud_area_fraction and integral_of_surface_downwelling_shortwave_flux_in_air just the nearest neighbour of the raw model output at 2.5 km resolution? Or is there some post-processing or downscaling applied to the analysis and forecast products like nearest neighbourhood area selection? In the pdf about MEPS linked to on the MEPS AROME Arctic wiki page, it is written that post-processed cloud area fractions are the median value of a neighbourhood area of 37.5x37.5 km, but I notice that the information is a few years old and applied to the old pp products.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.