noaa-emc / gsi Goto Github PK
View Code? Open in Web Editor NEWGridpoint Statistical Interpolation
License: GNU Lesser General Public License v3.0
Gridpoint Statistical Interpolation
License: GNU Lesser General Public License v3.0
In forked branch: https://github.com/CoryMartin-NOAA/GSI/tree/feature/gdaschgres
Currently, calcanl_gfs.py computes the full resolution analysis by interpolating the analysis increment. For the GDAS cycle, it runs enkf_chgres_recenter to regrid the GDAS forecast to the ensemble resolution for use in the recentering step. This regridding can be done after gdasfcst rather than during gdasanalcalc in order to limit the amount of time the workflow waits before the EnKF can start.
This change will also introduce changes to the existing python scripts to:
The recent changes in GSI EnKF IO for GFS (like the additional prefix for names of surface ensemble files, parallel netcdf reading/writing subroutines) are need to be "mirrored" in the IO modules for the regional applications including FV3-LAM.
This issue is only for changes in the IO interface on the FV3-LAM IO modules (gridio_fv3reg.f90) to allow the GSI EnKF for FV3-LAM could run without their original setups . The actual functions like IO for surface ensembles and parallel netcdf IO are to be considered in the future.
In setupt.f90, ges_z is used to check ges_ps and ges_th2m and ges_q2m allocation status. It causes GSI stop when ges_z is used in setupt.f90. The bug can be fixed by using ges_ps, ges_th2m and ges_q2m itself to check variable allocation status.
GSI failure case:
m_berror_stats_reg::berror_read_wgt_reg(PREWGT_REG): read error amplitudes "
berror_stats". mype,nsigstat,nlatstat = 0 42 192
setupt: ps already incorrectly alloc
STOP2 ABORTING EXECUTION w/code= 999
if (istatus==0) then
if(allocated(ges_z))then
write(6,*) trim(myname), ': ', trim(varname), ' already incorrectly alloc '
call stop2(999)
endif
allocate(ges_ps(size(rank2,1),size(rank2,2),nfldsig))
ges_ps(:,:,1)=rank2
do ifld=2,nfldsig
call gsi_bundlegetpointer(gsi_metguess_bundle(ifld),trim(varname),rank2,istatus)
ges_ps(:,:,ifld)=rank2
enddo
else
write(6,*) trim(myname),': ', trim(varname), ' not found in met bundle, ier= ',istatus
call stop2(999)
endif
The existing bufr libraries will cease to work after 2020 Dec 31, so the module loads must be updated.
Current GEFS is on its own branch, so any fixes already made to develop aren't applicable.
Discovered by accident that L127 global_gsi.x hangs when run with 1020 or more tasks. Subsequent debugging identified a problem in general_read_gfsatm_nc.
When the L127 global_gsi.x is run with 1020 or more tasks, logical variable procuse is .false. for 1 or more tasks. This is problematic. Only tasks for which procuse=.true. execute function open_dataset. general_read_gfsatm_nc executes open_dataset with paraopen=.true. (ie, parallel netcdf read). By default open_dataset assumes all tasks call the function. general_read_gfsatm_nc executes open_dataset with this default. Hence, the problem. open_dataset assumes all tasks execute the function but this is not true when running the L127 global_gsi.x with 1020 or more tasks.
Function open_dataset has an optional argument for a sub-communicator. Code added to general_read_gfstam_nc to construct a communicator including only the tasks for which procuse=.true. With this addition, L127 global_gsi.x runs to completion for tasks counts of 1020 or higher. This change does not alter analysis results.
This issue is opened to document this change.
#76692 on vlab (https://vlab.ncep.noaa.gov/redmine/issues/76692)
NCO implemented GFS.v15.3.2 on 6 July 2020 as described in the excerpt below taken from NCO RFC memo - July 2, 2020.
RFC 7036 – On WCOSS, implement GFS.v15.3.2 updates to the GSI fix file and global_satinfo.txt. This change is being made to address minimization issues in the GSI and tighten quality control for the seven CrIS water vapor channels. To be implemented on July 6 at 1400Z.
This issue is being opened to document the following:
Issue found while building GSI from the forked NOAA-EMC:master
I followed Kate's e-mail regarding the model update for python and cmake, and updated modules accordingly in GSI/modulefiles/modulefiles.ProdGSI.hera:
# python
module use -a /contrib/anaconda/modulefiles
module load anaconda/2.3.0
module use -a /contrib/cmake/modulefiles
module load cmake/3.9.0
During the build process, I ran into the following error message:
Configuring utility in adderrspec.fd
Configuring utility in adjustps.fd
Configuring utility in calc_increment_ens.fd
Configuring utility in calc_increment_serial.fd
Configuring utility in getnstensmeanp.fd
Configuring utility in getsfcensmeanp.fd
Configuring utility in getsfcnstensupdp.fd
Configuring utility in getsigensmeanp_smooth.fd
Configuring utility in getsigensstatp.fd
hey, incl dirs are /apps/intel/compilers_and_libraries_2018/linux/mpi/intel64/include/gfortran;/apps/intel/compilers_and_libraries_2018/linux/mpi/intel64/include
Configuring utility in gribmean.fd
Configuring utility in recenternemsiop_hybgain.fd
Configuring utility in recentersigp.fd
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
FRAMEMODULE
linked by target "global_gsi.x" in directory /scratch1/NCEPDEV/da/Emily.Liu/JEDI-GSI/GSI/src/gsi
FRAMEPACK
linked by target "global_gsi.x" in directory /scratch1/NCEPDEV/da/Emily.Liu/JEDI-GSI/GSI/src/gsi
IOINT_LIB
linked by target "global_gsi.x" in directory /scratch1/NCEPDEV/da/Emily.Liu/JEDI-GSI/GSI/src/gsi
WRFNETCDF_LIB
linked by target "global_gsi.x" in directory /scratch1/NCEPDEV/da/Emily.Liu/JEDI-GSI/GSI/src/gsi
-- Configuring incomplete, errors occurred!
I also checked out GSI master from gerrit:ProdGSI and modified the modulefiles as described above. The GSI build process completed successfully on HERA.
I am wondering if I missed something when I tried to build the GSI master checked out from GitHub?
This is used to generate an internal ensemble from randomly sampling the static B.
! generate_ens - if true, then generate internal ensemble based on existing background error
It was originally added to test the ensemble Var solver, but I would like to add the capability to write out a random sample of the static B to use as additive inflation in the EnKF. Apparently it hasn't been used in a while, and changes to general_sub2grid and/or ckgcov have broken this option. The traceback is below.
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
global_gsi 0000000001F3632D Unknown Unknown Unknown
libpthread-2.17.s 00002B7A759915D0 Unknown Unknown Unknown
libmpi.so.12.0 00002B7A74C4C2D2 Unknown Unknown Unknown
libmpi.so.12.0 00002B7A74A2981B Unknown Unknown Unknown
libmpi.so.12.0 00002B7A74C68810 Unknown Unknown Unknown
libmpi.so.12.0 00002B7A74C3DE50 Unknown Unknown Unknown
libmpi.so.12.0 00002B7A74B79A25 Unknown Unknown Unknown
libmpi.so.12 00002B7A749EB274 PMPI_Alltoallv Unknown Unknown
libmpifort.so.12. 00002B7A74595B21 pmpi_alltoallv_ Unknown Unknown
global_gsi 000000000050DD19 general_sub2grid_ 1627 general_sub2grid_mod.f90
global_gsi 00000000013AAEA0 ckgcov_ 157 bkgcov.f90
global_gsi 0000000000E7BFF3 hybrid_ensemble_i 1502 hybrid_ensemble_isotropic.F90
global_gsi 0000000000E7A5FA hybrid_ensemble_i 1245 hybrid_ensemble_isotropic.F90
global_gsi 0000000000E4F9C1 glbsoi_ 269 glbsoi.f90
global_gsi 000000000062A620 gsisub_ 201 gsisub.F90
global_gsi 000000000043AF7D gsimod_mp_gsimain 1840 gsimod.F90
global_gsi 000000000043AEB7 MAIN__ 631 gsimain.f90
global_gsi 000000000043AE5E Unknown Unknown Unknown
libc-2.17.so 00002B7A79407495 __libc_start_main Unknown Unknown
global_gsi 000000000043AD69 Unknown Unknown Unknown
This is a re-issue of Redmine ticket #71916:
The RadMon's summary and time series plots support displaying/hiding data from a single comparison source using a checkbox. When the user clicks the checkbox the comparison source toggles between shown (checked) and hidden (unchecked). The actual comparison source (i.e. the operational GDAS) is hard coded.
Recent changes were made to the javascript used to support the l127_gps, l127_rad, and l127_rad2 experiments to use a selection list to determine the source of the comparison data, retaining the checkbox to toggle the comparison on/off. This ticket will capture those changes, generalizing them and adding creation of the comparison list to the RadMon html generation script.
The current GSI can not handle the FV3LAM grid orientation W-E/S-N (SW corner is the first grid point).
The new code was added based on the current GSI FV3LAM interface to handle both grid orientation.
One new namelist option was added to make GSI work with two FV3LAM grid orientations.
grid_type_fv3_regional = 0 : decide grid orientation based on
grid_lat/grid_lon
1 : input is E-W N-S grid
2 : input is W-E S-N grid
Modify the Conventional data monitor (ConMon): * change the package level build script to use the cmake utility and the $target specific module files. * utilize the latest version of read_diag.f90 to enable support for NetCDF formatted cnvstat diagnostic files.
This issue began as VLab #63754. Development will continue in repository EdwardSafford-NOAA/GSI in branch esafford_ConMon_63754.
There are several coding standard issues with the routines in the GSI. Have been going through and correcting these issues within the code. Thus far, issues corrected have included:
This work was started in the ProdGSI repo, but was ported to NOAA-EMC/GSI. Have cleared through:
src/gsi/gsi_4dcouplermod.f90
This work can be found in:
https://github.com/MichaelLueken-NOAA/GSI/commits/feature/code_cleanup_2020
I have been attempting to generate the nc_diag files containing the GSI computed forward operator H(x); previous versions of the GSI have worked fine and are backwards compatible with the previous CRTM revision files (on RDHPCS Hera, /scratch1/NCEPDEV/global/glopara/crtm/) and the current (on RDHPCS Hera, /scratch2/NCEPDEV/nwprod/NCEPLIBS/fix/crtm_v2.3.0/).
I have tested the attempts to create the nc_diag files with two different executables (both built on RDHPCS Hera) from two different revisions:
/scratch2/BMC/gsienkf/whitaker/gsi/GSI-github-jswhit1/exec/global_gsi.x (works)
/scratch2/BMC/gsienkf/Henry.Winterbottom/UFS-RNR/exec/ufsrnr_gsi.x (fails)
My working directory on RDHPCS Hera is: /scratch2/BMC/gsienkf/Henry.Winterbottom/work/UFSRNR_DEMO/20191203060000/gsi/cntrl_hox
Within the above directory is a log file named gsi.log which contains the stdout from the failed run. I have also attached it.
The failed GSI experiments are failing in the setuprad.f90 routine suggesting (to me) that either the satellite bias-correction file is incorrect and/or there is a mismatch with the CRTM.
GSI stdout log: gsi.log
It was found that the code for the new BUFR encoding sequence 03-10-077 for VIIRS AMVs (type 260, subset NC005091), had a bug in the SatID range check. This lead to not setting itype properly and assimilating the VIIRS winds with inverse error 0, i.e. effectively not assimilating them (for the time since 5 Dec 2019).
The Satellite ID range check for winds with subset NC005091 should be 200-250, instead of 250-299, in read_satwnd.f90
The big fix was implemented and successfully tested:
feature/gfsda.v16.0.0_EUM_MetAVHRR_bufr
The branch with the VIIRS AMVs fix also: 1) adds code for new BUFR for EUMETSAT winds ; 2) adds code for new BUFR for NESDIS's Metop and AVHRR winds For more info on these changes, see issue #49 and issue #67
Add support for gpsro data into the existing format of the Conventional data monitor (ConMon). This is a reissue of Redmine issue 51902.
NESDIS is transitioning their Metop and AVHRR winds to Cloud computing, and they will be using a GOES-R like algorithm (similar to what is used for GOES-16&17) and the new BUFR encoding sequence 03-10-077.
Sample data for evaluation and testing is expected after late October.
We added code to reflect the new BUFR format in branch
feature/gfsda.v16.0.0_EUM_MetAVHRR_bufr
That branch also handles 1) EUMETSAT winds in new BUFR and 2) fix for VIIRS winds.
For more info on these changes, #49 and #66
The OznMon has not kept up with changes to the oznstat file structure.
Here's the results of an ncdump -h on an oznstat file from 2019091200, which was used as test data when writing the OznMon's read_diag.f90 file:
netcdf diag_gome_metop-a_ges.2019091200 {
dimensions:
nobs = UNLIMITED ; // (2905 currently)
variables:
int MPI_Task_Number(nobs) ;
float Latitude(nobs) ;
float Longitude(nobs) ;
float Time(nobs) ;
float Reference_Pressure(nobs) ;
int Analysis_Use_Flag(nobs) ;
float Observation(nobs) ;
float Inverse_Observation_Error(nobs) ;
float Obs_Minus_Forecast_adjusted(nobs) ;
float Obs_Minus_Forecast_unadjusted(nobs) ;
float Solar_Zenith_Angle(nobs) ;
float Scan_Position(nobs) ;
float Row_Anomaly_Index(nobs) ;
// global attributes:
:_NCProperties = "version=1|netcdflibversion=4.5.0|hdf5libversion=1.10.1" ;
:date_time = 2019091200 ;
:Number_of_state_vars = 1146 ;
}
Note the global attribute list. Now a dump from 2020070912:
netcdf diag_gome_metop-a_ges.2020070912 {
dimensions:
nobs = UNLIMITED ; // (2705 currently)
variables:
int MPI_Task_Number(nobs) ;
float Latitude(nobs) ;
float Longitude(nobs) ;
float Time(nobs) ;
float Reference_Pressure(nobs) ;
int Analysis_Use_Flag(nobs) ;
float Observation(nobs) ;
float Inverse_Observation_Error(nobs) ;
float Obs_Minus_Forecast_adjusted(nobs) ;
float Obs_Minus_Forecast_unadjusted(nobs) ;
float Solar_Zenith_Angle(nobs) ;
float Scan_Position(nobs) ;
float Row_Anomaly_Index(nobs) ;
// global attributes:
:date_time = 2020070912 ;
:Satellite_Sensor = "gome_metop-a" ;
:Satellite = "metop-a" ;
:Observation_type = "gome" ;
:pobs = 0. ;
:gross = 120. ;
:tnoise = 2.236 ;
}
This ticket will include adding the ability to read all of the current global attributes, removing those global attributes that no longer exist, and will evaluate if and how those global attributes should be used in the OznMon and appropriate implementation(s). Also the OznMon's read_diag.f90 module will be renamed to oznmon_read_diag.f90 to avoid confusion with similarly named modules.
This new issue is to include the capability to assimilate HDOBs in the V16 based on Xingren's HDOBs parallel experiments.
GSI code changes: read_fl_hdob.f90
gsiparm change: exglobal_analysis_fv3gfs.sh.ecf
global_convinfo.txt and error table changes are also applied.
This issue is documenting the research and development on evaluating Aeolus Doppler Wind Lidar wind profiles in the Global GFS model. Some aspects of this work is focusing on using Aeolus winds for TC/hurricanes characterization.
Relevant branches are:
gw.gfsv16b_aeolus_KA
and
gsi.fd_gfsda.v16.0.0_aeolus_varqc_KA_copy
This issue refs the HWRF GSI code changes for TDR and 88D thinning from Jason and OU.
This is to switch from assimilating satellite microwave antenna temperature observations to brightness temperatures.
As it is now, the bias between the observed antenna temperature and the simulated brightness temperature is being handled by the variational bias correction. For the platforms and feeds which have only antenna temperature, an accurate formula for converting antenna temperatures to brightness temperatures will be used. Doing this and assimilating brightness temperatures removes a burden of the variational bias correction.
Many (or all) other centers are assimilating brightness temperatures.
This branch gsi_fv3reg4coldstart contains recent fixes and development for FV3SAR GSI interface, including:
This issue tracks DA development for the GFSv16 implementation.
This github issue is a continuation of VLAB issue #65376 following the transition of the ProdGSI repository from VLAB to the NOAA-EMC/GSI repository in github.
As of 6/3/2020, GFS v16 DA development moved to github branch release/gfsda.v16.0.0.
LeoGeo AMVs are high-altitude winds retrieved from combined geostationary and polar-orbiting satellite imagery. They fill a data gap in AMVs global coverage, i.e. 50-70 N and S latitudinal bands.
Changes to GSI are needed in the following files to add the LeoGeo AMVs:
read_satwnd.f90
read_obs.F90
/fix/global_convinfo.txt
LeoGeo AMVs are assigned SatID 854
LeoGeo AMVs are assigned NCEP type 255
Branch used for LeoGeo evaluation: AMV_Genkova_parallel_ncio_LG
All regression tests passed:
(Hera) /scratch1/NCEPDEV/da/Iliana.Genkova/GSI/regression
Suggested reviewers: Jim Jung , Cathy Thomas, Xiujuan Su
In setupt. f90, ges_q2 and ges_th2 should be deallocated in subroutine final_vars_
This issue is to track efforts to introduce git-lfs to manage the fix files.
Forked GSI repo to my area: https://github.com/KateFriedman-NOAA/GSI
Created branch: https://github.com/KateFriedman-NOAA/GSI/tree/feature/git-lfs
added FV3GFS_NCIO_Fortran_FLAGS for GNU compilers
Moved gsdcloud to src/, make appropriate changes in CMakeLists.txt and a few bug fixes
pietc.f90 and pvqc_tables.f90
use kinds, only: dp,dpc
--> use kinds, only: r_kind, dp, dpc
This change is to avoid compiling error
error #6580: Name in only-list does not exist or is not accessible. [DP]
Don't know the exact reason, but either (1) removing all #ifdef block in kinds.F90 or (2) use r_kind first can solve this problem.
updated util/EnKF/arw/src/enspreproc_regional.fd/CMakeLists.txt
bug fixes for GNU compilers and regional EnKF.
updated ush/build.comgsi for community users
Hi. I'm trying to build GSI from the source contained in the release tarball https://github.com/NOAA-EMC/GSI/archive/gefs_v12.0.2.tar.gz. My build invocation looks like
cmake -DBUILD_CORELIBS=OFF -DBUILD_ENKF=OFF -DBUILD_GLOBAL=ON -DCMAKE_INSTALL_PREFIX=$PREFIX -DCMAKE_PREFIX_PATH=$PREFIX -DUSE_WRF=OFF ..
I have already built the required NCEP libraries (individually, each from their own git repo) and have them installed under $PREFIX
. The build makes some progress, but then gets to this:
Found BACIO library $PREFIX/lib/libbacio_4.a
Found BUFR library BUFR_LIBRARY-NOTFOUND
Could not find BUFR library, so building from libsrc
searching for source for bufr_v in
didn't find directory
WARNING: Did not find of bufr, looking for alternates
CMake Error at cmake/Modules/FindBUFR.cmake:44 (add_subdirectory):
add_subdirectory given source
"$SRCPREFIX/gsi/libsrc/bufr"
which is not an existing directory.
Call Stack (most recent call first):
CMakeLists.txt:226 (find_package)
find: ‘/sigio’: No such file or directory
find: ‘/sorc’: No such file or directory
CMake Error at cmake/Modules/findHelpers.cmake:137 (string):
string sub-command REGEX, mode MATCH needs at least 5 arguments total to
command.
Call Stack (most recent call first):
cmake/Modules/FindSIGIO.cmake:19 (findInc)
CMakeLists.txt:227 (find_package)
(I replaced some long paths there with $PREFIX
and $SRCPREFIX
for readability -- the values are correct, but shouldn't be interesting.)
I note in the output that libbacio
is found, but libbufr
isn't. However:
% cd $PREFIX/lib
% ls libbacio*
libbacio_4.a libbacio_8.a
% ls libbufr*
libbufr_4.a libbufr_4_DA.a libbufr_8.a libbufr_8_DA.a libbufr_d.a libbufr_d_DA.a
% ls libcrtm*
libcrtm.a
% ls libnemsio*
libnemsio.a
% ls libsigio*
libsigio.a
% ls libsfcio*
libsfcio.a
% ls libsp*
libsp_4.a libsp_8.a libsp_d.a
% ls libw3emc*
libw3emc_4.a libw3emc_8.a libw3emc_d.a
% ls libw3nco*
libw3nco_4.a libw3nco_8.a libw3nco_d.a
I also tried setting
export COREPATH=$PREFIX/lib
export BACIO_LIB=$PREFIX/lib/libbacio_4.a
export BUFR_LIB=$PREFIX/lib/libbufr_4.a
export CRTM_LIB=$PREFIX/lib/libcrtm.a
export NEMSIO_LIB=$PREFIX/lib/libnemsio.a
export SFCIO_LIB=$PREFIX/lib/libsfcio.a
export SIGIO_LIB=$PREFIX/lib/libsigio.a
export SP_LIB=$PREFIX/lib/libsp_4.a
export W3EMC_LIB=$PREFIX/lib/libw3emc_4.a
export W3NCO_LIB=$PREFIX/lib/libw3nco_4.a
but the build fails in the same way.
Maybe interestingly, if I set
export BACIO_LIB=$PREFIX/lib/libbacio_8.a
(i.e. libbacio_8.a
instead of libbacio_4.a
-- I'm guessing this is a kind=8 build of the library, whether or not that's what is actually desired for the GSI build), I still see
Found BACIO library $PREFIX/lib/libbacio_4.a
in the output of a new cmake
invocation, so that I'm not sure this environment variable is having any effect.
Can you offer any advice on what I might be doing wrong here? Thanks in advance!
@CoryMartin-NOAA I have cloned branch feature/files_for_jedi_aop20 from your feature/files_for_jedi.
I made some changes and additions in ffiles_for_jedi_aop20. Could you take a look at the changes in the branch? I will merge the changes back to feature/files_for_jedi once you give an OK.
The changes I made compare to feature/files_for_jedi can be found from the following link:
feature/files_for_jedi...emilyhcliu:feature/files_for_jedi_aop20
so berror file can be regenerated using GFSv16 parallel output (PR #52)
To secure the functionality of aerosol radiance impacts in GSI,
A new regression test, fv3aerorad, is added into the package.
Two fix files, fv3aerorad_anavinfo and fv3aerorad_satinfo.txt, are also included for the regression test.
This regression test is based on fv3aero which is for aerosol analysis.
Bugfix for reading external aerosol files in read_files.f90.
The capability is for GSI to read aerosol information from external files.
Before the fix, GSI can't count "aerf" files properly.
External reading for aerosol information is also utilized in "fv3aerorad" regression test.
The new aircraft observations from AMDAR, LATAM, TaMDAR and Canadian AMDar were assimilated in the GSI. The new VQC scheme was not applied the new aircraft observations. The O-A and O-B histograms from aircraft observations were studied. Different VQC parameters were evaluated for all aircraft observations.
Only global_convinfo.txt needs to be modified.
Under certain conditions, the EnKF will try to issue a diagnostic print that tries to access an unallocated array. This causes a segfault on Orion. Fix in PR #12 just removes the unallocated array from the print statement (does not change results).
GFS v15.3.2 was implemented in early July 2020 to address minimization issues in the GFS v15.3 GSI. GFS v15.3.2 adjusted the error limits for CrIS water vapor channels in fix file global_satinfo.txt. This helped improve v15.3 GSI minimization but EMC monitoring continues to detect minimization problems. The most recent abnormally terminated v15.3.2 minimization was 2020090300 gfs.
Experimentation, both single case and parallel testing (one month), have identified a simple change that improves GSI minimization. The change is to turn off variational quality control for aircraft temperature observations. This is achieved by setting the varqc parameters for five aircraft temperature observation types to 0.0 in fix file global_convinfo.txt. At the same time the gross check for these aircraft temperature observations are tightened.
This issue is opened to document the updating of global_convinfo.txt in release/gfsda.v15.3.3.
Initialized tropical cyclone Douglas in V15 GDAS was found to have stronger winds and lower center pressure depression than the observed. The only TCVITAL type of observations assimilated was SLP in GDAS. A few sensitivity tests were performed on the possible culprits; i.e. strong constraint , hybrid ensemble-var, and the position of the TC center in the first guess. It was found that the misalignment of the TC position in the first guess was the main reason for the erroneous initialization.
Two risk mitigation strategies were proposed and tested; Option 1: matching observed location to the first guess TC position, and Option 2: applying the forcing from option 1 at the observed TC location.
I'm using GSI (v3.7) as an observer operator in combination with ENKF (v1.3) to assimilate radiances from several sensors/satellites.
Doing some initial test I discovered that the ENKF (letkf version) wasn't incorporating any observation and traced back the error to the observation level pressure which was different from the level pressure defined at the GSI step.
I solved it turn it on the lwrite_peakwt
argument to true in the GSI namelist to save the correct observation level pressure in the diag files so the letkf can use that information later.
I didn't find any mention of this requirement in the documentation or the code but would be great to add it in a future version. Is there a way to contribute to that?
Thanks!
Several small issues with the RadMon have cropped up. None are major or significant so they will be combined into this ticket. Items include:
The following files have the same name and pose a problem on a case insensitive filesystem e.g. macOS.
Please consider renaming one or the other with a distinct name.
warning: the following paths have collided (e.g. case-sensitive paths
on a case-insensitive filesystem) and only one from the same
colliding group is in the working tree:
'util/Conventional_Monitor/image_gen/ush/Transfer.sh'
'util/Conventional_Monitor/image_gen/ush/transfer.sh'
Necessary changes to compile the GSI, EnKF, and utilities on Hera will be made to https://github.com/MichaelLueken-NOAA/GSI/tree/feature/hpc-stack-update.
This is a re-issue of Redmine ticket #76267 ( https://vlab.ncep.noaa.gov/redmine/issues/76267 ).
Original ticket description: I thought the conversion from theia to hera had been done long ago but it's not in ProdGSI/master. Perhaps I'm confusing that with the conversion to slurm job controls. Either way, the task is to port the RadMon to theia, making the appropriate /scratchX and module changes.
FV3LAMDA sometimes can't read NEXRAD bufr files due to checking subset status before reading data. This checking step is not necessary and should be turned off.
Before the fix:
For today's 12z LAMDA:
regional_gsianl_tm01_12.log: RADAR_BUFR_READ_ALL: problem opening level 2 bufr file "l2rwbufr"
regional_gsianl_tm02_12.log: RADAR_BUFR_READ_ALL: problem opening level 2 bufr file "l2rwbufr"
regional_gsianl_tm03_12.log: RADAR_BUFR_READ_ALL: problem opening level 2 bufr file "l2rwbufr"
regional_gsianl_tm04_12.log: RADAR_BUFR_READ_ALL: problem opening level 2 bufr file "l2rwbufr"
regional_gsianl_tm05_12.log: RADAR_BUFR_READ_ALL: problem opening level 2 bufr file "l2rwbufr"
File sizes for the 12z NAM nexrad file:
-rw-rw-r-- 1 nwprod prod 1218249536 Oct 21 13:15 nam.t12z.nexrad.tm00.bufr_d
-rw-rw-r-- 1 nwprod prod 829225392 Oct 21 12:18 nam.t12z.nexrad.tm01.bufr_d
-rw-rw-r-- 1 nwprod prod 951898336 Oct 21 11:23 nam.t12z.nexrad.tm02.bufr_d
-rw-rw-r-- 1 nwprod prod 951984000 Oct 21 10:30 nam.t12z.nexrad.tm03.bufr_d
-rw-rw-r-- 1 nwprod prod 1030995288 Oct 21 10:00 nam.t12z.nexrad.tm04.bufr_d
-rw-rw-r-- 1 nwprod prod 1072084336 Oct 21 10:00 nam.t12z.nexrad.tm05.bufr_d
-rw-rw-r-- 1 nwprod prod 1235644904 Oct 21 09:40 nam.t12z.nexrad.tm06.bufr_d
Conclusion: The smaller NAM nexrad files are the ones the code has problems reading. I recall seeing this before but this is the first time I was able to quantify it. It makes sense that the RAP files are giving the same error since they are always much smaller than the NAM ones due to the early data cutoff.
after the bug fix, gsi can read NEXRAD bufr files from both RAP and NAM dump:
[Shun.Liu@m110a2 log]$ grep RADAR_BUFR 12.log | grep num_radar
regional_gsianl_firstguess_tm06_12.log: RADAR_BUFR_READ_ALL: num_radars_0 = 107
regional_gsianl_tm00_12.log: RADAR_BUFR_READ_ALL: num_radars_0 = 78
regional_gsianl_tm01_12.log: RADAR_BUFR_READ_ALL: num_radars_0 = 92
regional_gsianl_tm02_12.log: RADAR_BUFR_READ_ALL: num_radars_0 = 91
regional_gsianl_tm03_12.log: RADAR_BUFR_READ_ALL: num_radars_0 = 93
regional_gsianl_tm04_12.log: RADAR_BUFR_READ_ALL: num_radars_0 = 98
regional_gsianl_tm05_12.log: RADAR_BUFR_READ_ALL: num_radars_0 = 107
regional_gsianl_tm**_12.log .0 is from NAM dump.
[Shun.Liu@m110a2 log]$ grep RADAR_BUFR 12.log.0 | grep num_radar
regional_gsianl_firstguess_tm06_12.log.0: RADAR_BUFR_READ_ALL: num_radars_0 = 106
regional_gsianl_tm00_12.log.0: RADAR_BUFR_READ_ALL: num_radars_0 = 86
regional_gsianl_tm01_12.log.0: RADAR_BUFR_READ_ALL: num_radars_0 = 92
regional_gsianl_tm02_12.log.0: RADAR_BUFR_READ_ALL: num_radars_0 = 95
regional_gsianl_tm03_12.log.0: RADAR_BUFR_READ_ALL: num_radars_0 = 106
regional_gsianl_tm04_12.log.0: RADAR_BUFR_READ_ALL: num_radars_0 = 107
regional_gsianl_tm05_12.log.0: RADAR_BUFR_READ_ALL: num_radars_0 = 109
The CMake methods for identifying both external packages (NetCDF, HDF5, etc.) and NCEPLIBS have progressed tremendously since they were first used with the GSI. The build system for the GSI needs to be updated to work with the new build system while also continuing to work with the module-based system used across NOAA, NCAR, and NASA.
Add use_gfs_ncio namelist option to the regional application HWRF, read GFS V16 netCDF files.
A new BUFR encoding sequence will be introduced in June 2020 for all EUMETSAT’s wind products. For full details and sample files, see the URL provided below. [Rev. 2]: Date changed to 7 October 2020. To facilitate user transition, AMV products encoded in the new BUFR sequence will be distributed on the GTS in parallel to the currently operational AMV products, from 15 July until 6 October 2020. For full details, see the URL provided below.
In order to update delz analysis for fv3 regional da, some new code development were added in the following subroutine. Request to merge those into the GSI master.
gridmod.F90
gsimod.F90
gsi_rfv3io_mod.f90
guess_grids.F90
pcgsoi.f90
read_guess.F90
There are several existing minor fixes that the RadMon package has needed, and some tweaks to the problem reporting are now needed as well. The full list is:
rm all old versions of gdas_radmon.v* and radmon_common.v*
rename gdas_radmon.v3.0.0 to gdas_radmon, rename radmon_common.v3.0.0 to radmon_common
rm known plotting inefficiencies
clean up warning messaging strategy and make extraction and copy functions output identical
modify criteria for low obs counts to reduce the number of warnings.
update install_glb.sh script to use correctly identify the last plot date.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.