Failures in GSI analysis due to inconsistent correlated error about global-workflow

[CatherineThomas-NOAA]

FYI @JessicaMeixner-NOAA @guillaumevernieres @CoryMartin-NOAA

[RussTreadon-NOAA]

Removing the cris-fsr_n21 entry from the correlated_observations section of global_anavinfo.l127.txt works.

Another option is to leave the anavinfo file alone and move code around in src/gsi/correlated_obsmod.F90

diff --git a/src/gsi/correlated_obsmod.F90 b/src/gsi/correlated_obsmod.F90
index 7a14cd32..17cd94ef 100644
--- a/src/gsi/correlated_obsmod.F90
+++ b/src/gsi/correlated_obsmod.F90
@@ -961,14 +961,18 @@ subroutine upd_varch_
             enddo
             nchanl1=jc

-            if(nchanl1==0) call die(myname_,' improperly set GSI_BundleErrorCov')
             if(.not.amiset_(GSI_BundleErrorCov(itbl))) then
-               if (iamroot_) write(6,*) 'WARNING: Error Covariance not set for ',trim(idnames(itbl))
+               if (iamroot_) write(6,*) trim(myname_), ' WARNING: Error Covariance not set for ',trim(idnames(itbl))
                cycle read_tab
             endif

             nch_active=GSI_BundleErrorCov(itbl)%nch_active
-            if(nch_active<0) return
+            if(nch_active<0) then
+               if (iamroot_) write(6,*) trim(myname_), ' WARNING: No active channels for ',trim(idnames(itbl))
+               return
+            endif
+
+            if(nchanl1==0) call die(myname_,' improperly set GSI_BundleErrorCov')

             if(GMAO_ObsErrorCov)then
                do jj=1,nch_active

The nchanl1==0 line is the one causing gsi.x to abort.

For cris-fsr_n21, amiset_(GSI_BundleErrorCov(itbl)) = .false. and execution cycles to the next read_tab.

gsi.x built with the above change successfully ran the 2021032418 case. The above change is in /scratch1/NCEPDEV/da/Russ.Treadon/git/gsi/develop_test

GSI ctests were run using the modified gsi.x. Tests pass.

[RussTreadon-NOAA]

To build g-w with the revised GSI source code, open a g-w PR to update the gsi_enkf.fd hash to 457510c

[RussTreadon-NOAA]

The gsi_enkf.fd hash will be updated to 457510c in RussTreadon-NOAA/feature/gsihash.