hydrologicengineeringcenter / hec-dss

source code for HEC-DSS (Data Storage System)

This is the HEC-DSS library source tree. DSS is a database optimized for timeseries data.

Documentation: https://www.hec.usace.army.mil/software/hec-dss/documentation.aspx

Windows Dependencies

heclib.dll libifcoremdd.dll libifportMD.dll libmmdd.dll KERNEL32.dll VCRUNTIME140D.dll ucrtbased.dll

RedHat/Centos -- Install Build Dependencies

yum install -y
git
gcc
gcc-gfortran
make
zlib-devel
java-1.8.0-openjdk-devel
gcc-c++

Ubuntu -- Install Build Dependencies

sudo apt-get install build-essential
sudo apt-get install gfortran
sudo apt-get install zlib1g-dev
apt-get install default-jdk

It has been reported the Centos binaries worked in ubunto,after installing the following:

zlib, math, quadmath, and gfortran for pydsstools to work in Ubuntu.

To build and test heclib

Linux

1.	git clone -j2 --recurse-submodules https://github.com/HydrologicEngineeringCenter/hec-dss.git
2.	In the hec-dss/heclib directory, execute make clean ; make
3.	In the hec-dss/test/C directory, execute ./unix_test
4.	In the hec-dss/test/Dss-C directory, execute make clean ; make test
5.	In the hec-dss/test/Fortran directory, execute make clean ; make test

Solaris sparc sun4v

developerstudio12.6 #pkg contents system/library/fortran-runtime export PATH=$PATH:/opt/developerstudio12.6/bin:/usr/gcc/7/bin export JAVA_HOME=/usr/jdk/instances/jdk1.8.0 export LD_LIBRARY_PATH=/usr/gcc/7/lib/sparcv9

1.  bash
2.	git clone -j2 --recurse-submodules https://github.com/HydrologicEngineeringCenter/hec-dss.git
3.	In the hec-dss/heclib directory, execute gmake clean ; gmake
4.	In the hec-dss/test/C directory, execute ./unix_test
5.	In the hec-dss/test/Dss-C directory, execute gmake clean ; gmake test
6.	In the hec-dss/test/Fortran directory, execute gmake -f Makefile.Solaris clean ; gmake -f Makefile.Solaris test

Windows

build_release.bat 

running the java tests

TO DO

User Interface

The primary user interface for DSS files is HEC-DSSVue. https://www.hec.usace.army.mil/confluence/dssdocs/dssvueum/

hec-dss's Issues

magic number 392 should be removed (it is very close to #define MAX_PATHNAME_LENGTH 393)

The number 393 is used in both Fortran and C code in several places. Usually related to a maximum total length of a pathname.

Add ability to specify version of DSS file to be created.

One option is to add arguments (optional) to the constructor, or should we consider a static method Dss.Create("file.dss", version)?

CondensedPaths in dotnet behaving different for DSS6 and DSS7

JNI crash when copying Records

To reproduce: copy a record from attached DSS (version 6) file to another dss file(version 6).

If destination is DSS7 there is no issue.

The error occurs on a specific data set:
//STEVENSON/ELEV/01NOV2020/1HOUR/DCP-REV/
//STEVENSON/ELEV/01NOV2021/1HOUR/DCP-REV/

Other records with 'ELEV' copy ok, so it appears to be specific to that record?

hs_err_pid1468.log
forecast-src.zip

update workflow to checkout dss-test-data so github workflow will have the data

First step will be to get dss-test-data into a github repo
second step checkout dss-test-data (perhaps as git submodule?)

Paired Data Label not being stored.

I was running a test using hecdss.dll in VBA and have found an issue when saving paired data. The labels are not getting stored. I have checked the label bytes when reading paired data and the match the bytes when writing paired data and they match. However, the labels do not get stored to the output. I have attached my test workbook and the dss file I was testing with.
dss7_vba.zip

GetTimeSeriesProfile does not accept a date range

GetTimeSeriesProfile in DSSReader.cs should be able to accept a date range. Also I believe you should change the GetPathWithoutRange to GetPathWithoutDate inside GetTimeSeriesProfile.

memory issue(s) when converting from DSS version 7 to DSS Version 6

Items to note from initial investigation:

4:17:58.751 -----DSS--- zread Handle 3; Version 15: //JATO/FLOW-OBSERVED/01Jul2023/1Hour/FOR:Realtime-Snow:S0S0/
14:17:58.752
14:17:58.752
14:17:58.752 ***** DSS *** ERROR in function ztsStore: Cannot allocate memory, size requested: %d
14:17:58.752 Error occurred in HEC-DSS version 6, message: Copying time series VDI to user header

Branch DSS-178 (in this repo) added a method to the test program for debugging
https://github.com/HydrologicEngineeringCenter/hec-dss/tree/DSS-178

convert James_river.dss   output-v6.dss

The record //JATO/Location Info////
has this record. is supplementalInfo: (without any data) a valid tag?

supplementalInfo:verticalDatumInfo:H4sIAAAAAAAACrMpSy0qyUxOzNFNSSwpzdXNzEvLVyjNyyyxVUorUbLjUlCwyUssySxLhcjb+Yd4uAbZ6KOIgRTl5CPMyEvMTbUL9fP28w/3s9HHkOGy0cdiqR0DABIMJZeLAAAA;

The above was found using the Display->Raw Data menu of DSSVue

In verticalDatum.c:290 The code, below, is overwriting a \0 in some cases. We may need to add the condition [&& len < delimitedStringSize]

int len = strlen(*delimitedString);
    if (len > 0 && (*delimitedString)[len-1] != delimiter) {
		(*delimitedString)[len++] = delimiter;
    }



in ztsStore: (about line 352)  -- using tss->pathname  after call to zstructFree(tss);
The calling  program will free tss;  do we need to free memory here?

```C		if (status != STATUS_OKAY) {
			zstructFree(tss);
			return zerrorProcessing(ifltab, DSS_FUNCTION_ztsStore_ID,
				zdssErrorCodes.CANNOT_ALLOCATE_MEMORY, 0, 0,
				zdssErrorSeverity.MEMORY_ERROR,
				tss->pathname, "Copying time series VDI to user header");
		}

See also : DSS-178 - Java crash when converting record from v7 to v6. That includes the following Junit test.

    public void convert_7_to_6_crash_DSS_178() throws Exception{
        String filename7 = TestFiles.getFile("James_River.dss",true);
        String filename6 = TestFiles.getFile("James_River6",false);

        HecDSSUtilities u = new HecDSSUtilities();
        u.setDSSFileName(filename7);
        Vector<hec.heclib.dss.CondensedReference> v7Catalog = u.getCondensedCatalog();

        u.convertVersion(filename6);
        u.close();
        u.setDSSFileName(filename6);
        Vector<hec.heclib.dss.CondensedReference> v6Catalog = u.getCondensedCatalog();

        assertEquals(v7Catalog.size()-2,v6Catalog.size());// drop location info in DSS6
    }

James_River.zip

fortran interface heclib.dll having linker issue

C:\project\hec-dss\heclib\VB_interface\x64\Debug\heclib.dll : fatal error LNK1169: one or more multiply defined symbols found

Investigation

source	function name	obj	dumpbin
zfname6.f	zfname	zfname6.obj	zfname_
zfname.c	zfname	zfname.obj	zfname
zfname_.c	zfname_	zfname_.obj	zfname_

Stack smashing detected

OS: Linux/Ubuntu

JRE: openjdk 8

package hec.heclib.dss;

import hec.heclib.TestFiles;
import hec.heclib.util.Heclib;
import org.junit.jupiter.api.Test;
import hec.heclib.dss.*;

public class StackSmashTest

{    @Test public void test_stack_smash_detected() throws Exception {
 smash(); 
 }
 private static void smash() throws Exception{ // 

HecDSSFileAccess.zsetMessageLevel(HecDataManager.MESS_METHOD_GLOBAL, HecDataManager.MESS_LEVEL_USER_DIAG); 

HecDSSFileAccess.zsetMessageLevel(HecDataManager.MESS_METHOD_GLOBAL, HecDataManager.MESS_LEVEL_INTERNAL_DIAG_2);
 String destFileName = TestFiles.getFile("output_dss_stack_smash.dss",false);
HecDSSUtilities u = new HecDSSUtilities(); 
u.setDSSFileName(destFileName,6);
u.open();  
HecDSSUtilities f = new HecDSSUtilities(); 
String fileInput = TestFiles.getFile("containsGrids7.dss",true); // existing file 
f.setDSSFileName(fileInput); f.copyFile(destFileName); 
u.close();  
}
}

PathAssistFilterByParts is Case Sensitive

https://github.com/HydrologicEngineeringCenter/hec-dss/blob/master/dotnet/Hec.Dss/PathAssist.cs#L49

Can we add the Regex Ignore Case option here. I'm finding that we are not returning records if the target path case does not match the case from the catalog.

Memory Access violation

Revisiting use of stringCopy(...) in hecdss.c

in hecdss.c every call such as:

 stringCopy(type, typeLength, tss->type, strlen(tss->type));

is also calling strlen on something that could be null.

java crash on JNI call to Hec_zinquire

Stack: [0x0000000026280000,0x0000000026380000], sp=0x000000002637f3d0, free space=1020k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j hec.heclib.util.Heclib.Hec_zinquire([ILjava/lang/String;)J+0
j hec.heclib.util.Heclib.zinquire([ILjava/lang/String;)J+6
j hec.heclib.dss.HecDssCatalog$2.run()V+90
v ~StubRoutines::call_stub

collection list has corrupted DSS paths

The JNI layer is not properly handling the character ending, after passing a string to Fortran. (DSS 6 only).

example of corrupted path returned.

"/BALD EAGLE CREEK FRA/RESSIM-STDOPNODIV/PROBABILITY-ELEV/FREQUENCY/SAYERS-POOL ELEV 1HOUR/C:000000|EXISTING C:500-YEAR:RESSIM-STDOPNODIV/ ÌÌÌÌÌÌÌÌÌÌÌÌÌÌ?Cä�"

remove StreamRatings and partners (data types not currently used) Not in DSS7

DATA IRTYPE(18) /500/, CRTYPE(18) /'RGN'/
DATA IRTYPE(19) /501/, CRTYPE(19) /'RGI'/
DATA IRTYPE(20) /505/, CRTYPE(20) /'RND'/
DATA IRTYPE(21) /510/, CRTYPE(21) /'RSM'/
DATA IRTYPE(22) /511/, CRTYPE(22) /'RSI'/
DATA IRTYPE(23) /515/, CRTYPE(23) /'RMD'/
DATA IRTYPE(24) /520/, CRTYPE(24) /'RCS'/
DATA IRTYPE(25) /521/, CRTYPE(25) /'RCI'/
DATA IRTYPE(26) /525/, CRTYPE(26) /'RSD'/
DATA IRTYPE(27) /530/, CRTYPE(27) /'RCP'/
DATA IRTYPE(28) /531/, CRTYPE(28) /'RPI'/
DATA IRTYPE(29) /535/, CRTYPE(29) /'RPD'/
DATA IRTYPE(30) /540/, CRTYPE(30) /'RPR'/
DATA IRTYPE(31) /541/, CRTYPE(31) /'RRI'/
DATA IRTYPE(32) /545/, CRTYPE(32) /'RRD'/
DATA IRTYPE(33) /550/, CRTYPE(33) /'RPV'/
DATA IRTYPE(34) /551/, CRTYPE(34) /'RVI'/
DATA IRTYPE(35) /555/, CRTYPE(35) /'RVD'/
DATA IRTYPE(36) /560/, CRTYPE(36) /'RST'/
DATA IRTYPE(37) /561/, CRTYPE(37) /'RTI'/
DATA IRTYPE(38) /565/, CRTYPE(38) /'RT2'/
DATA IRTYPE(39) /570/, CRTYPE(39) /'RCN'/
DATA IRTYPE(40) /571/, CRTYPE(40) /'RCF'/
DATA IRTYPE(41) /575/, CRTYPE(41) /'RN2'/
DATA NRTYPE /41/

conversion of Text Records from DSS version 6 to 7 is not working.

VBA throws Overflow error when writing doubles

There is an issue writing regular time series as doubles, VBA throws an Overflow error. Using the hec_dss_tsStoreRegular call, I set the saveAsFloat argument to 0 and an Overflow error is thrown. If I set the saveAsFloat to 1, the time series stores without a problem. Please let me know if there is workaround.

DSS shared library .dll/.so API for both Java and .net consumers.

Thinking about #31

Any thoughts if we should attempt to make a new dll/so called '[lib]dss.so / dss.dll '

Can that dss.so be used by both JNA and cross platform C# P/Invoke?

The new shared library would have the minimal and (nicer names) interface to DSS

Importing Shef data crashes an application (DssVue) when using Java 11

The crash below does not occur with Java 8.

A fatal error has been detected by the Java Runtime Environment:

EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00000001800614d5, pid=7656, tid=23608

Stack: [0x0000001600a00000,0x0000001600b00000], sp=0x0000001600af80b0, free space=992k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [javaHeclib.dll+0x614d5]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j hec.heclib.dss.Sheflib.Hec_RunShefDss([IILjava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;I[I[I[I)V+0

MacOS build

Compiled DLL Request

My apologies as this is not a bug report, but I am trying to get access to the compiled version of the hec-dss library for 64-bit windows to use with the HEC-DSS python library. The USACE website currently gives an error message for the download link to the binary files. I tried compiling myself, but the Fortran compilation is running into issues. Is there a way to get the latest compiled dll?

remove _LARGEFILE64_SOURCE

_LARGEFILE64_SOURCE is a "transition interface" we should try to remove and only support 64 bit.

https://www.gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html

can we retire files under hec-dss\heclib\heclib_f\src\Deprecated ?

To test -- exclude these from the build and see what happens

Solaris JNI Crash SIGSEGV

to reproduce with Java code

./gradlew test --tests hec.heclib.dss.*

hs_err_pid21174.log

Errors with Elevation data in DSS version 6 files, under specific circumstances.

DSS Version: 6-YO Library 7-IO
DSS is crashing (hec-HMS) or giving errors with Elevation data in DSS version 6 files, under specific circumstances.

forrtl: error (63): output conversion error, unit 6, file CONOUT$
Image              PC                Routine            Line        Source             
javaHeclib.dll     000000018017FDBD  Unknown               Unknown  Unknown
javaHeclib.dll     0000000180167FA0  Unknown               Unknown  Unknown
javaHeclib.dll     00000001801114AD  get_user_header_p          38  get_user_header_param.f
javaHeclib.dll     00000001800D48EA  zsrtsi6_                  281  zsrtsi6.f
javaHeclib.dll     00000001800E078A  zsrtsc6_                   50  zsrtsc6.f
javaHeclib.dll     000000018005D9EB  zsrtsc_                    23  zsrtscInterface.f
javaHeclib.dll     000000018028D268  Unknown               Unknown  Unknown
                   00000000034F97DB  Unknown               Unknown  Unknown
                   00000000033E71F8  Unknown     

possible error in stringToUserHeader(..)

to reproduce, enable AddressSanitizer in Dss-C and heclib_c projects.
run DSS-C test program.
error happing on memcpy(..)

int *stringToUserHeader(const char *str, int *userHeaderNumber)

str = "verticalDatumInfo:H4sIAAAAAAAACmWPwQoCIRRFW/sV4t6xJgwDRwiCdi3bSz1BcBRmnn5/mlREy3fu5VyeLrCgv9vAHxbzzH10iebocWIOmSGU6mjRF+i5uV5uZz4etfihrQYBSmUpmsNODqMWX9Di5NwKSGFFP1uEieGS4TVQQ0xv/anqldLiQ3qh2JDBbIe9klKLfjWr6FpDKvx/xJDNE5yqeqDgAAAA;verticalDatum:NGVD-29;"

userHeaderNumber = 58

test vertical datum operations (C API)
zset returned -1
Problem getting information: No such file or directory
File C:\project\hec-dss\test\bin\v6_c.dss not found.
    -----DSS---ZOPEN:  New File Opened,  File: v6_c.dss
                       Unit:    3;  DSS Version: 6-YO  Library 7-IO
Time series test   1: expecting SUCESS
Time series test   2: expecting SUCESS
Time series test   3: expecting SUCESS
Time series test   4: expecting SUCESS
Time series test   5: expecting SUCESS
Time series test   6: expecting SUCESS
Time series test   7: expecting SUCESS
Time series test   8: expecting SUCESS
=================================================================
==27316==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x1177e6b1bbfe at pc 0x7ffd94012fb1 bp 0x001525afd790 sp 0x001525afd7a8
READ of size 256 at 0x1177e6b1bbfe thread T0
==27316==WARNING: Failed to use and restart external symbolizer!
    #0 0x7ffd94012fe1 in _asan_wrap_GlobalSize+0x402db (C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.30.30705\bin\HostX64\x64\clang_rt.asan_dbg_dynamic-x86_64.dll+0x180042fe1)
    #1 0x7ff7d9deb1d5 in stringToUserHeader C:\project\hec-dss\heclib\heclib_c\src\Utilities\verticalDatum.c:273
    #2 0x7ff7d9d69ad5 in testStoreRetrieveTimeSeries C:\project\hec-dss\test\Dss-C\source\testVerticalDatum_c.c:508
    #3 0x7ff7d9d6c6ed in test_vertical_datums_c C:\project\hec-dss\test\Dss-C\source\testVerticalDatum_c.c:20
    #4 0x7ff7d9d72c26 in runTheTests C:\project\hec-dss\test\Dss-C\TestDssC.c:162
    #5 0x7ff7d9d73bf8 in main C:\project\hec-dss\test\Dss-C\TestDssC.c:76

Transition from C++/CLI to P/Invoke

what is the future of : heclib_c\src\Utilities\catsort.c

This file has several compiler warnings. It is not clear if this file is being used?
There is a Fortran wrapper catsort_(...)

Version of gridded data

Hi,
I am playing a bit with gridded data in DSS files. I am trying to access to some gridded data, but I am stuck at the level of gridStruct->_structVersion that has to be equal to VERSION_100 to be able to be read, see here:

hec-dss/heclib/heclib_c/src/Public/zspatialGridRetrieve.c

Line 245 in 689d3ca

if (gridStruct->_structVersion == VERSION_100) {

I am wondering what is defining this version records in HEC world.

Thanks.

Fortran Runtime Error - makes opening DSS file take 10 minutes in CAVI

Increase the F part of the DSS pathname

Fortran runtime error: Recursive call to nonrecursive procedure 'zset'

Occurring on Ubuntu : GNU Fortran (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0

Not seeing this with Centos : GNU Fortran (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3)

At line 1 of file ../DssInterface/v6and7/Fortran/zsetInterface.f Fortran runtime error: Recursive call to nonrecursive procedure 'zset'

After commenting out lots of code to be sure zset was not recursive, a proposed fix will be compiler option change

from
-fcheck=all

To

-fcheck=all,no-recursion

Could not load file or assembly 'Hec.Dss.Native, Version=1.1.7906.19918, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.

Could not load file or assembly 'Hec.Dss.Native, Version=1.1.7906.19918, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.

When using ubuntu-latest the build is failing.

The latest ubuntu in github actions has newer gfortran that is producing warnings/errors.

To reproduce use:

• GNU Fortran (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
• cd test/Fortran ; make test

We need to update the Fortran code to fix these errors, as a separate branch/PR, then we could update the build back to ubuntu-latest

Here is the commit that fixed the build:
a45ca6d

DSS grid, and origin considered by HEC-RAS

Hi,

I've done some tries to construct some DSS grid data to bring them to HEC-RAS.

I am wondering about _xCoordOfGridCellZero and _yCoordOfGridCellZero parameters of the zStructSpatialGrid struct.
Tell me if I am wrong, but it seems that HEC-RAC handles only grid with these parameters set to 0, and for this reason if I have a grid that have not its origin exactly proportional to the cell size, (_lowerLeftCellX x _cellSize , _lowerLeftCellY x _cellSize) in real world coordinate, my grid could not be perfectly well positioned.

For example, when I have a grid with a bottom left origin with real world coordinate (123.45 , 56,78) and a cell size of 1.0, I will have my DSS grid origin set to (_lowerLeftCellX = 123 , _lowerLeftCellY= 57), because I have to start from (0,0). So, in real world coordinate, my dss grid orgin will be (123.00 , 57.00) instead of the real origin.

Am I wrong? May be I missed something with HEC-RAS?

EXCEPTION_ACCESS_VIOLATION when running Java Application

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  hec.heclib.util.Heclib.Hec_zsitsc([ILjava/lang/String;[I[F[DIII[IILjava/lang/String;Ljava/lang/String;[DI[IILjava/lang/String;ILjava/lang/String;I[I)V+0
j  hec.heclib.util.Heclib.zsitsc([ILjava/lang/String;[I[F[DIII[IILjava/lang/String;Ljava/lang/String;[DI[IILjava/lang/String;ILjava/lang/String;I[I)V+38
j  hec.heclib.dss.HecTimeSeries.flushToDisk()I+1321
j  hec.heclib.dss.HecTimeSeries.write(Z)I+17
j  hec.heclib.dss.HecTimeSeries.write(Lhec/io/TimeSeriesContainer;)I+710
j  com.rma.io.DssFileManagerImpl.writeTS(Lhec/io/TimeSeriesContainer;Lhec/io/StoreOption;)I+60
j  hec2.rts.io.DssFileManagerTimeSeriesDao.writeMixedType(Lhec/dataexchange/services/io/MappedDataExchange;Lhec/dataexchange/services/io/TimeSeriesAdapter;Lhec/dataexchange/services/data/MappedData;Lhec/dataexchange/services/data/DataStore;Lhec/dataexchange/services/data/MappedData;Lhec/io/StoreOption;)V+1137
j  hec2.rts.io.DssFileManagerTimeSeriesDao.writeData(Lhec/dataexchange/services/io/DataAdapter;Ljava/util/List;Lhec/dataexchange/services/data/MaxInterpolate;Lhec/dataexchange/services/data/DataStore;Lhec/dataexchange/services/data/DataStore;ZLhec/dataexchange/services/io/DataExchangeStatisticsModel;)V+341
j  hec.dataexchange.services.engine.BatchProcess.call()Lhec/dataexchange/services/io/DataExchangeStatisticsModel;+746
j  hec.dataexchange.model.DxForecastCallable.call()Ljava/util/Map;+308
j  hec2.rts.dataexchange.ExchangeUtil.lambda$runExtractAction$0(Lusace/metrics/services/Metrics;Lhec/dataexchange/model/DxForecastCallable;)Ljava/util/Map;+30
j  hec2.rts.dataexchange.ExchangeUtil$$Lambda$228.call()Ljava/lang/Object;+8
j  java.util.concurrent.FutureTask.run()V+42
j  java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub

VM Arguments:
jvm_args: -Dexe.mainConfig=C:\APP\CWMS\CAVI-3.3.0_QA_Beta_27\CAVI\CAVI.config -Xmx2048m -Djava.library.path=../shared/lib -DLooseVersionCheck=true -DDisable.DbfFileHandling=true -Xverify:none -DLOGFILE=C:/Users/q0hecajd/AppData/Local/Temp/3/logs/cavi.log -DLogFileHasPid=true -DCWMS_HOME=. -DNoDxXsdValidation=true -DPLUGINS=../shared/jar/ext;jar/ext -Djavax.net.ssl.cwms.trustStore=../shared/config/CWMSServerCert.ts -Dhec.data.cwmsRating.RatingSet.databaseLoadMethod=eager -DTeaming.CwmsServers=true -DUSE_TABBED -DCOLLECTION -Doracle.metrics.clientid=CWMS CAVI Connection -Dhttp.agent=Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0) -Dcavi.netbeans.platformdir=./module/Dashboard -Dscripts.directory=C:/Users/q0hecajd/AppData/Roaming/HEC/HEC-DSSVue/scripts -Djava.security.policy=../shared/config/java.policy -Djava.rmi.server.hostname=127.0.0.1 -Dcom.sun.media.jai.disableMediaLib=true -DhideFrm=true -Djna.tmpdir=C:/Temp/cwms/temp -DUseEmbeddedDssVue=true -Dsun.rmi.server.exceptionTrace=true -Djava.rmi.server.hostname=127.0.0.1 -Doracle.caching.openallconnections=true -Dsun.java2d.d3d=false -Dsun.java2d.nodraw=true -Dcom.sun.media.jai.disableMediaLib=true -DschedulerEntries.directory=C:/Users/q0hecajd/AppData/Roaming/CWMS/CAVI/scheduler -Dprism.order=sw -Xms1024M -Xmx12000M -DLogFileHasPid=true
java_command: usace.cavi.client.CAVI
java_class_path (initial): classes;../HEC-HMS/lib/hms-command-1.1.jar;../shared/jar/callback-services-1.1-20210621.140919-1.jar;../shared/jar/context-specific-help-1.1.0.jar;../shared/jar/cwms-data-exchange-4.1.1.jar;../shared/jar/cwms-data-exchange-engine-4.1.1.jar;../shared/jar/cwms-data-status-summary-3.3.1.jar;../shared/jar/cwms-db-aspects-9.3.4.jar;../shared/jar/cwms-db-dao-9.3.4.jar;../shared/jar/cwms-db-jooq-9.3.4.jar;../shared/jar/cwms-db-jooq-aspect-9.3.4.jar;../shared/jar/cwms-db-jooq-codegen-7.0.0-SNAPSHOT.jar;../shared/jar/cwms-dbjob-controls-3.3.0.jar;../shared/jar/cwms-http-client-3.1.1.jar;../shared/jar/cwms-radar-client-3.1.1.jar;../shared/jar/cwms-radar-model-3.1.1.jar;../shared/jar/cwms-regi-tools-bridge-3.3.1-rc2.jar;../shared/jar/cwms-regi-tools-service-3.3.1-rc2.jar;../shared/jar/cwms-server-admin-1.0.0-rc5.jar;../shared/jar/cwms-server-admin-plugin-1.0.0-rc5.jar;../shared/jar/data-exchange-factories-4.1.1.jar;../shared/jar/data-exchange-services-4.1.1.jar;../shared/jar/data-exchange-ui-4.1.1.jar;../shared/jar/data-exchange-xml-4.1.1.jar;../shared/jar/dssgui-3.3.7.jar;../shared/jar/fia-cwms-plugin-3.2.2.jar;../shared/jar/global-context-provider-1.0.jar;../shared/jar/hec-client-app-4.0.0-rc1.jar;../shared/jar/hec-cwms-ratings-core-1.1.0.jar;../shared/jar/hec-cwms-ratings-gfx-1.1.0.jar;../shared/jar/hec-cwmsvue-3.3.1-rc2.jar;../shared/jar/hec-dashboard-api-dev-20210809.174218-2.jar;../shared/jar/hec-dashboard-launcher-1.0.jar;../shared/jar/hec-data-ui-4.0.0-rc6.jar;../shared/jar/hec-db-core-9.3.4.jar;../shared/jar/hec-db-jdbc-9.3.4.jar;../shared/jar/hec-db-radar-9.3.4.jar;../shared/jar/hec-geojson-1.0.jar;../shared/jar/hec-gt-crs-20.0.0.jar;../shared/jar/hec-gt-geotiff-20.0.0.jar;../shared/jar/hec-isocline-g2d-plot-1.0.0.jar;../shared/jar/hec-isocline-paint-1.0.0.jar;../shared/jar/hec-monolith-3.0.0-rc7.jar;../shared/jar/hec-nucleus-data-1.1.0.jar;../shared/jar/hec-nucleus-metadata-1.1.0.jar;../shared/jar/hec-osmmap-3.0.0.jar;../shared/jar/hec-r
Launcher Type: SUN_STANDARD

Compilation error

When trying to compile under ubuntu 22.04 with GNU Fortran 11.2.0, I have the following error:

putshf.f:227:32:

   49 |         CALL J2SDT ( J,M,CSDT,icen)
      |                            2    
......
  227 |                  call j2sdt(jul,itime,cbuff(idycode+4:idycode+13),i)
      |                                1
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (scalar and rank-1)
putshf.f:230:34:

   49 |         CALL J2SDT ( J,M,CSDT,icen)
      |                            2      
......
  230 |                  call j2sdt (jul, itime, cbuff(idycode+2:idycode+11),i)
      |                                  1
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (scalar and rank-1)

improve workflow/Actions

The current workflow only builds on ubuntu.

The new workflow will include running tests, and include centos

Fortran substring runtime errors: forrtl: severe (408): fort: (12):

Summary: When running HMS test suite a Fortran runtime error causes a failure.

It has been difficult to reproduce unless the full test suite is run.

The following errors are from different program runs, The difference is different DSS_JNI_MESSAGE_LEVEL values.

forrtl: severe (408): fort: (12): Variable CLINE has substring ending point 5 which is greater than the variable length of 4

forrtl: severe (408): fort: (12): Variable CLINE has substring ending point 65 which is greater than the variable length of 64

forrtl: severe (408): fort: (12): Variable CLINE has substring ending point 115 which is greater than the variable length of 4

DSS_JNI_MESSAGE_LEVEL=4

forrtl: error (63): output conversion error, unit 15, file C:\project\hms-testing-utils\hms-benchmark-runner\build\resources\test\Bench11_3.0\Bench11_3.0.out

Stack trace terminated abnormally.
forrtl: severe (408): fort: (12): Variable CLINE has substring ending point 5 which is greater than the variable length of 4

Stack trace terminated abnormally.
forrtl: The handle is invalid.
forrtl: severe (38): error during write, unit 15, file C:\project\hms-testing-utils\hms-benchmark-runner\build\resources\test\Bench11_3.0\Bench11_3.0.out
forrtl: severe (28): CLOSE error, unit 15, file "Unknown"

DSS_JNI_MESSAGE_LEVEL=5

lots of debug output starts going to console instead of output file.

DSS_JNI_MESSAGE_LEVEL=6

forrtl: severe (408): fort: (12): Variable CLINE has substring ending point 5 which is greater than the variable length of 4

DSS_JNI_MESSAGE_LEVEL=7

one run Ok. not consistent.
forrtl: severe (408): fort: (12): Variable CLINE has substring ending point 5 which is greater than the variable length of 4

Stack trace terminated abnormally.
forrtl: The handle is invalid.
forrtl: severe (38): error during write, unit 15, file C:\project\hms-testing-utils\hms-benchmark-runner\build\resources\test\Bench11_3.0\Bench11_3.0.out
forrtl: severe (28): CLOSE error, unit 15, file "Unknown"

DSS_JNI_MESSAGE_LEVEL=10

forrtl: severe (408): fort: (12): Variable CLINE has substring ending point 5 which is greater than the variable length of 4

DSS_JNI_MESSAGE_LEVEL=15

not Consistend. One run appeared OK. However, output switched to console (last of file log below)

-----DSS---Debug: Enter zset6; Flag: MUNI, Numb: 6, String:
09:36:08.098 =====DSS===Debug: zset; Entering
09:36:08.098 =====DSS===Debug: zset; Parameter: MUNIT
09:36:08.098 =====DSS===Debug: zset; character val:
09:36:08.098 =====DSS===Debug: zset; integer val: 6

--- another failing run ---
forrtl: severe (408): fort: (12): Variable CLINE has substring ending point 5 which is greater than the variable length of 4

Stack trace terminated abnormally.
forrtl: The handle is invalid.
forrtl: severe (38): error during write, unit 15, file C:\project\hms-testing-utils\hms-benchmark-runner\build\resources\test\Bench11_3.0\Bench11_3.0.out
forrtl: severe (28): CLOSE error, unit 15, file "Unknown"

Mark Constants as depricated?

This class has

public const double UndefinedDouble = -9999.0;

However, dss considers undefined as shown in DssReader:

private const double UNDEFINED_DOUBLE = -3.402823466e+38;
private const double UNDEFINED_DOUBLE_2 = -3.4028234663852886E+38;

investigate ways to find memory leaks in the JNI code (javaHeclib.dll)

Recently we have found some JNI memory leaks in the grid retrieval code.

on windows this code is a possibility

#define _CRTDBG_MAP_ALLOC
#include <stdlib.h>
#include <crtdbg.h>

intermittent issue in Catalog test

14:52:42.087 -----DSS--- zwrite Handle 3; Version 23: /MISSISSIPPI/ST. LOUIS/FLOW/01Jan1863/1Day/OBS/
Error - catalog length should be 10, but it is 0
14:52:42.089 Record NOT found; Fail in testCatalog Loc 14, zcatalog not 10 records long

Status = -1

strcasestr -- is causing a conflict on macOS

hec-dss/heclib/heclib_c/src/Utilities/verticalDatum.c

Line 151 in 4bc6ee0

const char *strcasestr(const char *haystack, const char *needle) {

Could this be renamed to something like hec_dss_strcasestr ?

zcollectionCat(..) is defined twice

zcollectionCat(long long *ifltab, const char *seedPathname, zStructCatalog *catStruct)

is defined in two places:

1. heclib_c\src\Public\zcollectionCat.c
2. heclib_c\src\DssInterface\v6and7\zcollectionCat.c

They have slightly different implementations.

_gfortran_os_error_at

I tried to compile hec-dss, but this error appears:

../heclib_f/Output/libhec_f.a(ymddat.f.o): In function ymddat_':
/users/user/Downloads/hec-dss/heclib/heclib_f/src/Support/ymddat.f:158: undefined reference to _gfortran_os_error_at' /users/user/Downloads/hec-dss/heclib/heclib_f/src/Support/ymddat.f:161: undefined reference to _gfortran_os_error_at'
/users/user/Downloads/hec-dss/heclib/heclib_f/src/Support/ymddat.f:161: undefined reference to _gfortran_os_error_at' /users/user/Downloads/hec-dss/heclib/heclib_f/src/Support/ymddat.f:161: undefined reference to _gfortran_os_error_at'
../heclib_f/Output/libhec_f.a(zpseudorts6.f.o): In function zpseudorts6_': /users/user/Downloads/hec-dss/heclib/heclib_f/src/zpseudorts6.f:73: undefined reference to _gfortran_os_error_at'
../heclib_f/Output/libhec_f.a(zpseudorts6.f.o):/users/user/Downloads/hec-dss/heclib/heclib_f/src/zpseudorts6.f:73: more undefined references to _gfortran_os_error_at' follow collect2: error: ld returned 1 exit status make[1]: *** [Output/libjavaHeclib.so] Error 1 make[1]: Leaving directory /users/user/Downloads/hec-dss/heclib/javaHeclib'
make: *** [all] Error 2
`

GNU Fortran (GCC) 10.3.0
NAME="CentOS Linux"
VERSION="7 (Core)"

proposal: move C and Fortran code into separate directory/project structures

I'm looking for feedback on this proposal.

It would create clarity if the code for each project had it's own directory. This would simplify makefiles, and give clarity on what project depends on what code.

current - heclib_c
-------
Dss7/Internal
Dss7/Public
Dss7/SemiPublic
Dss7/System
Dss7/Utilities
Dss7/Utilities/DateTime
DssInterface/CandFortran/C
DssInterface/v6and7/C
Dss6/C

proposed 
------------
heclib_c/src/headers
heclib_c/src/Internal
heclib_c/src/Public
heclib_c/src/SemiPublic
heclib_c/src/System
heclib_c/src/Utilities
heclib_c/src/Utilities/DateTime
heclib_c/src/DssInterface
heclib_c/src/DssInterface/v6and7
heclib_c/src/Dss6


current heclib_f 
-------
Dss6/Fortran
DssInterface/CandFortran/Fortran
DssInterface/v6and7/Fortran

proposed 
------------
heclib_f/src
heclib_f/src/DssInterface
heclib_f/src/DssInterface/v6and7

squeeze is not working for DSS 6 files.

status code- behavior change in ztsRetrieve

TestInvalidTimeWindow tries to read time series data, using a range of dates that do not exists in the DSS file.

The behavior appears to have changed.

In the table below the values are the returned status codes.

ztsRetrieve	latest code	january
DSS6	0	-1 (5)
DSS7	-1	-1

note: (5) indicates lower level code had status5.

Test setup:

path = "//TYRONE 4 NE BALD EAG/PRECIP-INC/01JUN1972/1HOUR/OBS/"
zStructTimeSeries* tss = zstructTsNewTimes(path, "02SEP2008", "0001",
			"11OCT2009", "2400");
		status = ztsRetrieve(ifltab, tss, 0, 2, 0);

data.zip