hydroffice / hyo2_bag

HydrOffice BAG Tools

• GitHub: https://github.com/hydroffice/hyo2_bag
• Project page: url
• License: LGPLv3 or IA license (See Dual license)

General Info

HydrOffice is a research development environment for ocean mapping. It provides a collection of hydro-packages, each of them dealing with a specific issue of the field. The main goal is to speed up both algorithms testing and research-2-operation.

The BAG Tools hydro-package collects tools for working with BAG files. BAG is a data format by the ONS-WG (Open Navigation Surface Working Group).

Dependencies

For the BAG library, you will need:

• python [>=3.5]
• numpy
• h5py
• lxml
• gdal
• PyInstaller [for freezing the tools]

For running some of the example scripts, you might also need:

• matplotlib

hyo2_bag's Issues

Notes on getting hyo2_bag running on Windows 11/Python3.12/venv

First of all, thanks a lot for putting together this package - I needed something simple to manipulate BAGs and the differences in versions / details were causing me some pain trying to use the raw GDAL cli.

I hit some snags getting it running on windows 11 in a python 3.12 venv, here's the notes on what I did to get it going. I'm just posting in case someone comes across it and has any issues setting up the environment. If you decide that any subset / cleaned up version of these could be included to improve the project, I would be happy to work on a patch.

Notes

There seems to be a dependency on hyo2.abc.logging that I couldn't find in the repo when running this. I swapped the calls to use the normal logger and removed the dependency and things seem to work (not suggesting that's necessarily the solution, just that it got my to point of being able to run.

I took notes as a worked through getting it running on Windows 11 in a venv with python 3.12 - here's what I found:

• Install OSGeo4W tools (I had already done this but they're needed)
• Update the PROJ_LIB env var to point at the installed proj folder (C:\OSGeo4W\share\proj ) to resolve "PROJ: proj_create_from_database: Cannot find proj.db"
• Install GDAL python manually from wheel (windows issue) ( https://github.com/cgohlke/geospatial-wheels/releases/tag/v2024.2.18)
• pip install pyproj python-dateutil (and probably others, my virtual env dependencies pasted below)

To get all the examples running, as I said I removed the missing abc.logging module and used the default python logger instead.

I found that running the tools scripts with a subprocess call was not using the same python environment as as my current env (the tools were runnable from my main shell, but failing with import errors in the subprocess call). As a workaround, I just switched to call their main functions and passed in the expected arguments.

I also removed __import__('pkg_resources').declare_namespace(__name__) from the main init file, probably breaks somethign with packaging but I couldn't get it running locally otherwise.

Here's the hacky branch I have going that lets me run the bag_* and run_* examples (so far, haven't touched the using_* examples yet): https://github.com/hydroffice/hyo2_bag/compare/master...heathhenley:hyo2_bag:dev_hh_example_tweaks?expand=1

Here's my pip freeze output:

certifi==2024.2.2
contourpy==1.2.0
cycler==0.12.1
fonttools==4.49.0
GDAL @ file:///C:/users/heath/Desktop/GDAL-3.8.4-cp312-cp312-win_amd64.whl#sha256=841c9569814fb5167e5b5652cf244f20a91051d1d6dd1ee97e72a0a9c0631d8c
h5py==3.10.0
hyo2.bag==1.2.0
kiwisolver==1.4.5
lxml==5.1.0
matplotlib==3.8.3
numpy==1.26.4
packaging==23.2
pillow==10.2.0
pyparsing==3.1.1
pyproj==3.6.1
python-dateutil==2.9.0.post0
setuptools==69.1.1
six==1.16.0
wheel==0.42.0

fails with h5py 3

bag explorer needs to be updated to work with h5py 3.

h5py changed the way the file mode is set when accessing .h5 files. The mode now needs to be given explicitly.

Test logs report

.EEE.....
======================================================================
ERROR: test_bag_File_filename (test_hydroffice_bag_base.TestBagBase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autopkgtest-lxc.niorsqvb/downtmp/autopkgtest_tmp/tests/test_hydroffice_bag_base.py", line 34, in test_bag_File_filename
    bag_0 = File(self.file_bag_0)
  File "/usr/lib/python3/dist-packages/hydroffice/bag/base.py", line 69, in __init__
    super(File, self).__init__(name=name, mode=mode, driver=driver,
  File "/usr/lib/python3/dist-packages/h5py/_debian_h5py_serial/_hl/files.py", line 444, in __init__
    fid = make_fid(name, mode, userblock_size,
  File "/usr/lib/python3/dist-packages/h5py/_debian_h5py_serial/_hl/files.py", line 215, in make_fid
    raise ValueError("Invalid mode; must be one of r, r+, w, w-, x, a")
ValueError: Invalid mode; must be one of r, r+, w, w-, x, a

======================================================================
ERROR: test_bag_File_open (test_hydroffice_bag_base.TestBagBase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autopkgtest-lxc.niorsqvb/downtmp/autopkgtest_tmp/tests/test_hydroffice_bag_base.py", line 30, in test_bag_File_open
    self.assertIsNotNone(File(self.file_bag_0))
  File "/usr/lib/python3/dist-packages/hydroffice/bag/base.py", line 69, in __init__
    super(File, self).__init__(name=name, mode=mode, driver=driver,
  File "/usr/lib/python3/dist-packages/h5py/_debian_h5py_serial/_hl/files.py", line 444, in __init__
    fid = make_fid(name, mode, userblock_size,
  File "/usr/lib/python3/dist-packages/h5py/_debian_h5py_serial/_hl/files.py", line 215, in make_fid
    raise ValueError("Invalid mode; must be one of r, r+, w, w-, x, a")
ValueError: Invalid mode; must be one of r, r+, w, w-, x, a

======================================================================
ERROR: test_bag_File_raise (test_hydroffice_bag_base.TestBagBase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autopkgtest-lxc.niorsqvb/downtmp/autopkgtest_tmp/tests/test_hydroffice_bag_base.py", line 27, in test_bag_File_raise
    File(self.file_fake_0)
  File "/usr/lib/python3/dist-packages/hydroffice/bag/base.py", line 69, in __init__
    super(File, self).__init__(name=name, mode=mode, driver=driver,
  File "/usr/lib/python3/dist-packages/h5py/_debian_h5py_serial/_hl/files.py", line 444, in __init__
    fid = make_fid(name, mode, userblock_size,
  File "/usr/lib/python3/dist-packages/h5py/_debian_h5py_serial/_hl/files.py", line 215, in make_fid
    raise ValueError("Invalid mode; must be one of r, r+, w, w-, x, a")
ValueError: Invalid mode; must be one of r, r+, w, w-, x, a

----------------------------------------------------------------------
Ran 9 tests in 0.003s

FAILED (errors=3)

full log https://ci.debian.net/data/autopkgtest/testing/amd64/h/hydroffice.bag/14823832/log.gz ,

This log is for v0.2.15, but the version in github gives the same error.