oceanmodeling / adcircpy Goto Github PK

View Code? Open in Web Editor NEW

33.0 33.0 25.0 3.06 MB

Python library for managing input and output files for the ADCIRC model

Home Page: https://adcircpy.readthedocs.io/en/latest/

License: GNU General Public License v3.0

Python 96.14% Shell 3.64% Roff 0.22%

adcircpy's People

Contributors

Stargazers

Watchers

adcircpy's Issues

error in using best_track_file

Hi all,

I just want to learn more about the format of the fort22 file to drive the adcirc simulation. When I use the simplest code to download Sandy's best track, it failed. I don't know how to solve this problem. Could you please give some advice to help me fix this error? Many thanks to you.

some screenshot is shown blow:

`RUNDAY` obtained from `BestTrack` wind forcing

When I try to set up a control file for a meteorologically-forced run from a fort_22 file, the RUNDAY ends up being too long in comparison to the time extent in the fort.22; this results in a SEGFAULT when attempting to run ADCIRC.
RUNDAY ends up as 1.79167 days but ADCIRC says its 1.79 days leading to the problem in the below. Perhaps should allow for some wiggle room with the RUNDAY calculation?

    NWS = 8  # wind forcing option
    SPIN_UP_TIME = 0.5  # how long to coldstart for in days
    MESH = "/work/meshes/SIRA25m.14"  # mesh name
    FORT13 = "/work/meshes/SIRA25m_P_TIDES_001.13"  # nodal attribute file

    OUTPUT_DIRECTORY = Path("/work/forced/ncep/running_simulations/")
    if not OUTPUT_DIRECTORY.exists():
        OUTPUT_DIRECTORY.mkdir(parents=True, exist_ok=True)

    ##########################################

    print(f"Writing input files for: {track_file} using NWS={NWS} wind forcing option")

    # open mesh file
    mesh = AdcircMesh.open(MESH, crs=4326)

    # add the nodal attributes file
    mesh.import_nodal_attributes(FORT13)
    for attr in mesh.get_nodal_attribute_names():
        mesh.set_nodal_attribute_coldstart_state(attr, True)
        mesh.set_nodal_attribute_hotstart_state(attr, True)

    # for each best track file, write out some model forcing files
    wind_forcing = BestTrackForcing.from_fort22(track_file, nws=NWS)
    print(wind_forcing.summary())
    mesh.add_forcing(wind_forcing)

    # instantiate driver object
    driver = AdcircRun(
        mesh,
    )

    # request global outputs every 30 minutes
    driver.set_elevation_surface_output(sampling_rate=timedelta(minutes=30))
    # request station output every 6 minutes 
    if station_file is not None:
        driver.set_elevation_stations_output(sampling_rate=timedelta(minutes=6))
        stations = np.loadtxt(station_file,dtype=float,delimiter=',')
        for idx,sta in enumerate(stations):
            driver.add_elevation_output_station(f"Station_{idx}",sta)

    # set numerical parameters 
    driver.timestep = 2.0
    #driver.NFOVER = [1 ,6, 0, 50, 8]
    driver.RUNID = track_file[-11:]
    driver.NSCREEN = 1800
    driver.NOLIBF = 1
    driver.DRAMP = SPIN_UP_TIME
    driver.smagorinsky = True
    driver.smagorinsky_coefficient=0.05
    driver.TAU0 = -3
    driver.gwce_solution_scheme ='explicit'
    driver.H0 = 0.10

    # write driver to disk
    driver.write(OUTPUT_DIRECTORY, overwrite=True)

Windows or Linux OS choice?

Hi @jreniel ,

I wonder if the adcircpy package is installed by using "pip install adcircpy" on the Linux OS? Because I tried to install it with pip on the Windows OS, some errors appeared like below. If the Linux OS is the right choice I will have a new try. Many thanks to you!

Regards,
Jiangchao

`BestTrackForcing` diagnostics

It could be useful to output some basic statistics of the BestTrackForcing file you're working with as a sanity check.

For example

best_track = BestTrackForcing(best_track_file, nws=8)
best_track.summary()

giving you things like

length of track
minimum pressure
maximum sustained winds
start and end locations
start and end times
Maximum relative track speed.

and anything else that could be useful to ensure the tracks are not bogus automatically.

elevation greater than error when running with NEMS configuration

when running ADCIRC inside NEMS, the following error occurs:

** ERROR: Elevation.gt.ErrorElev, ADCIRC stopping. **

this error occurs when using a fort.15 generated with adcircpy with the following code:

# open mesh file
mesh = AdcircMesh.open(fort14_filename, crs=4326)

# init tidal forcing and setup requests
tidal_forcing = Tides()
tidal_forcing.use_all()
wind_forcing = WindForcing(17, 3600)
wave_forcing = WaveForcing(5, 3600)

mesh.add_forcing(tidal_forcing)
mesh.add_forcing(wind_forcing)
mesh.add_forcing(wave_forcing)

slurm = SlurmConfig(
    account=None,
    ntasks=nems.processors,
    run_name=run_name,
    partition=partition,
    walltime=wall_clock_time,
    nodes=int(numpy.ceil(nems.processors / 68)) if tacc else None,
    mail_type='all' if email_address is not None else None,
    mail_user=email_address,
    log_filename=f'{name}.log',
    modules=[],
    path_prefix='$HOME/adcirc/build',
    launcher=launcher,
    extra_commands=[f'source {module_file}'],
)

# instantiate AdcircRun object.
driver = AdcircRun(
    mesh=mesh,
    start_date=nems.start_time,
    end_date=nems.start_time + nems.duration,
    spinup_time=timedelta(days=5),
    server_config=slurm,
)
driver.import_stations(Path(repository_root()) / 'examples/data/stations.txt')
driver.set_elevation_stations_output(timedelta(minutes=6), spinup=timedelta(minutes=6))
driver.set_elevation_surface_output(timedelta(minutes=6), spinup=timedelta(minutes=6))
driver.set_velocity_stations_output(timedelta(minutes=6), spinup=timedelta(minutes=6))
driver.set_velocity_surface_output(timedelta(minutes=6), spinup=timedelta(minutes=6))

`make_plot`

plot_mesh myfort.14 --crs=EPSG:4326

Does not appear to produce a plot nor show anything to the screen. I guess this functionality isn't yet implemented?

`NotImplementedError` when passing end time to output setter

When calling one of the AdcircRun output setting functions, if passing spinup_end or end to the function, the following error occurs:

NotImplementedError: specific spinup end time is not implemented

This comes from this function:
https://github.com/JaimeCalzadaNOAA/adcircpy/blob/ceee09407b6099d715e4f2374c1b04676169cd57/adcircpy/fort15.py#L2286-L2327

use appdirs for storing hamtide datasets

We need to do some app_data_dir sorcery here as well (for when len()>0)
@jreniel in #53 (comment)

https://github.com/JaimeCalzadaNOAA/adcircpy/blob/f6e7c571d7d0e48d225c8b33acbf01395c7f3221/adcircpy/forcing/tides/hamtide.py#L21-L24

WindForcing base class property setting raises error

class WindForcing(Forcing, ABC):
    def __init__(self, nws: int, interval_seconds: int):
        self.NWS = nws
        self.WTIMINC = interval_seconds

self.WTIMINC = interval_seconds
AttributeError: can't set attribute

Reinstante adcircpy/best_track_file.py basic functionality.

The current output of best_track_file entrypoint needs to be piped to a file best_track_file storm_id > fort.22 but we can reinstate basic functionality that was commented out on a debug cycle, particularly a --save argument. Some thought can be given to improve the argument parser behavior.

update README.md

Remove conda method of intsllation and replace with python virtualenv installation method.

spinup start and end times asserted to be timedeltas instead of datetimes when setting mesh output

setting outputs on an AdcircRun object, if passing start and end times for spinup:

nems = ModelingSystem(
    start_time=datetime(2008, 8, 23), 
    end_time=datetime(2008, 8, 23) + timedelta(days=14.5), 
    interval=timedelta(hours=1), 
    atm=AtmosphericMeshEntry(forcings_directory / 'wind_atm_fin_ch_time_vec.nc'), 
    wav=WaveMeshEntry(forcings_directory / 'ww3.Constant.20151214_sxy_ike_date.nc'), 
    ocn=ADCIRCEntry(382)
)
spinup = ModelingSystem(
    start_time=nems.start_time - spinup_time, 
    end_time=nems.start_time, 
    interval=nems.interval, 
    ocn=copy.deepcopy(nems['OCN']), 
    **nems.attributes
)

slurm_config = ...

driver = AdcircRun(
    mesh=mesh, 
    start_date=nems.start_time, 
    end_date=nems.end_time, 
    spinup_time=timedelta(days=5), 
    server_config=slurm_config
)

driver.set_elevation_surface_output(
    nems.interval, 
    spinup=spinup.interval, 
    spinup_start=spinup.start_time, 
    spinup_end=spinup.end_time
)

they are treated as timedelta objects,
https://github.com/JaimeCalzadaNOAA/adcircpy/blob/3cc26e9a9b9ec5a6456db265c591095f29ca59e3/adcircpy/driver.py#L761-L762
https://github.com/JaimeCalzadaNOAA/adcircpy/blob/3cc26e9a9b9ec5a6456db265c591095f29ca59e3/adcircpy/driver.py#L786-L791
raising an AssertionError:

name = 'spinup_start', var = datetime.datetime(2008, 8, 10, 12, 0)

    @staticmethod
    def _certify__OUT__(name, var):
        # certifies TOUTS* and TOUTF*
        msg = f'Error: {name} argument must be either {None}, '
        msg += f'{int} or an instance of type {timedelta}.'
>       assert isinstance(var, (type(None), timedelta, int)), msg
E       AssertionError: Error: spinup_start argument must be either None, <class 'int'> or an instance of type <class 'datetime.timedelta'>.

C:\Environments\CoupledModelDriver\lib\site-packages\adcircpy\driver.py:791: AssertionError

Run debugging

Given a log file and a crashed fort.63.nc, it could be useful to have Fort63 produce an animation automatically finding the appropriate bbox.
This could be a simple function that calls Fort63.animation but additionally reads the log file, finds the node which first exhibited the instability then draws a small region around this area.

Track times appear not correct

The hotstart run time is too short as compared to the duration of the wind forcing present in the best track file.

best_track_run ../sira25m_tides.14 external --external_track=1R1.trk --fort13 ../sira25m_tides.13 --constituents=major --spinup-days=15.0 --elev=30. --mete=30. --velo=30. --skip-run --crs='epsg:4326' --output-directory=testing/ --overwrite --timestep=1.0

This produces a run 16.625 days long. However the wind forcing duration lasts 2008101617 to 2008102020 nearly four simulation days.

The track file is attached (just remove suffix)

1R1.trk.txt

import failure when importing `AdcircRun` from `adcircpy.driver.driver`

The following error happens only in the Github Action (~~I can't reproduce it on my local Windows~~)

https://github.com/noaa-ocs-modeling/CoupledModelDriver/runs/2272056215?check_suite_focus=true

________________ ERROR collecting tests/test_configurations.py _________________
ImportError while importing test module '/home/runner/work/CoupledModelDriver/CoupledModelDriver/tests/test_configurations.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_configurations.py:7: in <module>
    from coupledmodeldriver.configuration import (
coupledmodeldriver/configuration.py:9: in <module>
    from adcircpy.driver.driver import AdcircRun
/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/site-packages/adcircpy/__init__.py:6: in <module>
    from adcircpy.driver.driver import AdcircRun
E   ModuleNotFoundError: No module named 'adcircpy.driver'

`best_track_run` accepts external track

Extending functionality introduced in #81 to enable the best_track_run factory to accept externally defined best track files.

`plot_track`

The plot_track functionality does not appear to work for example when called with best_track_file. The function plot_trajectory doesn't exist anymore; perhaps this should be plot_track?
https://github.com/JaimeCalzadaNOAA/adcircpy/blob/3e6e40498bc0f8dda47b23158142e2ef6e5f9764/adcircpy/cmd/best_track_file.py#L42

CI testing system

It appears unit tests are configured but do not run automatically. Is there a reason for this?
Github actions seem to be the way to go for CI testing here.

empty weir boundaries when reading HSOFS `fort.14`

when inputting a HSOFS fort.14 from

/scratch2/COASTAL/coastal/save/shared/models/meshes/hsofs/grid_v1/fort.14

the weir boundaries (24:.+) are only IDs and are not written with data in the output fort.14:

/scratch2/COASTAL/coastal/save/shared/working/zach/hera_hsofs_sandy_rewritten_mesh_20210325/coldstart/fort.14

This issue blocks noaa-ocs-modeling/CoupledModelDriver#44

Support another source for HAMTIDE to be served online, perhaps via FTP

The resource is always a path to the hard disk, never a URL.

Ok, that makes sense. I wasn't sure if we wanted to support another source for HAMTIDE to be served online though, perhaps via FTP. I will revert this change for now

start_date should always be a datetime object. If it´s not, ~~it should be cast into one~~ throw a TypeError.

Yes, however the function was acting upon it as if it was a timedelta by calling .total_seconds(), so this provides a fix for that that hopefully doesn't break anything existing

Originally posted by @zacharyburnettNOAA in #70 (comment)

refresh codecov token

It seem that codecov might need a token refresh. I'm not fully familiar with this but it seems to be the reason why the tests are failing.

reading in many best tracks...

Creating 10 or so BestTrack objects in a loop leads to a jam up.
For example, something like...

       storms = [f"{track}R1.trk" for track in track_numbers]]
       files = [pathlib.Path(f"{pold}/{storm}") for storm in storms]
       mesh = AdcircMesh.open(mesh_path, crs=4326)
       bbox = mesh.get_bbox(output_type="bbox")
       for k, (storm_name, storm) in enumerate(zip( files, storms)):                                     
           t1 = time.time()                      
           print("about to read it in...")                                                                           
           bt = BestTrackForcing(storm_name, nws=8)
           try:                                   
               print("about to clip...")          
               bt.clip_to_bbox(bbox, mesh.crs)     
           except:                                                                       
               print(f"failure on storm {storm}")        
               pass                                                           
           filenew = pathlib.Path(f"{pnew}/{storm}")
           print("about to write")
           bt.write(filenew, overwrite=True)
           print(f"Processed storm {storm}")
           print(f"Elapsed time is: {time.time()-t1}")

Coops REST data fetching over 31 days fails

On https://github.com/jreniel/AdcircPy/blob/master/AdcircPy/Validation/COOPS/TidalStations.py, if the date range is larger than 31 days, the data fetching will fail.

using local HAMTIDE resource fails to find `Q1` constituent

running from CoupledModelDriver with ADCIRCpy v1.0.24:

(coupledmodeldriver)  Zachary.Burnett@hfe08  /scratch2/COASTAL/coastal/save/shared/working/zach
 ➜ python scripts/hera_hsofs_sandy_rewritten_mesh.py
[2021-03-25 12:34:17,149] cplmdldrv INFO    : generating 1 "hera" configuration(s) in "/scratch2/COASTAL/coastal/save/shared/working/zach/hera_hsofs_sandy_rewritten_mesh_20210325T123417"
[2021-03-25 12:34:17,150] cplmdldrv INFO    : opening mesh "/scratch2/COASTAL/coastal/save/shared/models/meshes/hsofs/grid_v1/fort.14"
[2021-03-25 12:36:26,595] cplmdldrv INFO    : wrote NEMS coldstart configuration: nems.configure, config.rc, model_configure
[2021-03-25 12:36:27,141] cplmdldrv INFO    : writing NEMS hotstart configuration: nems.configure, config.rc, model_configure
[2021-03-25 12:36:27,172] cplmdldrv INFO    : rewriting original mesh "/scratch2/COASTAL/coastal/save/shared/models/meshes/hsofs/grid_v1/fort.14"
Traceback (most recent call last):
  File "/scratch2/COASTAL/coastal/save/shared/working/zach/scripts/hera_hsofs_sandy_rewritten_mesh.py", line 76, in <module>
    write_adcirc_configurations(
  File "/scratch2/COASTAL/coastal/save/shared/repositories/CoupledModelDriver/coupledmodeldriver/adcirc.py", line 392, in write_adcirc_configurations
    driver.write(
  File "/scratch2/COASTAL/coastal/save/shared/repositories/adcircpy/adcircpy/driver.py", line 408, in write
    super().write('coldstart', output_directory / coldstart,
  File "/scratch2/COASTAL/coastal/save/shared/repositories/adcircpy/adcircpy/fort15.py", line 311, in write
    f.write(self.fort15(runtype))
  File "/scratch2/COASTAL/coastal/save/shared/repositories/adcircpy/adcircpy/fort15.py", line 90, in fort15
    f.append(self.get_tidal_forcing())
  File "/scratch2/COASTAL/coastal/save/shared/repositories/adcircpy/adcircpy/fort15.py", line 344, in get_tidal_forcing
    amp, phase = self.tidal_forcing.tidal_dataset(
  File "/scratch2/COASTAL/coastal/save/shared/repositories/adcircpy/adcircpy/forcing/tides/dataset.py", line 29, in __call__
    return self.get_amplitude(constituent, vertices), \
  File "/scratch2/COASTAL/coastal/save/shared/repositories/adcircpy/adcircpy/forcing/tides/hamtide.py", line 49, in get_amplitude
    return self._get_interpolation('elevation', 'AMPL', constituent,
  File "/scratch2/COASTAL/coastal/save/shared/repositories/adcircpy/adcircpy/forcing/tides/hamtide.py", line 125, in _get_interpolation
    dataset = self._get_dataset(variable, constituent)
  File "/scratch2/COASTAL/coastal/save/shared/repositories/adcircpy/adcircpy/forcing/tides/hamtide.py", line 101, in _get_dataset
    self.datasets[variable][constituent]['path'] = path
KeyError: 'Q1'

Originally posted by @zacharyburnettNOAA in noaa-ocs-modeling/CoupledModelDriver#44 (comment)

`BestTrackForcing` clipping speed-up

I have thousands of synthetic storms that I need to clip against the mesh's extent.
Currently one track composed of hourly data with a small mesh of 100k vertices takes 38 seconds to clip. This implies that it would take 3.07 days to clip all my tracks.
We need to figure out what the bottleneck is to speed this operation up!
Unfortunately the data is not for me to share.

Development of GfsForcing class

Development of class to manage GFS wind forcing files.

write `adcircpy` NOAA technical memorandum

https://docs.google.com/document/d/14IOPzHWGrDuOrj4Gxduf34I73zWqXNIcUzDSVUz8hLk

tricountourf test failing

in the techreport branch on 420d3f4, related to #57

https://github.com/JaimeCalzadaNOAA/adcircpy/runs/2115671220?check_suite_focus=true

======================================================================
ERROR: test_make_plot_flat_domain (tests.mesh.test_adcirc_mesh.AdcircMeshTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/unittest/mock.py", line 1325, in patched
    return func(*newargs, **newkeywargs)
  File "/home/runner/work/adcircpy/adcircpy/tests/mesh/test_adcirc_mesh.py", line 224, in test_make_plot_flat_domain
    h.make_plot()
  File "/home/runner/work/adcircpy/adcircpy/adcircpy/mesh/figures.py", line 87, in decorator
    axes = f(*argv, **kwargs)
  File "/home/runner/work/adcircpy/adcircpy/adcircpy/mesh/mesh.py", line 425, in make_plot
    self.tricontourf(
  File "/home/runner/work/adcircpy/adcircpy/adcircpy/mesh/figures.py", line 87, in decorator
    axes = f(*argv, **kwargs)
  File "/home/runner/work/adcircpy/adcircpy/adcircpy/mesh/base.py", line 192, in tricontourf
    axes.tricontourf(self.triangulation, self.values, **kwargs)
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/matplotlib/tri/tricontour.py", line 321, in tricontourf
    return TriContourSet(ax, *args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/matplotlib/tri/tricontour.py", line 40, in __init__
    ContourSet.__init__(self, ax, *args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/matplotlib/contour.py", line 816, in __init__
    kwargs = self._process_args(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/matplotlib/tri/tricontour.py", line 52, in _process_args
    tri, z = self._contour_args(args, kwargs)
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/matplotlib/tri/tricontour.py", line 107, in _contour_args
    self._process_contour_level_args(args[1:])
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/matplotlib/contour.py", line 1172, in _process_contour_level_args
    raise ValueError("Contour levels must be increasing")
ValueError: Contour levels must be increasing

----------------------------------------------------------------------
Ran 63 tests in 4.008s

FAILED (errors=1)

Add shoreline redering to best track plot.

There's a previous commit

I did some quick searches with the keyword "shoreline" and "shapefile" but no commits turned up. Do you have a recollection when this commit was made to narrow it down?

def __fetch_coastline(self):
    cache = _get_cache_directory()
    file_path = str(Path(
        cache + '/ne_110m_coastline/ne_110m_coastline.shp'))
    if not os.path.isfile(file_path):
        zip_file_path = str(Path(cache + '/ne_110m_coastline.zip'))
        url = 'http://naciscdn.org/naturalearth/'
        url += '110m/physical/ne_110m_coastline.zip'
        urllib.request.urlretrieve(url, zip_file_path)
        _zip = zipfile.ZipFile(zip_file_path)
        outdir = str(Path(cache + '/ne_110m_coastline'))
        os.makedirs(outdir, exist_ok=True)
        for name in _zip.namelist():
            data = _zip.read(name)
            outfile = os.path.join(outdir, name)
            f = open(outfile, 'wb')
            f.write(data)
            f.close()
        os.remove(zip_file_path)
    DataSource = ogr.Open(file_path)
    Layer = DataSource.GetLayer()
    vertices_collection = list()
    for feature in Layer:
        Geometry = feature.GetGeometryRef()
        points = Geometry.GetPoints()
        vertices_collection.append(np.asarray(points))
    return vertices_collection

This is a very old protoype which uses GDAL bindings directly. We need to check if the URL is still alive, and replace the _get_cache_directory() with appdirs module. Then just plot the geometry as background. It's not very high res but is a small file and it gives you a good idea of how far the storm is from the land.

Originally posted by @JaimeCalzadaNOAA in #81 (comment)

add class to generate shell script for server deployment of input files.

incorrect besttrack start time in `WTIMINC` but only for `nws=8`

when running from besttrackon nws=8, ADCIRC gives the following error:

ERROR: getHollandStormData: The fort.22 file ends before RNDAY.
ERROR: windTerminate: ADCIRC terminating.

I've checked that the modeled forcing start time (at the bottom of fort.15) is consistent with the beginning of the fort.22, but @WPringle tells me that it should actually be consistent with the beginning of spinup time, and that is why it is breaking (it's starting late and ending past the end of the fort.22)

I'm working on a fix that uses the spinup start time instead of the run start.

no convex hull

I can guarantee you that a convex hull is not what represents the mesh's outer boundary. This breaks a lot of things.
https://github.com/jreniel/adcircpy/blob/5618de27360bced8d796168b6e70c1c0c0959b7c/adcircpy/mesh/base.py#L310

There's no reason for the "hull" of a mesh to be convex. In fact in most cases it is concave! Making a convex assumption is simply mathematically incorrect.
This is similar to the conversation we were having a few year ago about not using alpha shapes. Alpha shapes and the convex hull are not appropriate to compute the mesh's outer shell.

Dunamai Version.from_any_vcs() on setup.py is not compatible with pip deployments.

@zacharyburnettNOAA I've updated pip to v1.0.13 but dunamai doesn't seem to be pip friendly. Can you test and suggest a fix? Thanks!

Originally posted by @JaimeCalzadaNOAA in #37 (comment)

Visualize nodal attributes

Per #129 (comment)
Add support for visualizing scalar and vectorial nodal attributes.

`CRSError` in `BestTrackRunCliTestCase`

======================================================================
ERROR: test_best_track_run (tests.test_best_track_run.BestTrackRunCliTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/adcircpy/adcircpy/tests/test_best_track_run.py", line 51, in test_best_track_run
    best_track_run.main()
  File "/home/runner/work/adcircpy/adcircpy/adcircpy/cmd/best_track_run.py", line 57, in main
    BestTrackRunCommand(args).run()
  File "/home/runner/work/adcircpy/adcircpy/adcircpy/cmd/best_track_run.py", line 25, in __init__
    bt.clip_to_bbox(self.mesh.get_bbox(output_type='bbox'), self.mesh.crs)
  File "/home/runner/work/adcircpy/adcircpy/adcircpy/forcing/winds/best_track.py", line 524, in clip_to_bbox
    utm_crs = CRS(
  File "/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/pyproj/crs/crs.py", line 313, in __init__
    self._local.crs = _CRS(self.srs)
  File "pyproj/_crs.pyx", line 2326, in pyproj._crs._CRS.__init__
pyproj.exceptions.CRSError: Invalid projection: +proj=utm +zone=18P +ellps=WGS84 +type=crs: (Internal Proj Error: proj_create: Error 1027 (Invalid value for an argument): utm: Invalid value for zone)

visualize a fort.63

How does one create an animation (.gif, etc.) of a fort.63?

how to control numerical parameters chosen for fort15 creation?

There seems to be a significant amount of assumptions regarding the numerical parameters chosen for a simulation (e.g., timestep, IM-type, horizontal dissipation, non-fatal override, etc.).
How does one tune these options to their particular case at the scripting level?

specifying adcircpy version to anything greater than X results in matching with version 1.10b0

When installing with the constraint adcircpy>=1.0.20, for instance, the matched version becomes 1.10b0:

2021-03-02T17:29:36.2375381Z Searching for adcircpy>=1.0.20
2021-03-02T17:29:36.2377907Z Reading https://pypi.org/simple/adcircpy/
2021-03-02T17:29:36.3776097Z Downloading https://files.pythonhosted.org/packages/cb/2d/58dde0d59777923eb41bd5fad8ee5c151cdd0285e1e40685e916e95d569e/adcircpy-1.10b0.tar.gz#sha256=90c3d18b80fc243ca265b27e29e527e9b7ed93c387152cc4fce3761623d707c8
2021-03-02T17:29:36.4839137Z Best match: adcircpy 1.10b0
2021-03-02T17:29:36.4841675Z Processing adcircpy-1.10b0.tar.gz

the above is from this log: https://pipelines.actions.githubusercontent.com/aNnw3unkJGHGsOvXnnoce9Nn91yWP2yBPga7Tp0xnDUq4sGz8u/_apis/pipelines/1/runs/177/signedlogcontent/5?urlExpires=2021-03-02T17%3A32%3A31.3409289Z&urlSigningMethod=HMACV1&urlSignature=00Q61MTiSjVsLhwPxPhFy6sv8G32y4hlJpi9Yoz6a70%3D

@JaimeCalzadaNOAA would you mind yanking the 1.10b0 release from pypi? This would solve this problem

Release to pip not going through apparently due to dunamai issue.

See results of action on last 2 releases.

Linter bot failing for unknown reason.

https://github.com/JaimeCalzadaNOAA/adcircpy/runs/2693802968?check_suite_focus=true

Building TPXO database fails

I just installed adcircpy (I use Anaconda on Windows 10) and am now trying to look at my fort.14 grid. When trying to import
from AdcircPy.Model import AdcircMesh
or
from AdcircPy import read_mesh
I get the following error

URLError: <urlopen error ftp error: URLError("ftp error: error_perm('550 tpxo9_netcdf.tar.gz: No such file or directory')")>

Note, I have not installed Adcirc (or the TPXO database) as I'm just using the grid in SWAN at the moment. I haven't properly dug into the code yet but from the description of adcircpy I should be able to simply read the mesh. Any idea why I need TPXO for that? Thanks for your help.

update circleci tests

OSError when downloading hamtide on Windows

when running tests in CoupledModelDriver on my local (Windows) machine, the following error is raised:

OSError: [Errno -37] NetCDF: Write to read only: b'https://icdc.cen.uni-hamburg.de/thredds/dodsC/ftpthredds/hamtide/m2.hamtide11a.nc'

here is the full pytest output:

FAILED           [ 33%][2021-03-02 10:20:50,534] adcirc          INFO    : writing config to "output\local_shinnecock_ike\runs\test_case_1"
[2021-03-02 10:19:49,922] adcirc          WARNING : mesh values (nodal attributes) not found at "input\shinnecock_ike\mesh\fort.13"
[2021-03-02 10:19:53,153] configuration WARNING : could not create symbolic link: [WinError 1314] A required privilege is not held by the client: 'output\\local_shinnecock_ike\\model_configure' -> 'output\\local_shinnecock_ike\\atm_namelist.rc'
[2021-03-02 10:19:56,326] configuration WARNING : could not create symbolic link: [WinError 1314] A required privilege is not held by the client: 'output\\local_shinnecock_ike\\model_configure' -> 'output\\local_shinnecock_ike\\atm_namelist.rc'
Cookie file cannot be read and written: (null)

tests\test_configurations.py:30 (test_local_shinnecock_ike)
def test_local_shinnecock_ike():
        platform = 'local'
        mesh = 'shinnecock'
        storm = 'ike'
    
        input_directory = Path('.') / 'input' / f'{mesh}_{storm}'
        mesh_directory = download_mesh(mesh, storm, input_directory)
        forcings_directory = input_directory / 'forcings'
    
        output_directory = Path('.') / 'output' / f'{platform}_{mesh}_{storm}'
        reference_directory = Path('.') / 'reference' / f'{platform}_{mesh}_{storm}'
    
        runs = {f'test_case_1': (None, None)}
    
        nems = ModelingSystem(
            start_time=datetime(2008, 8, 23),
            end_time=datetime(2008, 8, 23) + timedelta(days=14.5),
            interval=timedelta(hours=1),
            atm=AtmosphericMeshEntry(forcings_directory / 'wind_atm_fin_ch_time_vec.nc'),
            wav=WaveMeshEntry(forcings_directory / 'ww3.Constant.20151214_sxy_ike_date.nc'),
            ocn=ADCIRCEntry(11),
        )
    
        nems.connect('ATM', 'OCN')
        nems.connect('WAV', 'OCN')
        nems.sequence = [
            'ATM -> OCN',
            'WAV -> OCN',
            'ATM',
            'WAV',
            'OCN',
        ]
    
        tidal_forcing = Tides()
        tidal_forcing.use_all()
        wind_forcing = AtmosphericMeshForcing(nws=17, interval_seconds=3600)
        wave_forcing = WaveWatch3DataForcing(nrs=5, interval_seconds=3600)
    
>       write_adcirc_configurations(
            nems,
            runs,
            mesh_directory,
            output_directory,
            email_address='[email protected]',
            platform=Platform.LOCAL,
            spinup=timedelta(days=12.5),
            forcings=[tidal_forcing, wind_forcing, wave_forcing],
            overwrite=True,
        )

C:\Repositories\CoupledModelDriver\tests\test_configurations.py:69: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
C:\Repositories\CoupledModelDriver\coupledmodeldriver\adcirc.py:298: in write_adcirc_configurations
    driver.write(
C:\Environments\CoupledModelDriver\lib\site-packages\adcircpy\driver.py:411: in write
    super().write('hotstart', output_directory / hotstart,
C:\Environments\CoupledModelDriver\lib\site-packages\adcircpy\fort15.py:312: in write
    f.write(self.fort15(runtype))
C:\Environments\CoupledModelDriver\lib\site-packages\adcircpy\fort15.py:91: in fort15
    f.append(self.get_tidal_forcing())
C:\Environments\CoupledModelDriver\lib\site-packages\adcircpy\fort15.py:345: in get_tidal_forcing
    amp, phase = self.tidal_forcing.tidal_database(
C:\Environments\CoupledModelDriver\lib\site-packages\adcircpy\forcing\tides\hamtide.py:44: in __call__
    amp = self.get_amplitude(constituent, vertices)
C:\Environments\CoupledModelDriver\lib\site-packages\adcircpy\forcing\tides\hamtide.py:51: in get_amplitude
    return self._get_interpolation(
C:\Environments\CoupledModelDriver\lib\site-packages\adcircpy\forcing\tides\hamtide.py:97: in _get_interpolation
    zi = self._get_resource(
C:\Environments\CoupledModelDriver\lib\site-packages\adcircpy\forcing\tides\hamtide.py:84: in _get_resource
    return Dataset(base_url + fname)
src\netCDF4\_netCDF4.pyx:2330: in netCDF4._netCDF4.Dataset.__init__
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   OSError: [Errno -37] NetCDF: Write to read only: b'https://icdc.cen.uni-hamburg.de/thredds/dodsC/ftpthredds/hamtide/m2.hamtide11a.nc'

src\netCDF4\_netCDF4.pyx:1948: OSError

`mesh.import_nodal_attribute` scrambles nodal attributes

When I read in a mesh and an associated nodal attribute file two issues emerge:

by default it won't include them in the fort.15 unless I turn them on like this:

    # open mesh file
    mesh = AdcircMesh.open(MESH, crs=4326)

    # add the nodal attributes file
    mesh.import_nodal_attributes(FORT13)
    for attr in mesh.get_nodal_attribute_names():
        mesh.set_nodal_attribute_coldstart_state(attr, True)
        mesh.set_nodal_attribute_hotstart_state(attr, True)

which is fairly non-intuitive if you ask me. I guess you have a reason for that.

When I go to write the files, the fort.13 has a nodal attribute field that is completely scrambled. I would suspect that something is being automatically renumbered?

   driver = AdcircRun(mesh)
   driver.write(OUTPUT_DIRECTORY, overwrite=True)

Consider using code formatting and codes

See https://github.com/krober10nd/SeismicMesh/blob/e0dae182e2840148d3cef7aed49c162b941e98f3/.circleci/config.yml#L10
Also a Makefile like so wouldn't be too bad since I'm always forgetting some commands
https://github.com/krober10nd/SeismicMesh/blob/master/Makefile

`example3` no automatic start date specified

When I go to run example 3 (or in general when I specify a BestTrack file) the start date and end date do not appear to be automatically specified and this leads to an error when figuring out the tidal potentials.

 from datetime import timedelta
 from pathlib import Path
 
 from adcircpy import AdcircMesh, AdcircRun, Tides
 from adcircpy.forcing.winds import BestTrackForcing
 from adcircpy.server import SlurmConfig
 from tests import download_mesh
 
 MESH_URL = 'https://www.dropbox.com/s/1wk91r67cacf132/NetCDF_shinnecock_inlet.tar.bz2?dl=1'
 
 DATA_DIRECTORY = Path(__file__).parent.absolute() / 'data'
 INPUT_DIRECTORY = DATA_DIRECTORY / 'input' / 'NetCDF_Shinnecock_Inlet'
 OUTPUT_DIRECTORY = DATA_DIRECTORY / 'output' / 'example_3'
 
 download_mesh(
     url=MESH_URL, directory=INPUT_DIRECTORY,
 )
 
 # open mesh file
 mesh = AdcircMesh.open(INPUT_DIRECTORY / 'fort.14', crs=4326)
 
 # initialize tidal forcing and constituents
 tidal_forcing = Tides()
 tidal_forcing.use_all()
 mesh.add_forcing(tidal_forcing)
 
 # initialize wind forcing
 wind_forcing = BestTrackForcing('Sandy2012')
 mesh.add_forcing(wind_forcing)
 
 # initialize Slurm configuration
 slurm = SlurmConfig(
     account='account',
     ntasks=1000,
     run_name='adcircpy/examples/example_3.py',
     partition='partition',
     walltime=timedelta(hours=8),
     mail_type='all',
     mail_user='[email protected]',
     log_filename='example_3.log',
     modules=['intel/2020', 'impi/2020', 'netcdf/4.7.2-parallel'],
     path_prefix='$HOME/adcirc/build',
 )
 
 # instantiate driver object
 driver = AdcircRun(mesh, spinup_time=timedelta(days=15), server_config=slurm)
 
 # write driver state to disk
 driver.write(OUTPUT_DIRECTORY, overwrite=True)

results in

(firedrake) (base) keiths-MacBook-Pro:prep Keith$ python debug.py 
100% [..........................................................................] 4260693 / 4260693Traceback (most recent call last):
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/forcing/tides/tides.py", line 459, in start_date
    return self.__start_date
AttributeError: 'Tides' object has no attribute '_Tides__start_date'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "debug.py", line 49, in <module>
    driver.write(OUTPUT_DIRECTORY, overwrite=True)
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/driver.py", line 404, in write
    super().write('coldstart', output_directory / coldstart, overwrite)
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/fort15.py", line 431, in write
    f.write(self.fort15(runtype))
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/fort15.py", line 161, in fort15
    f.append(self.get_tidal_forcing())
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/fort15.py", line 445, in get_tidal_forcing
    forcing = self.mesh.forcings.tides(constituent)
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/forcing/tides/tides.py", line 39, in __call__
    return self.get_tidal_constituent(constituent)
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/forcing/tides/tides.py", line 112, in get_tidal_constituent
    self.get_nodal_factor(constituent),
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/forcing/tides/tides.py", line 174, in get_nodal_factor
    return self.EQ75
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/forcing/tides/tides.py", line 394, in EQ75
    return np.sin(self.I) * np.cos(self.I / 2.0) ** 2 / 0.37988
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/forcing/tides/tides.py", line 578, in I
    return np.arccos(0.9136949 - 0.0356926 * np.cos(self.N))
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/forcing/tides/tides.py", line 582, in N
    return np.deg2rad(self.DN)
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/forcing/tides/tides.py", line 586, in DN
    return self.get_lunar_node()
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/forcing/tides/tides.py", line 346, in get_lunar_node
    - 0.0022064139 * self.hour_middle
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/forcing/tides/tides.py", line 590, in DYR
    return self.forcing_start_date.year - 1900.0
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/forcing/tides/tides.py", line 474, in forcing_start_date
    return self.start_date - self.spinup_time
  File "/Users/Keith/Desktop/PostDoctoralWork/codes/firedrake/lib/python3.7/site-packages/adcircpy/forcing/tides/tides.py", line 462, in start_date
    raise AttributeError(msg)

NFOVER treatment

In the case the user has compiled ADCIRC with DEBUG_ELEVATION compiler flag, the parameter NFOVER becomes a vector of 4 values.
Can we handle this case inside fort.15? Currently, it causes a segfault
For example

driver.NFOVER = [1 ,6, 0, 50, 8]

best_track_file error

After installing adcircpy via pip install adcircpy I wanted to use the best_track_file functionality to produce a fort.20 from a best track (Hurricane Sandy).
Unfortunately it seems that some modules were moved around creating a segfault.
MWE

(base) keiths-MacBook-Pro:KeithJaredR Keith$ best_track_file -h
Traceback (most recent call last):
  File "/Applications/miniconda3/bin/best_track_file", line 5, in <module>
    from adcircpy.cmd.best_track_file import main
  File "/Applications/miniconda3/lib/python3.7/site-packages/adcircpy/cmd/best_track_file.py", line 4, in <module>
    from adcircpy.forcing.winds.best_track import BestTrackForcing
  File "/Applications/miniconda3/lib/python3.7/site-packages/adcircpy/forcing/winds/best_track.py", line 10, in <module>
    from adcircpy.forcing.winds.base import _WindForcing
ModuleNotFoundError: No module named 'adcircpy.forcing.winds.base'

Tests fail due to inability to download natural eart coastline from GitHub workflows

GHW tests returns

=========================== short test summary info ============================
FAILED tests/test_besttrack.py::test_from_atcf - requests.exceptions.Connecti...
============= 1 failed, 19 passed, 1 warning in 152.74s (0:02:32) ==============

This is because best_track.plot_track(show=True) tries to download data and the server may block.
Some potential fixes:

provide the data on a container
mock the request call
disable the coastline plotting

Development OwiForcing class

Development of OwiForcing class to manage OWI wind forcing files.

oceanmodeling / adcircpy Goto Github PK

adcircpy's People

Contributors

Stargazers

Watchers

Forkers

adcircpy's Issues

Recommend Projects

Recommend Topics

Recommend Org