insarlab / pysolid Goto Github PK
View Code? Open in Web Editor NEWA Python wrapper for solid Earth tides
License: GNU General Public License v3.0
A Python wrapper for solid Earth tides
License: GNU General Public License v3.0
It seems straightforward to pass array and scalar variables from Fortran subroutine to Python function, as shown here: https://notmatthancock.github.io/2017/02/10/calling-fortran-from-python.html. Thus, we should eliminate the writing/reading of solid.txt
file and pass solid.for
results directly to point/grid.py
.
Relevant link: https://numpy.org/doc/stable/f2py/f2py.getting-started.html#the-quick-and-smart-way.
Contributions are welcomed!
When using calc_solid_earth_tides_grid
through MintPy for two separate runs involving overlapping frames, I notice unexpected behavior.
Specifically, I would expect the overlapping region between the two to be consistent after referencing epochs in the stack to the same reference date and point, but I instead find a non-zero, sub-mm level mean offset (~0.1 mm, and a range of ~0.5mm) and stdev (~0.33 mm):
Here is the residual within the overlapping region:
Please let me know how I can help to debug, if I should clarify anything, and if also perhaps it would be best to share the input files I used on the MintPy end. Many thanks
(python 3.11.3 windows)
when running python -c "import pysolid; print(pysolid.__version__)"
i get 0.3.2
which means pysolid was installed succesfully ?
when i run python PySolid/tests/grid.py
i get the following error :
D:\test_gnss\PySolid\tests\grid.py
Traceback (most recent call last):
File "D:\test_gnss\gnss\Lib\site-packages\pysolid\grid.py", line 45, in calc_solid_earth_tides_grid
from pysolid.solid import solid_grid
ImportError: DLL load failed while importing solid: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\test_gnss\PySolid\tests\grid.py", line 33, in
tide_u) = pysolid.calc_solid_earth_tides_grid(dt_obj, atr, verbose=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\test_gnss\gnss\Lib\site-packages\pysolid\grid.py", line 50, in calc_solid_earth_tides_grid
raise ImportError(msg)
ImportError: Cannot import name 'solid' from 'pysolid'!
Maybe solid.for is NOT compiled yet.
Check instruction at: https://github.com/insarlab/PySolid.
I was running a MintPy installation with conda (conda install -c conda-forge --file ~/tools/MintPy/requirements.txt
), and it failed to find the pysolid
package in conda-forge for my M1 Mac:
PackagesNotFoundError: The following packages are not available from current channels:
- pysolid
Current channels:
- https://conda.anaconda.org/conda-forge/osx-arm64
- https://conda.anaconda.org/conda-forge/noarch
- https://repo.anaconda.com/pkgs/main/osx-arm64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/r/osx-arm64
- https://repo.anaconda.com/pkgs/r/noarch
forwarded from Paul Rosen (on trying to install it on an M2 machine):
F2py appears to be skipping code in solid.fort that is not supported, in my case an implied DO in a DATA statement that is common in old fortran. A possible solution is to modify the setup to use –std=legacy in gfortran, but I don’t know how robust that is for all compilers and all machines.
More details
Pip failed until I had Michael Aivazis guide me through what is actually happening with pip and we got the code to rebuild the source with the right pip command line switch (--no-binary).
I ran f2py by hand in verbose mode and it told me that the compiler did not understand a particular implied do array definition so it was skipping the code! So I have no confidence that the code is actually correctly compiled. According to Aivazis, you must use –std=legacy to get old code like this to compile properly, but I have no idea how to do that with the pip infrastructure.
it might be solved now that you can Conda install it even for osx-arm
... but it's probably a bad sign for future stability that a new macbook causes the standard compilation to fail.
A bug found with a solution suggested by @fastice:
I ran into a problem with pysolid.calc_solid_earth_tides_grid().
Basically I am running multiple threads at the same time from the same directory, which means that the individual threads are overwriting or deleting the
solid.txt
file.As simple fix so that files from multiple threads would not conflict would be to make the file names unique with something like.
import uuid txt_file = os.path.abspath(f’solid.{uuid.uuid1().hex}.txt') if os.path.isfile(txt_file): os.remove(txt_file)No need to change soon on my account, I found a workaround but others might run into a similar problem in a production environment.
This unique TXT filename should be passed to the Fortran code directly as a function argument.
The setup.py
file has been recently updated to specify the license as license='GPL-3.0-or-later'
.
Anyway in the classifiers section it is is specified:
License :: OSI Approved :: GNU General Public License v3 (GPLv3)
which corresponds to GPL-3.0-only
, while GPL-3.0-or-later
corresponds to
License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+)
Please clarify the licensing.
PySolid currently does not support Python 3.12+ because it relies on numpy.distutils
to build the compiled Fortran extension. numpy.distutils
has been removed for Python 3.12+ builds of numpy:
https://numpy.org/devdocs/reference/distutils_status_migration.html#distutils-status-migration
Reccomendations there are to upgrade to:
setuptools
which we cannot do because it doesn't support FortranI (@jhkennedy) lean towards scikit-build-core out of familiarity, but I haven't used mesons enough to have a well-formed opinion.
In addition to the x86_64
builds currently supported by pysolid
, it would be great if you could support an arm64
build for MacOS. This would enable newer Mac users to use pysolid
. Also, since pysolid
is a dependency of MintPy
, MintPy
is not currently installable on arm64
Macs.
A pip-installable version of pysolid
would fix my immediate issue, but long-term it would be good to have conda support as well.
I have tried to install pysolid
via pip, but ran into low-level errors when trying to build the pysolid
wheels. I'm happy to provide the error messages here if they would be useful.
When running test_SET_grid.py
the test fails after running the solid_grid
function because the output solid.txt
file contains empty values (#12).
Sample of file with empty values:
# program solid -- UTC version -- 2018jun01
# year, month, day = 2020 12 25
# hour, minute, second = 14 7 44
# S, N, W, E = 33.479166795 33.800000000 -118.200000000 -117.796666828
# step_lat, num_lat = -0.009166663 35
# step_lon, num_lon = 0.009166663 44
, , , ,
, , , ,
, , , ,
, , , ,
etc...
Open an issue here to keep track of things for pysolid, so that insarlab/MintPy#648 could be focused on mintpy itself.
@pbrotoisworo are you able to use pysolid on Windows? If so, how do you install it, especially how do you compile the Fortran code?
I want to install this package in centos, and this machine haven't anaconda.
So I have to build and install locally.
But I encountered this error when I followed the steps of readme.
ERROR: Command errored out with exit status 1: /usr/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/pyserver_dev/PySolid/setup.py'"'"'; __file__='"'"'/home/pyserver_dev/PySolid/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_iefdv6v/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/python3/include/python3.6m/pysolid Check the logs for full command output.
Something we might try after this PR: you can specify the dependencies dynamically with newer setuptool
. I didn't include it since I wasn't sure how it would interact with PySold's current setuptools version pinning
https://github.com/opera-adt/dolphin/blob/main/pyproject.toml#L23-L34
Originally posted by @scottstanie in #69 (comment)
We need to add the compilation process into the setup.py
so that one could run the following, as an alternative to run f2py
and set PYTHONPATH
.
cd ~/tools/PySolid
python setup.py build
python setup.py install
Relevant question on StackOverflow: https://stackoverflow.com/questions/14805236/translate-f2py-compile-steps-into-setup-py.
Contributions are welcomed!
pip check
is failing because PySolid's pyproject.toml
has setuptools_scm
as a requirement via this line:
https://github.com/insarlab/PySolid/blob/main/pyproject.toml#L39
So, it thinks it's a runtime dependency that is missing. We should fix that in PySolid.
Originally posted by @jhkennedy in conda-forge/mintpy-feedstock#11 (comment)
pysolid
is now available on PyPI (https://pypi.org/project/pysolid/). But pip install pysolid
does not work, because a Fortran compiler is required to build the package but is not available via pip, as far as I know. It will be great if pip install
could work!
I agree with you, that's the timing difference along the orbit! If it's not accounted for, this timing variation will introduce a small ramp along the azimuth direction for a long track (I described this in section III.E.2 in my 2022 TGRS paper, after Dr. Dennis Milber brought it up in our email communications); and for the multiple frames stitching, as in your case here, it looks like a jump in our current implementation!
I intended to write a swath
mode, in addition to the current grid
and point
mode, for this scenario. The implementation will be much easier if we have a true API-like talking between the python and fortran code (#2), instead of the current talking via text file. Therefore, I have not done it yet.
Originally posted by @yunjunz in #54 (comment)
Since solid.for
could return numpy array since #56, we could implement this feature.
With the latest release of numpy 1.22
, pysolid now has a reshape
error for the pysolid.grid
module as below:
python ${HOME}/tools/PySolid/tests/test_SET_grid.py
Test solid Earth tides calculation for a spatial grid at a specific time
PYSOLID: ----------------------------------------
PYSOLID: datetime: 2020-12-25T14:07:44
PYSOLID: SNWE: (33.4666668, 33.8, -118.2, -117.7833335)
SOLID : calculate solid Earth tides in east/north/up direction
SOLID : shape: (36, 45), step size: -0.0092 by 0.0092 deg
SOLID : calculating / writing data to txt file: /root/tools/PySolid/solid.txt
PYSOLID: read data from text file: /root/tools/PySolid/solid.txt
Traceback (most recent call last):
File "/root/tools/PySolid/tests/test_SET_grid.py", line 34, in <module>
tide_u) = pysolid.calc_solid_earth_tides_grid(dt_obj, atr, verbose=True)
File "/root/tools/miniconda3/lib/python3.9/site-packages/pysolid/grid.py", line 101, in calc_solid_earth_tides_grid
tide_e = fc[:, 0].reshape(length, width)
ValueError: cannot reshape array of size 1614 into shape (36,45)
If I pin the numpy version to <1.22
, there is no issue. Thus, this seems to be an f2py
module behavior change introduced in numpy 1.22
release.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.