csiro-hydrogeology / pyela Goto Github PK

https://readthedocs.org/projects/pyela/builds/9486403/

reading sources... [ 90%] source/visual
reading sources... [100%] source/visual3d
qt.qpa.xcb: could not connect to display 
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl, xcb.

Aborted (core dumped)

Swan Coastal plains data, even with a smaller cube to visuaise. Not responding and blank canvas.

mayavi 4.7.1
vtk 8.2.0
pyqt5 5.13.0

Immediate driving need is that LithologiesClassesOverlayVisual3d still has some assumptions about the easting and northing column names. One also still needs to define global variables for column names.

Probably need a parent class that stores the concepts to the column names, and that most objects would inherit from.

Was getting very large magnitude AHD heights after blending COndamine DEM

Diagnosing in debug mode, it looks like a change of value from np.float32 to np.float64 on assigning (as an aside, unintuitive)

v[i] = read_raster_value(self.dem_raster, self.data_grid, x[i], y[i])

v.dtype
dtype('float64')
self.data_grid.dtype
dtype('float32')
v[0]
-3.3999999521443642e+38
self.data_grid[0][0]
-3.4e+38
self.dem_raster.nodata
-3.4e+38

so, further comparison in whether v[i] is equal to raster.nodata fails.

Perhaps because of the warning at the end after coverage run run_tests.py .

----------- coverage: platform linux, python 3.7.3-final-0 -----------
Name                         Stmts   Miss Branch BrPart  Cover
--------------------------------------------------------------
ela/__init__.py                  0      0      0      0   100%
ela/_version.py                  2      2      0      0     0%
ela/classification.py          132     37     58      6    69%
etc
ela/visual3d.py                154    154     14      0     0%
--------------------------------------------------------------
TOTAL                         1092    582    272     24    48%

============================================================ 34 passed, 5 warnings in 2.44 seconds =============================================================
Coverage.py warning: No data was collected. (no-data-collected)

Consider an area such as around Canberra; east there are numerous bores with 'Shale' lithology. At interpolation this seems to dominate including far away from it where there should be other lithologies. A high density of uniform classification near one geolocation may have undue influence far away from it. Proximity should have more weight in this circumstance.

dem_array = surface_array(bungendore_raster, x_min, y_min, x_max, y_max, grid_res) on a raster from ELVIS that was saved with grid coordinates. May not have been the same CRS as the one from Bungendore.

Tried in a blank conda env with pip install ela see if in installed dependencies correctly. Unfortunately not quite. from ela.visual import * trggers missing modle PIL. Seems to be if the module pillow is missing, but this is in the latest requirements.txt file on master, so likely I need to update the package on pipy.

Tried to have mayavi related features unit tested. OK on my machine with a graphical env, but not on travis.
For the record, but not a battle I intend to fight actively.

  Fatal Python error: Aborted
Current thread 0x00007f0ceef72740 (most recent call first):
  File "/home/travis/miniconda/envs/ela_test/lib/python3.7/site-packages/pyface/ui/qt4/init.py", line 40 in <module>
etc. etc.
 File "/home/travis/build/csiro-hydrogeology/pyela/ela/visual3d.py", line 3 in <module>

/home/travis/.travis/functions: line 104:  9312 Aborted                 (core dumped) coverage run -m pytest

The command "coverage run -m pytest" exited with 134.

@DennisHgj had bumped into this issue

Short:
pip install mayavi and pip install pyqt5 lead to an "access denied" error even if this is done under the user's folder.
TODO capture the error message.

Mayavi is now (as opposed to a year ago) available for py3 on conda. pyqt5 is not, but there is a pyqt conda package that may be the same thing as pyqt5 on pip. This is worth checking; the less we install from pip, in general, the better.

Overlayed view: code works on the Perth Peel data, but on the Condamine this error occures. The DEM and point bore data does display, but not the volumes. The color coding of the bore also looks wrong. Repro with exploratory-lithology-analysis\case_studies\Condamine\Condamine_Facies.ipynb

Consider below. The one missing value propagates as black over twice the section depth it needs to. Artefact of the interpolation and the way we craft the PolyData (?) for tube viz.

1/1/2000 becomes 1with1with2000 and sand/clay/gravel sandwithclaywithgravel. Not sure this helps; at any rate it should be with spaces to avoid the formation of composite terms. I do have facilities to split terms with 'with' but this is a crutch.

Plotting a histogram of FromDepth and ToDepth values is too crude a way to determine the span of heights over which to interpolate with kNN. Need a helper tool that reliably calculates the number of data points available at a certain height.

Trying to fix the travis build failure

import geopandas as gpd

after running unit tests in a new environment created with:

name: ela_test
channels:
  - conda
  - conda-forge
dependencies: 
  - rasterio=1.0.21
  - cartopy=0.17.0
  - geopandas=0.4.1
  - pandas=0.24.2
  - nltk=3.4.4
  - scikit-learn=0.21.2
  - matplotlib=3.1.0
  - vtk=8.2.0
  - pytest-cov=2.7.1
  - cython=0.29.12
  - coverage=4.5.3
  - wordcloud=1.5.0
  - pyqt=5.9.2
  - mayavi=4.7.1
  - codecov=2.0.15
  - pip
  - pip:
    #  - mayavi==4.7.1 # no need now that suitable conda pkg
    - git+https://github.com/jmp75/striplog@master#egg=striplog

There are 16 lithology classes defined, however the display seems to miss some. The display may be incorrect.

commit a4b989761c00085a9e5819a3f00f77f391c37454 (HEAD -> testing, origin/testing)
Date:   Sat Oct 5 17:13:24 2019 +1000
pyela/pyvista_sample/CBR_Sample.py

Reviewing / testing a contribution for pyvista use. Some of the code has hard coded use of 'x' and 'y' for column names expected in the data frame.

On some data sets such as the Condamine where depth of bore can go really deep, the data cube may end up with more than a 1000 depths slices but less in X/Y. This ends up looking odd (really long). Is it possible to have something that is more spatially meaningful (albeit with a scaling factor still needed because in isometric view then the Z axis may be completely squashed).