paulross / totaldepth Goto Github PK

In setup.py XML_FORMAT_FILES and the data_files root use '/' as separator. This needs to be platform sensitive.

There are are a large number of performance and scale tests that write their results to stdout/stderr. These results then have to be munged into a form that can be displayed.

It would be superior if these performance tests wrote to file, say in gnuplot format.

Hi,
I'm getting an error on the pip install...

using windows and VSC, versions;
VSC 1.66.2
python 3.9.7.

error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/

ERROR: Failed building wheel for TotalDepth

Sometimes there is missing data in well log file and the call to convert fails since it tries to evaluate NoneType times a float

2020-02-23 19:58:12,907 - PlotLogs.py - 16597 - (MainThread) - CRITICAL - PlotLogPasses._processFileLIS(): In "'dummy.lis'", error type <class 'TypeError'>: TypeError("unsupported operand type(s) for *: 'NoneType' and 'float'") 2020-02-23 19:58:12,909 - PlotLogs.py - 16597 - (MainThread) - CRITICAL - Traceback (most recent call last): File "/home/igormorgado/venv/lib/python3.7/site-packages/TotalDepth-0.3.0-py3.7-linux-x86_64.egg/TotalDepth/PlotLogs.py", line 401, in _processFileLIS self._plotLISUsingLgFormats(myFi, lpIdx, aPrs, fpOut) File "/home/igormorgado/venv/lib/python3.7/site-packages/TotalDepth-0.3.0-py3.7-linux-x86_64.egg/TotalDepth/PlotLogs.py", line 584, in _plotLISUsingLgFormats logging.debug('PlotLogPasses._plotLISUsingLgFormats(): PlotRecords={:s}.'.format(str(thePrs))) File "/home/igormorgado/venv/lib/python3.7/site-packages/TotalDepth-0.3.0-py3.7-linux-x86_64.egg/TotalDepth/LIS/core/FileIndexer.py", line 369, in __str__ str(self.logPass), File "/home/igormorgado/venv/lib/python3.7/site-packages/TotalDepth-0.3.0-py3.7-linux-x86_64.egg/TotalDepth/LIS/core/LogPass.py", line 259, in longStr self.xAxisLastValOptical, File "/home/igormorgado/venv/lib/python3.7/site-packages/TotalDepth-0.3.0-py3.7-linux-x86_64.egg/TotalDepth/LIS/core/LogPass.py", line 366, in xAxisLastValOptical Units.opticalUnits(self._rle.xAxisUnits) File "/home/igormorgado/venv/lib/python3.7/site-packages/TotalDepth-0.3.0-py3.7-linux-x86_64.egg/TotalDepth/LIS/core/Units.py", line 769, in convert retVal = myUcc.convert(v, u_1, u_2) File "/home/igormorgado/venv/lib/python3.7/site-packages/TotalDepth-0.3.0-py3.7-linux-x86_64.egg/TotalDepth/LIS/core/Units.py", line 602, in convert return self.unitConvertor(u_1).convert(v, self.unitConvertor(u_2)) File "/home/igormorgado/venv/lib/python3.7/site-packages/TotalDepth-0.3.0-py3.7-linux-x86_64.egg/TotalDepth/LIS/core/Units.py", line 633, in convert baseVal = val * self.mult TypeError: unsupported operand type(s) for *: 'NoneType' and 'float'

I just tracked down a bug in my code where the last depth frame in a LIS file ended up missing. The bug stemmed from a line where I wanted to read a selection of the channels over the entire depth:

logPass.setFrameSetChX(file, channelMnemList, logPass.xAxisFirstEngVal, logPass.xAxisLastEngVal)

My expected behaviour was to get every frame from logPass.xAxisFirstEngVal to and including logPass.xAxisLastEngVal, while I got every frame to but not including the latter, since TotalDepth here follows the Python convention on ranges.

This makes it somewhat inconvenient to get all frames for a selection of channels, since I suppose it means that you have to specify a stop depth after logPass.xAxisLastEngVal in order to get every frame. Therefore, I have two possible suggestions on how to change this behaviour:

1. Change the behaviour of LogPass.setFrameSetChX(...) so that it gets the frames to and including the specified stop frame depth. This would make it more convenient to use, although it might not be quite Pythonic.
2. Make the Xstart and Xstop arguments optional. If they are not specified, then default to getting every frame.

Reference issue #5

I am installing TotalDepth for the first time, and I tried using pip as recommended by the documentation. Specifically, I'm trying to install on top of a Python 3.6 Anaconda distribution on macOS 10.13.4. However, this is giving me the following error:

$ pip install totaldepth

Collecting totaldepth
  Using cached TotalDepth-0.2.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/0k/6fvjhp7x2659gf4_pbq0zkj40000gp/T/pip-install-3_oz2lvj/totaldepth/setup.py", line 125, in <module>
        ext_modules = cythonize("src/TotalDepth/LIS/core/*.pyx"),
      File "/anaconda3/lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 897, in cythonize
        aliases=aliases)
      File "/anaconda3/lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 777, in create_extension_list
        for file in nonempty(sorted(extended_iglob(filepattern)), "'%s' doesn't match any files" % filepattern):
      File "/anaconda3/lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 102, in nonempty
        raise ValueError(error_msg)
    ValueError: 'src/TotalDepth/LIS/core/*.pyx' doesn't match any files
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/0k/6fvjhp7x2659gf4_pbq0zkj40000gp/T/pip-install-3_oz2lvj/totaldepth/

Indeed, the directory src/TotalDepth/LIS/core/ does not contain any .pyx files in the PyPi version.

Method time.clock() was deprecated since 3.3 and removed in 3.8 leading to many utils not work anymore.

Hi there,

I am trying to install TotalDepth on a Windows environment and it fails on this line due to CFLAGS not present in Windows.

I do not know if there is more issues regarding this yet.

(fthis is my first bug-report, I'm sorry if I missed something)

DAT_parser.py uses np.object three times, which is not supported anymore by numpy 1.20 and above. The error message suggests to replace np.object with object, which worked for me.

Hi @paulross ,
Thanks for making TotalDepth.

I'm using Python 3.8 and used pip3 install TotalDepth. So I have version 0.3.1 and I'm using Ubuntu 20.04 LTS.
I downloaded an example LIS from the TotalDepth GitHub, "DILLSON-1_WELL_LOGS_FILE-037.LIS"
I followed the "Reading a LIS File Log Data" example and my Python 3 code looks like:

import TotalDepth
from TotalDepth.LIS.core import File
from TotalDepth.LIS.core import FileIndexer
from TotalDepth.LIS.core import LogiRec
import os

fpath = os.path.expanduser('~/DILLSON-1_WELL_LOGS_FILE-037.LIS')
lis_file = TotalDepth.LIS.core.File.FileRead(fpath)
lis_index = TotalDepth.LIS.core.FileIndexer.FileIndex(lis_file)

log_passes = list(lis_index.genLogPasses())
print(log_passes[0].logPass.longStr())
log_passes[0].logPass.setFrameSet(lis_file)

This produces an error TotalDepth.LIS.core.LogPass.ExceptionLogPass: LogPass.setFrameSet(): no frames to load.

print(log_passes[0].logPass.longStr()) produces
<TotalDepth.LIS.core.LogPass.LogPass object at 0x7f6b1810fdf0>: DFSR: <TotalDepth.LIS.core.LogiRec.LrDFSRRead object at 0x7f6b18100910>: "Data format specification record" Frame plan: <TotalDepth.LIS.core.Type01Plan.FrameSetPlan object at 0x7f6b1810fe50>: indr=4 frame length=298 channels=104 Channels: [b'BS ', b'TOD ', b'TIME', b'ETIM', b'CS ', b'DIFF', .... b'CGR '] RLE: RLEType01: func=None: [] X axis: No data. Frame set: None
Is this expected? It happens with every LIS file that I try. I was expecting to be able to access the data as a Numpy array. I was also confused that it says X axis: No data.

I noticed in the example that it says:
"To read all the frame data from the file we call setFrameData(LisFile) on the log pass:"
>>> log_passes[0].logPass.setFrameSet(lis_file)
However, I don't think the setFrameData method exists.

Thanks for any pointers that you can provide.

Hello Paul
I'm a newbie to Python and LIS format processing (!), but learning quickly.
In the example .lis file I have, there are 8 log passes.
How do I assign the frame set correctly between all 8 log passes?
Your example in the docs is only for the case where there is 1 log pass, and the whole frame set goes to it.
Thanks in advance,
Dave

These are legacy files that support the Python unittest module. Since we now use pytest we can remove these.

Thanks for making TotalDepth.
I'm having an issue installing using:
pip3 install TotalDepth

The following error occurs during the build process

    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -c src/TotalDepth/LIS/core/src/cp/cpLISRepCode.cpp -o build/temp.linux-x86_64-3.6/src/TotalDepth/LIS/core/src/cp/cpLISRepCode.o -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -flto -fuse-linker-plugin -ffat-lto-objects -Isrc/TotalDepth/LIS/core/src/cpp/ -std=c++14
    src/TotalDepth/LIS/core/src/cp/cpLISRepCode.cpp:9:10: fatal error: cpLISRepCode.h: No such file or directory
     #include "cpLISRepCode.h"
              ^~~~~~~~~~~~~~~~
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Any help would be appreciated?

I have been trying to use Python pickling with the output from the LIS Indexer, e.g.

`
import pickle
from TotalDepth.LIS.core import File
from TotalDepth.LIS.core import FileIndexer
lisFile = File.FileRead('file.lis', keepGoing=True)
lisIndex = FileIndexer.FileIndex(lisFile)

pickled = pickle.dumps(lisIndex) # pickle the index
lisIndexLoad = pickle.loads(pickled) # this gives an error, new() takes 2 positional arguments but 5 were given
`

I am not very familiar with "Pickling" in Python, but I was hoping this would provide a method to store in lisIndex and reuse it? However, I have not seen an example of this in the documentation so perhaps it is not possible?

I am getting an error converting a folder of .lis files on running the function TotalDepth.LIS.core.File.FileRead(path_to_file)

three of the files give error

File "/anaconda3/lib/python3.7/site-packages/TotalDepth/LIS/core/FileIndexer.py", line 451, in init
fn = self._despatchLrType[lrTy]

KeyError: 78

the last one gives

File "/anaconda3/lib/python3.7/site-packages/TotalDepth/LIS/core/TifMarker.py", line 163, in reportError
raise ExceptionTifMarker(theMsg)

ExceptionTifMarker: TIF read() expected 0x98, got tell: 0x96, Shortfall: 0x2

Any ideas? The files are .lis files from the OGA database https://ndr.ogauthority.co.uk - I've uploaded one of them if it helps https://github.com/morgothsvoidpointer/hello-world/blob/master/11_30a-B11_dwl_DWL_MWD_317972493.lis

There are a lot of performance tests in tests/unit which really slow down unit testing. These should be moved to tests/integration.