Coder Social home page Coder Social logo

komle's Introduction

komle

komle, a python library for WITSML, uses PyXB to marshal/unmarshal xml files according to the schemas.

Some of the features are:

  • WITSML data model bindings for schema v1.4.1.1 and v2.0
    • Note just one version can be used in the same runtime, due to namespace collision
  • WITSML to dict, for use in a pandas dataframe or json
  • Unit converter based on witsmlUnitDict
  • Soap client to request data from a witsml server, according to the webservice description
  • Validation that xml files conforms to the WITSML schema
  • Support for the generated write schemas, to be used for WMLS_AddToStore
    • Note that write_bindings can not be imported at the same time as read_bindings. See below for details.

Install

pip3 install git+ssh://[email protected]/kle043/komle.git

Or if the repo is cloned

pip3 install -U .

Getting started

from komle.bindings.v1411.read import witsml
from komle import utils as ku
import pandas as pd # Not part of komle setup

with open('log.xml', 'r') as log_file:
    # logs is a regular python object according to the witsml schema
    logs = witsml.CreateFromDocument(log_file.read())

# Print the witsml documentation for logs
print(logs._element().documentation())

# Print the schema location for logCurveInfo, nice to have for reference
print(logs.log[0].logCurveInfo[0]._element().xsdLocation().locationBase)

print([l.name for l in logs.log])

# Convert logdata to a dict
log = logs.log[0]

data_dict = ku.logdata_dict(log)

# Create a dataframe, if you have installed pandas
df_data = pd.DataFrame(data_dict)

# Do the same for the plural logCurveInfo element
df_curve = pd.DataFrame(ku.plural_dict(log.logCurveInfo))

witsml.CreateFromDocument works on any witsml object, like trajectorys, mudLogs, tubulars etc, and returns a python representation according to the schema. Nodes are converted to there corresponding python types and accessed like any other python object, the exception is leaf nodes with attributes where one must call value() since primitive types in python does not have custom attributes. For example mdTop.value() where mdTop also has the attribute mdTop.uom, also see examples/hello_witsml.py.

Usage of different schemas

The difference between the schemas is described here. In summary,

  • Read Schemas: [...] a copy of the normative files except that all choices, elements and attributes are optional. [...] these schema files must represent the XMLout response from the WITSML WMLS_GetFromStore method.
  • Write Schemas: [...] a copy of the normative files except that some unique identifier attributes have had their optionality changed. [...] these schema files must represent the XMLin input to the WITSML WMLS_AddToStore method.
  • Update Schemas (not currently supported): [...] a copy of the normative files with all elements and attributes optional except that all unique identifier attributes and uom attributes are mandatory. [...] these schema files must represent the XMLin input to the WITSML WMLS_UpdateInStore method.
  • Delete Schemas (not currently supported): [...] a copy of the normative files with all elements and attributes optional except for all object uids and parentage-pointers which are mandatory. [...] these schema files must represent the QueryIn input to the WITSML WMLS_DeleteFromStore method.

As a practical matter, any program needing to work on both read and write (and update/delete) should only import the read bindings, since they have the least restrictions. The read bindings will be valid also for write/update/delete, as long as the mandatory elements and attributes are present. For validation of the write schema, a separate test program which only imports the write bindings must be used. (Note that the multiprocessing module can not easily be used for this purpose, since the child processes will inherit the parent process' imports.)

komle's People

Contributors

kle043 avatar migueldb avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

komle's Issues

Filenames need updating

Hell there

I was looking at the following example and in order to get it to run I need to update the filenames:

examples/hello_witsml2.py

It works if I point them at the filenames with a (Case Conflict) suffix.

Interestingly, it's loading komle.bindings.v20 but the .xml files have a <eml:Aliases authority="1.4.1.1">' property

Object type 'ns1:<>' is not supported

I am trying to query data from a WITSML server using StoreClient. I have validated that the url, credentials, and endpoints(wells, wellbores, etc) work by utilizing PDS WITSMLstudio.

I've adapted the example at the end of hello_witsml.py:

import os
import pyxb
from komle.bindings.v1411.read import witsml
from komle.soap_client import StoreClient
from komle.uom_converter import conversion_factor, get_unit
from komle import utils as ku
import pandas as pd # Not included in komle setup.py
from lxml import etree


# This will fail if you don't have credentials to a witsml server
store_client = StoreClient(service_url='https://aaa.bbbbb.com/WMLS/WMLSService.svc',
                           username='[email protected]',
                           password='someAwesomePassword')

id_logs = store_client.get_logs(witsml.obj_log())
print(id_logs)

wellbores = store_client.get_wellbores(witsml.obj_wellbore())
print(wellbores)

id_logs returns komle.soap_client.StoreException: -101: Object type 'ns1:logs' is not supported.

and

wellbores returns komle.soap_client.StoreException: -101: Object type 'ns1:wellbores' is not supported.

Again, I've validated via WITSMLstudio that both logs and wellbores are accessible with my URL and credentials.

Thanks for any help you can provide!

Unable to convert DOM node

Hi there

I'm trying out your code to see if I can import some Volve data but I'm not able to read in a file without getting the following error:

Unable to convert DOM node {http://www.witsml.org/schemas/1series}priv_objectGrowing at <unknown>[1:4462] to binding

Some additional info:

Exception has occurred: UnrecognizedContentError
Invalid content priv_objectGrowing at <unknown>[1:4462] (expect {http://www.witsml.org/schemas/1series}itemState or {http://www.witsml.org/schemas/1series}serviceCategory or {http://www.witsml.org/schemas/1series}comments or {http://www.witsml.org/schemas/1series}acquisitionTimeZone or {http://www.witsml.org/schemas/1series}defaultDatum or {http://www.witsml.org/schemas/1series}privateGroupOnly or {http://www.witsml.org/schemas/1series}extensionAny or {http://www.witsml.org/schemas/1series}extensionNameValue)
  File "/tmp/build/80754af9/python_1573076469108/work/Modules/pyexpat.c", line 461, in EndElement

Looking a bit closer, it appears that there's a bunch of tags with a priv_ prefix that are casing the issue. Is it possible to add a try except for UnrecognizedContentError if there's an unofficial tag and have these tags ignored?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.