emmo-repo / emmontopy

Library for representing and working with ontologies in Python

EMMOntoPy

Library for representing and working with ontologies in Python.

EMMOntoPy is a Python package based on the excellent Owlready2, which provides a natural and intuitive representation of ontologies in Python. EMMOntoPy extends Owlready2 and adds additional functionality, like accessing entities by label, reasoning with FaCT++ and parsing logical expressions in Manchester syntax. It also includes a set of tools, like creating an ontology from an Excel sheet, generation of reference documentation of ontologies and visualisation of ontologies graphically. EMMOntoPy is freely available for on GitHub and on PyPI under the permissive open source BSD 3-Clause license.

EMMOntoPy was originally developed to work effectively with the Elemental Multiperspective Material Ontology (EMMO) and EMMO-based domain ontologies.
It has now two sub-packages, ontopy and emmopy, where ontopy is a general package to work with any OWL ontology, while emmopy provides extra features that are specific to EMMO.

Owlready2, and thereby also EMMOntoPy, represents OWL classes and individuals in Python as classes and instances. OWL properties are represented as Python attributes. Hence, it provides a new dot notation for representing ontologies as valid Python code. The notation is simple and easy to understand and write for people with some knowledge of OWL and Python. Since Python is a versatile programming language, Owlready2 does not only allow for representation of OWL ontologies, but also to work with them programmatically, including interpretation, modification and generation. Some of the additional features provided by EMMOntoPy are are listed below:

Access by label

In Owlready2 ontological entities, like classes, properties and individuals are accessed by the name-part of their IRI (i.e. everything that follows after the final slash or hash in the IRI). This is very inconvenient for ontologies like EMMO or Wikidata, that identify ontological entities by long numerical names. For instance, the name-part of the IRI of the Atom class in EMMO is ‘EMMO_eb77076b_a104_42ac_a065_798b2d2809ad’, which is neither human readable nor easy to write. EMMOntoPy allows to access the entity via its label (or rather skos:prefLabel) ‘Atom’, which is much more user friendly.

Turtle serialisation/deserialisation

The Terse RDF Triple Language (Turtle) is a common syntax and file format for representing ontologies. EMMOntoPy adds support for reading and writing ontologies in turtle format.

FaCT++ reasoning

Owlready2 has only support for reasoning with HermiT and Pellet. EMMOntoPy adds additional support for the fast tableaux-based [FaCT++ reasoner] for description logics.

Manchester syntax

Even though the Owlready2 dot notation is clear and easy to read and understand for people who know Python, it is a new syntax that may look foreign for people that are used to working with Protégé. EMMOntoPy provides support to parse and serialise logical expressions in Manchester syntax, making it possible to create tools that will be much more familiar to work with for people used to working with Protégé.

Visualisation

EMMOntoPy provides a Python module for graphical visualisation of ontologies. This module allows to graphically represent not only the taxonomy, but also restrictions and logical constructs. The classes to include in the graph, can either be specified manually or inferred from the taxonomy (like all subclasses of a give class that are not a subclass of any class in a set of other classes).

Tools

EMMOntoPy includes a small set of command-line tools implemented as Python scripts:

• ontoconvert: Converts ontologies between different file formats. It also supports some additional transformation during conversion, like running a reasoner, merging several ontological modules together (squashing), rename IRIs, generate catalogue file and automatic annotation of entities with their source IRI.
• ontograph: Vertasile tool for visualising (parts of) an ontology, utilising the visualisation features mention above.
• ontodoc: Documents an ontology.
• excel2onto: Generate an EMMO-based ontology from an excel file. It is useful for domain experts with limited knowledge of ontologies and that are not used to tools like Protégé.
• ontoversion: Prints ontology version number.
• emmocheck: A small test framework for checking the consistency of EMMO and EMMO-based domain ontologies and whether they confirm to the EMMO conventions.

Some examples of what you can do with EMMOntoPy includes:

• Access and query RDF-based ontologies from your application. This includes several different flavors of RDF (OWL, Turtle (ttl), and more).

• Access and query EMMO-based ontologies from your application.

• Extend EMMO with new domain or application ontologies. This can be done both statically with easy readable Python code or dynamically within your application.

• Generate graphs and documentation of your ontologies. EMMOntoPy includes ontodoc: A dedicated command line tool for this. You find it in the tools/ sub directory.

• Check that an EMMO-based domain or application ontology adhere to the conventions of EMMO.

• Interactively explore an ontology in any Python interpreter, e.g., IPython. Tab-completion makes exploration easy and fast. Below is an example of an IPython session where we check the relations of Matter in EMMO utilizing the emmopy.get_emmo function:

  In [1]: from emmopy import get_emmo
  
  In [2]: emmo = get_emmo()
  
  In [3]: emmo.Matter
  Out[3]: physicalistic.Matter
  
  In [4]: emmo.Matter.is_a
  Out[4]:
  [physicalistic.Physicalistic,
    physical.Physical,
    mereotopology.hasPart.some(physicalistic.Massive),
    physical.hasTemporalPart.only(physicalistic.Matter)]
  

Documentation and examples

The Owlready2 documentation is a good starting point. The EMMOntoPy package also has its own dedicated documentation.

This includes a few examples and demos:

• demo/vertical shows an example of how EMMO may be used to achieve vertical interoperability. The file define-ontology.py provides a good example for how an EMMO-based application ontology can be defined in Python.

• demo/horizontal shows an example of how EMMO may be used to achieve horizontal interoperability. This demo also shows how you can use EMMOntoPy to represent your ontology with the low-level metadata framework DLite. In addition to achieve interoperability, as shown in the demo, DLite also allow you to automatically generate C or Fortran code base on your ontology.

• examples/emmodoc shows how the documentation of EMMO is generated using the ontodoc tool.

Installation

Install with:

pip install EMMOntoPy

Required Dependencies

• Python 3.7 or later.
• Owlready2 v0.23 or later.

Optional Dependencies

• Graphviz: Needed for graph generation. With support for generation pdf, png and svg figures for tests and generation of documentation automatically (ontodoc).

• pandoc: Only used for generated documentation from markdown to nicely formatted html or pdf. Tested with v2.1.2.

• pdfLaTeX or XeLaTeX and the upgreek LaTeX package (included in texlive-was on RetHat-based distributions and texlive-latex-extra on Ubuntu) for generation of pdf documentation. If your ontology contains exotic unicode characters, we recommend XeLaTeX.

• Java. Needed for reasoning.

• Optional Python packages:

  • graphviz: Generation of documentation and graphs.
  • PyYAML: Required for generating documentation with pandoc.
  • blessings: Clean output for emmocheck.
  • Pygments: Coloured output for emmocheck.
  • rdflib: Required for ontoversion-tool.
  • semver: Required for ontoversion-tool.
  • pydot: Used for generating graphs. Will be deprecated.
  • pyparsing: Used for parsing Manchester syntax

See docs/docker-instructions.md for how to build a docker image.

Known issues

• Invalid serialising to turtle: Due to rdflib issue #1043 ontoconvert may produce invalid turtle output (if your ontology contains real literals using scientific notation without a dot in the mantissa). This issue was fixed after the release of rdflib 5.0.0. Hence, install the latest rdflib from PyPI (pip install --upgrade rdflib) or directly from the source code repository: GitHub if you need to serialise to turtle.

Attributions and credits

EMMOntoPy is maintained by EMMC-ASBL. It has mainly been developed by SINTEF, specifically:

• Jesper Friis (jesper-friis)
• Francesca L. Bleken (francescalb)
• Casper W. Andersen (CasperWA)
• Bjørn Tore Løvfall (lovfall)

Contributing projects

• EMMC-CSA; Grant Agreement No: 723867

  The EMMC-ASBL organization takes on the efforts of continuing and expanding on the efforts of the CSA.

• MarketPlace; Grant Agreement No: 760173

• OntoTrans; Grant Agreement No: 862136

• BIG-MAP; Grant Agreement No: 957189

• OpenModel; Grant Agreement No: 953167

emmontopy's Issues

Make fact++ reasoner available and default in tools

I am not sure if this is already implemented and needs only to be updated documentation.

Add support for simple property-based ontology annotations like dcterms:license

Unable to use the vertical interoperability demo .py files

Hello,

I'm using Pycharm to run the codes, my OS is win10, and the Python version is 3.8.3.
When running the "define_ontology.py" file, it shows the error message:

Traceback (most recent call last):
File "C:/Users/Choc/PycharmProjects/pythonProject/define_ontology.py", line 196, in
class WeldedComponent(emmo.Component):
File "C:\Users\Choc\anaconda3\lib\site-packages\emmo\ontology.py", line 110, in getattr
attr = self.get_by_label(name)
File "C:\Users\Choc\anaconda3\lib\site-packages\emmo\ontology.py", line 420, in get_by_label
raise NoSuchLabelError('Ontology "%s" has no such label: %s' % (
emmo.ontology.NoSuchLabelError: Ontology "emmo-inferred" has no such label: Component

I actually have no idea how to deal with that, so I comment out the "class WeldedComponent(emmo.Component):" part first in order to make it runs well. Then, I keep running the "plot_ontology.py" file, but it still shows error message no matter the "define_ontology.py" file runs well or not:

Traceback (most recent call last):
File "C:/Users/Choc/PycharmProjects/pythonProject/plot_ontology.py", line 17, in
graph = onto.get_graph(entities=classes, relations='all',
File "C:\Users\Choc\anaconda3\lib\site-packages\emmo\ontology.py", line 672, in get_graph
return OntoGraph(self, **kwargs)
File "C:\Users\Choc\anaconda3\lib\site-packages\emmo\graph.py", line 227, in init
self.add_entities(entities=entities, relations=relations,
File "C:\Users\Choc\anaconda3\lib\site-packages\emmo\graph.py", line 242, in add_entities
self.add_nodes(entities, nodeattrs=nodeattrs, **attrs)
File "C:\Users\Choc\anaconda3\lib\site-packages\emmo\graph.py", line 303, in add_nodes
self.add_node(name, nodeattrs=nodeattrs, **attrs)
File "C:\Users\Choc\anaconda3\lib\site-packages\emmo\graph.py", line 297, in add_node
self.dot.node(label, label=label, **kw)
File "C:\Users\Choc\anaconda3\lib\site-packages\graphviz\dot.py", line 130, in node
name = self._quote(name)
File "C:\Users\Choc\anaconda3\lib\site-packages\graphviz\lang.py", line 72, in quote
if is_html_string(identifier) and not isinstance(identifier, NoHtml):
TypeError: expected string or bytes-like object

Any advice is appreciated!

UPDATE: I try to run these codes on Jupyter Notebook. And the "define_ontology.py" file runs well (I have no idea why).
But the "plot_ontology.py" file still shows the same TypeError😥. Please help.

Ontograph: Include instances in graph, currently it fails if ontology contains instances. Color instances differently.

do something

Add reasoning with FaCT++

Update publish workflow

Utilize the dedicated PyPI publish action and run only on published GitHub releases.

Provide 'support' for same entities with different namespaces

We have to decide on how to tackle this problem and what EMMOpython should support and not.
As of now the ontology.get_by_label_all(xxx), will return all different xxx, also with different namespaces, but ontology.xxx will only return the first it finds.

If both are present in different namespaces it should at least give a warning, and some way to decide which one to use.

Alternatively it should be impossible to import different domain ontologies if they have overlapping terms.

Check Error with Owlready2-0.26

Currently Owlready2 dependency to have version 0.25 has version 0.26 gives and error in namespace.

Fix emmodoc

This requires updating ontograpy.py

Update the user case ontology

A some classes needs to be refined:

• 'phase' and 'rve' are currently defined in the same way. What are the differences?
• Is 'fem_unit_cell' a model? I guess yes. Would 'fem_element' be a better name?

Use rdflib in Ontology.save() to support more file formats

Currently will onto.save('myonto.ttl') save in rdfxml format instead of turtle.

Tool for publishing domain ontologies

Add a tool or set of tools that streamline the workflow of publishing an versioned ontology, like how EMMO is made available on GitHub pages: https://emmo-repo.github.io/

It would need a reasonable trade-off between flexibility and simplicity. Maybe it could be implemented as a single command-line tool with sub-commands like git.

licence does not work with metadata

onto.metadata.licence(append('LGPL/CC'))

Traceback (most recent call last):
File "periodic-table.py", line 115, in
onto.metadata.licence.append(en(
File "/home/francescab/python-environments-linux/emmopy/lib/python3.8/site-packages/owlready2/namespace.py", line 1239, in getattr
values = [self.namespace.ontology._to_python(o, d) for o, d in self.namespace.world._get_triples_sp_od(self.storid, Prop.storid)]
AttributeError: 'NoneType' object has no attribute 'storid'

Test the commented out line in examples/periodic-table.py in branch peridic-table-example

Remove deprecated emmo/ontograph.py that uses pydot

Missing https://emmc.info/emmo-inferred

Hi,
running demo/vertical/define_ontology.py ends up with an error. Minimum working example in Python3, just to demonstrate the problem.

from emmo import get_ontology
emmo = get_ontology()
emmo.load()
onto = get_ontology('onto.owl')
onto.imported_ontologies.append(emmo)
onto.sync_attributes()
onto.sync_reasoner()
... owlready2.base.OwlReadyJavaError: Java error message is: ... Could not load imported ontology: https://emmc.info/emmo-inferred.

I believe that the URI needs to be set differently (I believe should be more like http://www.emmc.info/emmc-csa/... ). I can get further if overloading base_iri='emmo-inferred.owl' in get_ontology() with e.g. base_iri='emmo-core.owl'. It would be also beneficial having ontology source on a local disk since many computers run on intranet without internet access.

Best regards. Vit

Ontograph: Include multiple parents/inheritance

Branches with several parents end up floating alone without connection to either of their parents in ontograph

Include all annotations in .get_annotations()

Currently, only 'comment's are returned in get_annotations.
It would be nice to get other annotations such as 'elucidation' etc

Add periodic_table to examples

Fix loading imported ttl from web such that emmocheck works for crystallography.ttl

Enhance ontology.sync_attributes() to also update class names

Add a keyword argument to control how the OWL name of OWL classes (typically added via Python) should be updated. Possible options could be:

• do not change name
• set name to prefix + "_" + uuid where prefix is any string (e.g. "EMMO") and uuid is
  an uuid. This is the default for EMMO.
• set name to the name of the corresponding Python class
• set name equal to labels

Update Dockerfile to install correct pandoc

Also, make sure that emmo-python is installed locally (pip install...) by the image

Ontograph: Make default root

Also use the catalog file to map web URLs, not only local files.

Make a list of tests that should function:
*Load locally with and without catalog
*Load ttl/owl
*Load from web with and without catalog

An issue is that the load function is too complicated and has too much 'ad-hoc' solutions. We need to sperate out different functions.

Suggeted overall 'algorithm':

1. Check catalogfile existence -> return a mapping dictionary name: url
2. Check format
3. if owl -> go to step ????
4. if not owl (but format supported by rdflib): -> convert to owl in local temp

Add support for the FaCT++ reasoner

Add support for the FaCT++ reasoner. Ideally this should be done in Owlready2 and not EMMO-python.

The HermiT and Pellet reasoners are very slow and requires a lot of memory when run on EMMO, while FaCT++ seems to be much faster.

Make EMMO-python available on pypi (installable with pip)

add --no-catalog and default as in emmocheck for ontograph

requried for using ontograph .... emmo-inferred directly

Emmocheck fails if Physicaluantities and MeaurementsUnits are not imported from emmo. Make sure that it does not fail if whole of EMMO is not imported.

setup.py

Hi,
in setup.py, please use setuptools instead of distutils.core. Distutils.core is obsolete and does not install owl files, unable to run examples under tests/. I attach both setup.py and MANIFEST files working for me, installing necessary files.

setup+MANIFEST.zip

Best regards. Vit

fix redudant getlabel, get_preferred_label, get_label

There is a getlabel in ontogaph.py, get_preferred_label in patch and it is imported as get_label in emmocheck.

Suggestion:
Move get_preferred_label to utils, make a get_label that imports get_preferred_label. Add getlabel funciontality and use then get_label in ontograph.
Check where_else it might be in use.

Update documentation of tools re reasoner

Ontograph documentation states Hermit and Pellet as available reasoners and Hermit as default. Update documentation to Fact++ (default).

Add Wu&Palmer measure

Check that it is not in Owlready2 first

Correctly load ontologies like crystallography that imports both local and online sub-ontologies

Fix flake8 errors

Import emmo .ttl from emmo-repo.github.io

Importing .ttl works well when the ontology is cloned on the local computer. It should be possible to import .ttl directly from web as well.

ontograph with several roots fails

when several roots the name of the file is made by joining root with head and tail. root is not a string. Use get_label.

Update emmocheck test_namespaces to newer emmo-inferred

New rules for namespaces in emmo require updates in test_namespace

AttributeError from ontodoc

I am trying to run ontodoc on an emmo compliant ontology, but get the following error:
Traceback (most recent call last):
File "/home/emmo/.local/bin/ontodoc", line 141, in
docpp = main()
File "/home/emmo/.local/bin/ontodoc", line 127, in main
docpp.process()
File "/home/emmo/.local/lib/python3.8/site-packages/emmo/ontodoc.py", line 813, in process
self.process_alls()
File "/home/emmo/.local/lib/python3.8/site-packages/emmo/ontodoc.py", line 741, in process_alls
self.lines[i: i] = self.ontodoc.itemsdoc(
File "/home/emmo/.local/lib/python3.8/site-packages/emmo/ontodoc.py", line 341, in itemsdoc
doc.append(self.itemdoc(item, header_level))
File "/home/emmo/.local/lib/python3.8/site-packages/emmo/ontodoc.py", line 241, in itemdoc
'', level=header_level, label=label, lowerlabel=label.lower()))
AttributeError: 'NoneType' object has no attribute 'lower'

Specify version ranges for dependencies

It would be safer and more helpful for dependencies of EMMO if

1. Minimum versions are specified for dependencies; and
2. A range of versions are allowed for dependencies.

The latter can be achieved either via ~= or >=X.Y,<Z, where, for example, Z = X + 1.

Not immediately installable with pip

When installing this with the pip package manager, it fails due to the import of itself (emmo).

This can be fixed by avoiding the import of itself and instead retrieve the necessary information "manually", i.e., through raw file access, scanning, and parsing.

Ontodoc: PDF is not generated, produces error.

Remove the .ttl namespace when loading domain-crystallography in EMMO-python

Windows paths are not handled properly

From CIF-ontology, I tried to run generate_cif.py locally on my Windows 10 environment with Python 3.8.

First, an issue arised with Windows paths for the PyCIFRW dependency, which has been reported and subsequently fixed.
When running the script now, it fails for similar reasons (Windows path handling) via the this dependency, specifically in the line self.top = self.world.get_ontology(cif_top).load(), where self.world = emmo.World().

Full traceback:

C:\Users\c_w_a\Documents\GitHub\CIF-ontology\cif_top.ttl does not look like a valid URI, trying to serialize this will break.
Traceback (most recent call last):
  File ".\dic2owl\dic2owl\generate_cif.py", line 211, in <module>
    self = gen = main()
  File ".\dic2owl\dic2owl\generate_cif.py", line 191, in main
    gen = Generator(dicfile="cif_core.dic", base_iri=base_iri)
    self.top = self.world.get_ontology(cif_top).load()
  File "C:\Users\c_w_a\.virtualenvs\EMMO\lib\site-packages\emmo\ontology.py", line 295, in load
    self._load(only_local=only_local, filename=filename, format=format,
  File "C:\Users\c_w_a\.virtualenvs\EMMO\lib\site-packages\emmo\ontology.py", line 397, in _load
    g.parse(resolved_url, format=fmt)
  File "C:\Users\c_w_a\.virtualenvs\EMMO\lib\site-packages\rdflib\graph.py", line 1062, in parse
    source = create_input_source(
  File "C:\Users\c_w_a\.virtualenvs\EMMO\lib\site-packages\rdflib\parser.py", line 193, in create_input_source
    input_source = URLInputSource(absolute_location, format)
  File "C:\Users\c_w_a\.virtualenvs\EMMO\lib\site-packages\rdflib\parser.py", line 113, in __init__
    file = urlopen(req)
  File "C:\Users\c_w_a\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\c_w_a\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 525, in open
    response = self._open(req, data)
  File "C:\Users\c_w_a\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 547, in _open
    return self._call_chain(self.handle_open, 'unknown',
  File "C:\Users\c_w_a\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 502, in _call_chain
    result = func(*args)
  File "C:\Users\c_w_a\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 1390, in unknown_open
    raise URLError('unknown url type: %s' % type)
urllib.error.URLError: <urlopen error unknown url type: c>

My setup:
Windows 10 (version 20H2, build 19042.928)
Python 3.8.1
Virtual environment through virtualenvironment-powershell.
Installed EMMO from PyPI (version 1.0.0)

While I normally use WSL 2.0 (Linux kernel in Windows), I would prefer not to for this particular project.

Convert-imported update in utils

Look at error in domain-ontologies for domain-atomistic

Homogenise call to reasoner in emmo.Ontology.sync_reasoner()

Pellet should be the default

make tests for checking upgrade of Owlready2

Add option of documenting imported ontologies in ontodoc and ontograph

Even though ontograph has no --imported option, it will plot imported ontologies. The reason for this is that the selection of classes to plot is not based on ontologies, but on branches.

Add semantics for dimensionality to the user case ontology in the demo

This will be very useful to e.g. semantically describe that the stiffness_tensor is a 6x6 array and not just 36 real numbers. We can assign a name (and possibly a length) to the dimensions directly in the ontology. This would fit perfect with SOFT/dlite. To make advantage of this, the onto2meta.py module should be updated.

Use PyPI token in publish workflow

As stated by @jesper-friis here we can now use the token to access the EMMO-Python package site on PyPI for publishing this package.

Correct turtle serialisation

rdflib prior to 5.0.0 serialises double literals as e.g. "1e12.0" if the mantissa has no point in it, which is invalid turtle. Requirering rdflib 5.0.0 should fix this issue by serialise the literal as "1e12".

See RDFLib/rdflib#1043