oeg-upm / morph-rdb

Virtual Knowledge Graph Creation from RDB with R2RML

DISCLAIMER: This software is no longer maintained, although still operational. If you are interested in a materialisation engine instead of a query rewriting engine, you can make use of Morph-KGC

Morph-RDB

Morph-RDB (formerly called ODEMapster) is an RDB2RDF engine developed by the Ontology Engineering Group, that follows the R2RML specification (http://www.w3.org/TR/r2rml/).

Morph-RDB supports two operational modes: data upgrade (generating RDF instances from data in a relational database) and query translation (SPARQL to SQL). Morph-RDB employs various optimisation techniques in order to generate efficient SQL queries, such as self-join elimination and subquery elimination.

Morph-RDB has been tested with real queries from various Spanish/EU projects and has proven to work faster than other state-of-the-art tools available. At the moment, Morph-RDB works with MySQL, PostgreSQL, H2, CSV files and MonetDB.

How to use

User guides: For those who want to use this project on an user level, you can find a little guide to on the main branch wiki : https://github.com/oeg-upm/morph-rdb/wiki

If, on the other hand, you want to edit the project or at least work from an IDE such as Eclipse, we suggest you to follow this steps:

• Download the source code.
• Once unziped, you may notice that the imports doesn´t match the actual directories. In order to avoid changing all the imports or all the directories, import this way: Import -> Maven -> Existing Maven Project, and select as root the folder where you unziped the project (it may take a few minutes).
• Now that it´s finally imported, you can run the file es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner.scala (in other to pass the arguments in Eclipse, right click, Run As -> Run Configuration -> Arguments, and remember to imput both the path to the .properties file and it´s full name).

• In case the program doesn´t find the file log4j.properties, move it from "morph-examples" to "morph-r2rml-rdb", thought this file isn´t essential.

Acknowledgement

From January 2016 to January 2018, the development of morph-RDB has been supported by the Mobile Age project (http://www.mobile-age.eu/). After that period, the development has been supported by the Datos 4.0 Spanish national proect.

Authors

• Freddy Priyatna

morph-rdb's Issues

.bat files of mysql-examples need to be updated

Hi, I have notice that the .bat files of the mysql examples point to a folder lib/, I think they should point to dependancy/ isn´t it?

Also .jar files of dependency folder need to be updated.

Kind regards

mapping generation does'nt work

Hi,

I tryed out your installation description at https://github.com/oeg-upm/morph-rdb/wiki.
But unfortunately generation of mappings does'nt work for me

cmd, I use is:
"java -cp .:morph.jar:lib/* es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner . morph.properties"

properties content is:
no_of_database=1
database.name[0]=xxx
database.driver[0]=com.mysql.jdbc.Driver
database.url[0]=jdbc:mysql://127.0.0.1:3306/
database.user[0]=xxx
database.pwd[0]=xxx
database.type[0]=mysql

I get the following exception:

Generating R2RML Mappings ...
es.upm.fi.dia.oeg.morph.r2rml.rdb.mappingsgenerator.exception.R2RMLException: Critical error in database schema.
at es.upm.fi.dia.oeg.morph.r2rml.rdb.mappingsgenerator.control.R2RMLProcess.R2RMLBuild(R2RMLProcess.java:535)
at es.upm.fi.dia.oeg.morph.r2rml.rdb.mappingsgenerator.main.R2RMLMapper.run(R2RMLMapper.java:238)
at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunnerFactory.createRunner(MorphBaseRunnerFactory.scala:51)

Do you have any idea?

Best regards,
Michael

❓ Optional `predicateObjectMap` if template's parameters are not NULL

Example table

foo	bar
sam
bob	key

R2RML configuration

<#MyMap>
    rr:logicalTable [ rr:tableName "my_table ];
    rr:subjectMap [ rr:template "https://example.com/{foo}" ];

   rr:predicateObjectMap [
       rr:predicate <https://example.com/schema#myPredicate>;
       rr:objectMap [ rr:template "https://example.com/{foo}/{bar}" ];
   ];
.

Question
I would like to create this object map only if bar is not NULL. Typically, the above would only generate the following triple:

<https://example.com/bob> <https://example.com/schema#myPredicate <https://example.com/bob/key>

At the moment, it generates the following triple which I don't want:

<https://example.com/sam> <https://example.com/schema#myPredicate <https://example.com/sam/>

Note: I, however, would like to keep all the other predicateObjectMaps not relying on bar. Any idea on how to do so?

Thank you very much!

Received error while using generator.

Hello,
I am trying to generate mappings for a given ttl file. I edited the batch.r2rml.properties file as such:

mappingdocument.file.path=TriplesMap_City.ttl
output.file.path=batch-result.nt

RDF/XML|RDF/XML-ABBREV|N-TRIPLE|TURTLE|N3

output.rdflanguage=N-TRIPLE

no_of_database=1
database.name[0]=xxxx
database.driver[0]=com.mysql.jdbc.Driver
database.url[0]=jdbc:mysql://127.0.0.1:3306/xxxx
database.user[0]=xxxx
database.pwd[0]=xxxx
database.type[0]=mysql

and received the following error after running ./batch.bat

java -cp morph.jar;lib/* es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner examples  batch.r2rml.properties

java.lang.NoClassDefFoundError: scala/Option
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
at java.lang.Class.privateGetMethodRecursive(Unknown Source)
at java.lang.Class.getMethod0(Unknown Source)
at java.lang.Class.getMethod(Unknown Source)
at sun.launcher.LauncherHelper.validateMainClass(Unknown Source)
at sun.launcher.LauncherHelper.checkAndLoadMain(Unknown Source)
Caused by: java.lang.ClassNotFoundException: scala.Option
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
... 7 more
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main"

The links within the ttl file are broken, so I don't know if this is related to that or not. Any suggestions?

Thanks!
Georgia

Call without properties file

Is there a way to call it without any properties file?
something like: r2rml.jar data.csv data.r2rml output.ttl.

Split properties file

Right now the property file includes configurations:

• That usually do not change (e.g., the database configuration)
• That change every time the program is run (e.g., the mapping document, output file)

These two types of information should be in different configuration files.

For example, for running all the examples the database configuration has to be updated in every file with the same data. And for creating a new example the database configuration has to be included, even if it is always the same.

Bad sql when column name is "id"

Hi, I tried morph-rdb and it was great until my column name is "id". When a column name is "id" in postgresql, the generated sql as below assigns a alias table prifix (like v_2063 )to this column without adding this alias to the table:
SELECT v_2063."id" AS "x",'http://www.my.org/data#hasValue'::text AS "uri_hasValue350741132",v_2063."kpivalue" AS "y",31078738::integer AS "mappingid_x",31771827::integer AS "mappingid_y"
FROM "public"."kpi71"
WHERE "kpivalue" IS NOT NULL
then database will failed to execute the sql, is this a bug?

Invalid SQL syntax

It looks like the sql query generated from a Sparql query has invalid syntax:

2015-01-12 08:39:04,894 [main] INFO  es.upm.fi.dia.oeg.morph.base.DBUtility$ - Executing query: SELECT device.model AS "modelName",device.manufacturer AS "manufacturerName",268877525 AS "mappingid_modelName",1415843874 AS "mappingid_manufacturerName"
FROM T_METS_DEVICE device,  ( SELECT device.manufacturer  AS "manufacturer"
FROM T_METS_DEVICE device
group by manufacturer
 ) v_5544 ON ((device.manufacturer = v_5544.manufacturer) AND (v_5544.manufacturer IS NOT NULL) AND (device.model IS NOT NULL) AND (device.manufacturer = v_5544.manufacturer))
,  ( SELECT device.id  AS "id",device.serial  AS "serial",device.name  AS "name",device.description  AS "description",device.ip  AS "ip",device.domain  AS "domain",device.disposition  AS "disposition",device.exposure  AS "exposure",device.status  AS "status",device.model  AS "model",device.latitude  AS "latitude",device.longitude  AS "longitude",device.elevation  AS "elevation"
FROM T_METS_DEVICE device
 ) v_3444 ON (((v_3444.serial IS NOT NULL) AND (datastream.device_id = v_3444.id)) AND (device.model IS NOT NULL))
,  ( SELECT device.model  AS "model",device.manufacturer  AS "manufacturer"
FROM T_METS_DEVICE device
group by model
 ) v_7335 ON ((((v_7335.manufacturer IS NOT NULL) AND (v_7335.model IS NOT NULL) AND (device.model = v_7335.model)) AND ((device.manufacturer = v_7335.manufacturer) AND (device.model = v_7335.model))) AND (device.model IS NOT NULL))
WHERE device.manufacturer IS NOT NULL

2015-01-12 08:39:04,908 [main] ERROR R2RMLRunner - Exception occured: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'ON ((device.manufacturer = v_5544.manufacturer) AND (v_5544.manufacturer IS NOT ' at line 5
Exception in thread "main" com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'ON ((device.manufacturer = v_5544.manufacturer) AND (v_5544.manufacturer IS NOT ' at line 5
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
    at com.mysql.jdbc.Util.getInstance(Util.java:386)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1052)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3609)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3541)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2002)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2163)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2618)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2568)
    at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:842)
    at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:681)
    at es.upm.fi.dia.oeg.morph.base.DBUtility$.execute(DBUtility.scala:71)
    at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBDataSourceReader.execute(MorphRDBDataSourceReader.scala:19)
    at es.upm.fi.dia.oeg.morph.base.querytranslator.engine.QueryResultTranslator$$anonfun$translateResult$1.apply(QueryResultTranslator.scala:25)
    at es.upm.fi.dia.oeg.morph.base.querytranslator.engine.QueryResultTranslator$$anonfun$translateResult$1.apply(QueryResultTranslator.scala:21)
    at scala.collection.immutable.Map$Map1.foreach(Map.scala:116)
    at es.upm.fi.dia.oeg.morph.base.querytranslator.engine.QueryResultTranslator.translateResult(QueryResultTranslator.scala:21)
    at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner.run(MorphBaseRunner.scala:171)
    at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner$.main(MorphRDBRunner.scala:72)
    at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner.main(MorphRDBRunner.scala)

Logical tables are defined as sql queries in the R2RML file:

<TriplesMapProductManufacturer>
  a rr:TriplesMap;

  rr:logicalTable [ 
        rr:sqlQuery  """SELECT device.manufacturer manufacturer FROM T_METS_DEVICE device GROUP BY manufacturer""" 
    ];

  rr:subjectMap [    
    a rr:Subject;
    rr:template "http://linkeddata.metscale.es/resource/Manufacturer/{manufacturer}";
    rr:termType rr:IRI;
    rr:class s:Organization;
  ];     

  rr:predicateObjectMap [ 
    rr:predicateMap [ rr:constant s:name ]; 
    rr:objectMap    [ rr:column "manufacturer"; rr:termType rr:Literal; ]; 
  ]; 
.

<TriplesMapProductModel>
  a rr:TriplesMap;

  rr:logicalTable [
        rr:sqlQuery  """SELECT device.model model, device.manufacturer manufacturer FROM T_METS_DEVICE device GROUP BY model""" 
    ];

  rr:subjectMap [    
    a rr:Subject;
    rr:template "http://linkeddata.metscale.es/resource/Manufacturer/{manufacturer}/Model/{model}";
    rr:termType rr:IRI;
    rr:class s:ProductModel;
  ];

  rr:predicateObjectMap [ 
    rr:predicateMap [ rr:constant s:name ]; 
    rr:objectMap    [ rr:column "model"; rr:termType rr:Literal; ]; 
  ];

  rr:predicateObjectMap [ 
    rr:predicateMap [ rr:constant s:manufacturer ]; 
    rr:objectMap    [ 
       rr:parentTriplesMap <TriplesMapProductManufacturer>;
       rr:joinCondition [ rr:child "manufacturer" ; rr:parent "manufacturer" ; ]
    ];
  ];        
.

And the SPARQL query is:

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX iec: <http://linkeddata.metscale.es/ontology/IndustrialEnergyConsumption#>
PREFIX s: <http://schema.org/>
PREFIX ssn: <http://purl.oclc.org/NET/ssnx/ssn#>

SELECT ?sensorName ?modelName ?manufacturerName
WHERE {
    ?sensor a iec:PowerAnalyzer_ActiveEnergy .
    ?sensor ssn:onPlatform ?platform .
    ?platform s:model ?model .
    ?model s:manufacturer ?manufacturer .
    ?model s:name ?modelName .
    ?manufacturer s:name ?manufacturerName
}

/cc @fpriyatna @ocorcho

R2RMLTermMap.isBlankNode erroneous test

Hi Freddy,

A small bug that probably nobody has seen so far as it concerns blank nodes:
def isBlankNode(): Boolean = {
Constants.R2RML_BLANKNODE_URI.equals(this.termType)
}

this.termType is an option, so it should be
Constants.R2RML_BLANKNODE_URI.equals(this.termType.get)
Otherwise the test always returns false.

In morph-rdb this method is never used anyway... so probably the method can even be removed?

Franck.

Some JAVA code examples?

Hi, is it possible to share a simple java example how this tool can be used within java code? I am continuously running into errors. Here is what I tried

	MorphRDBProperties properties = 
			MorphRDBProperties.apply(fileFolder, propFileName);

	MorphRDBRunnerFactory runnerFactory = new MorphRDBRunnerFactory();
	MorphBaseRunner r2rmlRunner = runnerFactory.createRunner(properties);
	
	System.out.println("run result : " + r2rmlRunner.run()+"\n\n");   // results in "success"

	MorphBaseResultSet resultSet = r2rmlRunner.getQueryResultWriter().resultSet();
	System.out.println(resultSet.getColumnName(1));                      //shows the right string
	System.out.println("object = " + resultSet.getObject(1));             // leads to errors

Thanks for any support!

🐛 Columns are not extracted from [ rr:graphMap [ rr:template ] ]

Hi there 👋 A found a small issue with graphMap.

Example

<#AnswersTriplesMap>
    # Note: we need to explicitly JOIN the assessments' table so that we can get `care_recipient_id` to
    #       generate the correct graph URI.
    rr:logicalTable [ rr:tableName "my_table" ];
    rr:subjectMap [
        rr:template "https://example.com/{identifier}";
        rr:graphMap [ rr:template "https://example.com/score/{customer_id}" ];
    ];

Expected result
The customer_id column is selected and the graph is set.

Actual result
The customer_id column is not selected and this leads to the following error:

[main] ERROR es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBNodeGenerator - error occured when translating result: The column name V_71150_customer_id was not found in this ResultSet.

Workaround
There is one, yay! Using a predicateObjectMap to explicitly fetch customer_id make the graphMap work.

Percent Decoding in Query Translator

[Reported by Franck Michel]

Query Translator does not take into account the percent decoded value when translating URIs.

Given a mapping: http://example.org/{COLUMN_NAME}", an URI http://example.org/Some%20Value is translated into "... WHERE COLUMN_NAME = "Some Value"

link to R2RML spec

Hi,

A quick comment. In the description of the project.

The link http://www.w3.org/2001/sw/rdb2rdf/r2rml/ is to the draft version of R2RML. I guess you want to link it to the latest R2RML spec: http://www.w3.org/TR/r2rml/ :-)

Morph-rdb on Windows for MSSQL db

Hi All

Has anyone tried to run Morph-rdb on windows with connection to MS SQL server for query re-writes, then kindly share the info.

From this https://github.com/oeg-upm/morph-rdb/wiki/Manual-Installation, its not clear how/what to run to generate R2ML mappings and how to run them.

Thanks

SPARQL sub query feature

Hi,

May I ask a if there is any example like sparql sub query? I tried to write a sparql query which includes a sub query in my case. But the result generated is not correct. The query looks like below.

SELECT ?structure ?formula
WHERE{
?structure rdf:type core:Structure;
structure:Composition ?composition.
?composition structure:formula ?formula.
{
select ?structure
where{
?structure structure:Composition ?composition1;
structure:formula ?formula1.
Filter (?formula1 in ("MnCl2", "YClO"))
}
}
}

Thank you in advance.

Best,
Huanyu

test on different scenarios (next on empty iterator error) and (ERROR: missing FROM-clause entry)

Hello,


I am developing a benchmark for OBDA and Chase systems, ForBackBench. ForBackBench integrates 7 scenarios, and I am trying to execute SPARQL queries using Morph-RDB. Some queries worked perfectly, and I got the results. However, I got errors on others. Here is an example of these queries from OWL2Bench scenario: Q1.txt is the full query, Q2.txt I tried to shrink the query in Q1:

Files:
Note:Extension of the files was changed (or added) to .txt so it can be uploaded to github.
gav-mapping.r2rml.ttl.txt
Q1.txt
configQ1.r2rml.properties.txt
Q2.txt
configQ2.r2rml.properties copy.txt

Case 1 (Q1.txt file):

[main] INFO es.upm.fi.dia.oeg.morph.rdb.querytranslator.MorphRDBQueryTranslator - 
this.mapInferredTypes = ?0 -> Set()
?3 -> Set()
?1 -> Set(, )
?2 -> Set()
java.util.NoSuchElementException: next on empty iterator
	at scala.collection.Iterator$$anon$2.next(Iterator.scala:39)
	at scala.collection.Iterator$$anon$2.next(Iterator.scala:37)
	at es.upm.fi.dia.oeg.morph.base.querytranslator.MorphBaseQueryTranslator$$anonfun$22.apply(MorphBaseQueryTranslator.scala:1466)
	at es.upm.fi.dia.oeg.morph.base.querytranslator.MorphBaseQueryTranslator$$anonfun$22.apply(MorphBaseQueryTranslator.scala:1464)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
	at scala.collection.Iterator$class.foreach(Iterator.scala:891)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
	at scala.collection.MapLike$DefaultKeySet.foreach(MapLike.scala:174)
	at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
	at scala.collection.AbstractSet.scala$collection$SetLike$$super$map(Set.scala:47)
	at scala.collection.SetLike$class.map(SetLike.scala:92)
	at scala.collection.AbstractSet.map(Set.scala:47)
	at es.upm.fi.dia.oeg.morph.base.querytranslator.MorphBaseQueryTranslator.translate(MorphBaseQueryTranslator.scala:1464)
	at es.upm.fi.dia.oeg.morph.base.querytranslator.MorphBaseQueryTranslator.translate(MorphBaseQueryTranslator.scala:1514)
	at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner$$anonfun$1.apply(MorphBaseRunner.scala:218)
	at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner$$anonfun$1.apply(MorphBaseRunner.scala:216)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
	at scala.collection.immutable.List.foreach(List.scala:392)
	at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
	at scala.collection.immutable.List.map(List.scala:296)
	at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner.translateSPARQLQueriesIntoSQLQueries(MorphBaseRunner.scala:216)
	at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner.run(MorphBaseRunner.scala:182)
	at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner$.main(MorphRDBRunner.scala:82)
	at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner.main(MorphRDBRunner.scala)
[main] ERROR es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner$ - Exception occured: next on empty iterator
Exception in thread "main" java.util.NoSuchElementException: next on empty iterator
	at scala.collection.Iterator$$anon$2.next(Iterator.scala:39)
	at scala.collection.Iterator$$anon$2.next(Iterator.scala:37)
	at es.upm.fi.dia.oeg.morph.base.querytranslator.MorphBaseQueryTranslator$$anonfun$22.apply(MorphBaseQueryTranslator.scala:1466)
	at es.upm.fi.dia.oeg.morph.base.querytranslator.MorphBaseQueryTranslator$$anonfun$22.apply(MorphBaseQueryTranslator.scala:1464)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
	at scala.collection.Iterator$class.foreach(Iterator.scala:891)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
	at scala.collection.MapLike$DefaultKeySet.foreach(MapLike.scala:174)
	at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
	at scala.collection.AbstractSet.scala$collection$SetLike$$super$map(Set.scala:47)
	at scala.collection.SetLike$class.map(SetLike.scala:92)
	at scala.collection.AbstractSet.map(Set.scala:47)
	at es.upm.fi.dia.oeg.morph.base.querytranslator.MorphBaseQueryTranslator.translate(MorphBaseQueryTranslator.scala:1464)
	at es.upm.fi.dia.oeg.morph.base.querytranslator.MorphBaseQueryTranslator.translate(MorphBaseQueryTranslator.scala:1514)
	at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner$$anonfun$1.apply(MorphBaseRunner.scala:218)
	at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner$$anonfun$1.apply(MorphBaseRunner.scala:216)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
	at scala.collection.immutable.List.foreach(List.scala:392)
	at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
	at scala.collection.immutable.List.map(List.scala:296)
	at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner.translateSPARQLQueriesIntoSQLQueries(MorphBaseRunner.scala:216)
	at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner.run(MorphBaseRunner.scala:182)
	at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner$.main(MorphRDBRunner.scala:82)
	at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner.main(MorphRDBRunner.scala)

Case 2 (Q2.txt file): I tried to shrink the query in Q1 by removing the (rdf:type) from the query and this time I got this error:

[main] INFO es.upm.fi.dia.oeg.morph.base.DBUtility$ - Executing query:
SELECT  DISTINCT V_8490."var_1" AS "1",V_8490."mappingid_1" AS "mappingid_1"
FROM  ( SELECT V_23573."c0" AS "var_0",1336775847::integer AS "mappingid_0",V_23573."c1" AS "var_1",1998137093::integer AS "mappingid_1"
FROM (SELECT DISTINCT AA."c0" as c0, EA."c1" as c1 FROM  "src_Dean" as AA, "src_Specification" as BA, "src_Humanities" as CA, "src_Chair" as DA, "src_isStudentOf" as EA WHERE AA."c0" = BA."c0" AND BA."c0" = CA."c0" AND CA."c0" = DA."c0" AND DA."c0" = EA."c0" ) V_23573
WHERE V_23573."c1" IS NOT NULL

UNION
SELECT V_42602."c1" AS "var_0",1335061928::integer AS "mappingid_0",V_42602."c0" AS "var_1",1108706191::integer AS "mappingid_1"
FROM (SELECT DISTINCT AA."c0" as c0, EA."c1" as c1 FROM  "src_Dean" as AA, "src_Specification" as BA, "src_Humanities" as CA, "src_Chair" as DA, "src_isStudentOf" as EA WHERE AA."c0" = BA."c0" AND BA."c0" = CA."c0" AND CA."c0" = DA."c0" AND DA."c0" = EA."c0" ) V_42602
WHERE V_42602."c0" IS NOT NULL

 ) "V_8490"
 INNER JOIN  ( SELECT V_9514."var_1" AS "var_1",V_9146."var_3" AS "var_3",V_9514."var_2" AS "var_2",V_9514."mappingid_1" AS "mappingid_1",V_9146."mappingid_3" AS "mappingid_3",V_9514."mappingid_2" AS "mappingid_2"
FROM  ( SELECT V_65470."c0" AS "var_1",1107204185::integer AS "mappingid_1",V_65470."c1" AS "var_2",1523084197::integer AS "mappingid_2"
FROM (SELECT DISTINCT AA."c0" as c0, AA."c1" as c1 FROM  "src_hasOtherStaff" as AA, "src_BasketBallFan" as BA, "src_Director" as CA, "src_tenured" as DA, "src_isPartOf" as EA WHERE AA."c1" = DA."c1" AND DA."c1" = EA."c1" AND AA."c0" = BA."c0" AND BA."c0" = CA."c0" AND CA."c0" = DA."c0" AND DA."c0" = EA."c0" ) V_65470
WHERE V_65470."c1" IS NOT NULL

UNION
SELECT V_21485."c1" AS "var_1",747870628::integer AS "mappingid_1",V_21485."c0" AS "var_2",421191743::integer AS "mappingid_2"
FROM (SELECT DISTINCT AA."c0" as c0, AA."c1" as c1 FROM  "src_hasOtherStaff" as AA, "src_BasketBallFan" as BA, "src_Director" as CA, "src_tenured" as DA, "src_isPartOf" as EA WHERE AA."c1" = DA."c1" AND DA."c1" = EA."c1" AND AA."c0" = BA."c0" AND BA."c0" = CA."c0" AND CA."c0" = DA."c0" AND DA."c0" = EA."c0" ) V_21485
WHERE V_21485."c0" IS NOT NULL

 ) "V_9514"
 INNER JOIN  ( SELECT V_39148."c0" AS "var_2",1403716230::integer AS "mappingid_2",V_39148."c1" AS "var_3",1673916398::integer AS "mappingid_3"
FROM (SELECT DISTINCT AA."c0" as c0, AA."c1" as c1 FROM  "src_isPartOf" as AA, "src_BasketBallFan" as BA, "src_hasCollegeDiscipline" as CA WHERE AA."c1" = CA."c1" AND AA."c0" = BA."c0" AND BA."c0" = CA."c0" ) V_39148
WHERE V_39148."c1" IS NOT NULL
 ) "V_9146" ON ("V_9514"."var_2" = "V_9146"."var_2")
 ) "V_8672" ON ("V_8490"."var_1" = "V_8672"."var_1")

[main] ERROR es.upm.fi.dia.oeg.morph.base.DBUtility$ - Error while executing SQL Query: SELECT  DISTINCT V_8490."var_1" AS "1",V_8490."mappingid_1" AS "mappingid_1"
FROM  ( SELECT V_23573."c0" AS "var_0",1336775847::integer AS "mappingid_0",V_23573."c1" AS "var_1",1998137093::integer AS "mappingid_1"
FROM (SELECT DISTINCT AA."c0" as c0, EA."c1" as c1 FROM  "src_Dean" as AA, "src_Specification" as BA, "src_Humanities" as CA, "src_Chair" as DA, "src_isStudentOf" as EA WHERE AA."c0" = BA."c0" AND BA."c0" = CA."c0" AND CA."c0" = DA."c0" AND DA."c0" = EA."c0" ) V_23573
WHERE V_23573."c1" IS NOT NULL

UNION
SELECT V_42602."c1" AS "var_0",1335061928::integer AS "mappingid_0",V_42602."c0" AS "var_1",1108706191::integer AS "mappingid_1"
FROM (SELECT DISTINCT AA."c0" as c0, EA."c1" as c1 FROM  "src_Dean" as AA, "src_Specification" as BA, "src_Humanities" as CA, "src_Chair" as DA, "src_isStudentOf" as EA WHERE AA."c0" = BA."c0" AND BA."c0" = CA."c0" AND CA."c0" = DA."c0" AND DA."c0" = EA."c0" ) V_42602
WHERE V_42602."c0" IS NOT NULL

 ) "V_8490"
 INNER JOIN  ( SELECT V_9514."var_1" AS "var_1",V_9146."var_3" AS "var_3",V_9514."var_2" AS "var_2",V_9514."mappingid_1" AS "mappingid_1",V_9146."mappingid_3" AS "mappingid_3",V_9514."mappingid_2" AS "mappingid_2"
FROM  ( SELECT V_65470."c0" AS "var_1",1107204185::integer AS "mappingid_1",V_65470."c1" AS "var_2",1523084197::integer AS "mappingid_2"
FROM (SELECT DISTINCT AA."c0" as c0, AA."c1" as c1 FROM  "src_hasOtherStaff" as AA, "src_BasketBallFan" as BA, "src_Director" as CA, "src_tenured" as DA, "src_isPartOf" as EA WHERE AA."c1" = DA."c1" AND DA."c1" = EA."c1" AND AA."c0" = BA."c0" AND BA."c0" = CA."c0" AND CA."c0" = DA."c0" AND DA."c0" = EA."c0" ) V_65470
WHERE V_65470."c1" IS NOT NULL

UNION
SELECT V_21485."c1" AS "var_1",747870628::integer AS "mappingid_1",V_21485."c0" AS "var_2",421191743::integer AS "mappingid_2"
FROM (SELECT DISTINCT AA."c0" as c0, AA."c1" as c1 FROM  "src_hasOtherStaff" as AA, "src_BasketBallFan" as BA, "src_Director" as CA, "src_tenured" as DA, "src_isPartOf" as EA WHERE AA."c1" = DA."c1" AND DA."c1" = EA."c1" AND AA."c0" = BA."c0" AND BA."c0" = CA."c0" AND CA."c0" = DA."c0" AND DA."c0" = EA."c0" ) V_21485
WHERE V_21485."c0" IS NOT NULL

 ) "V_9514"
 INNER JOIN  ( SELECT V_39148."c0" AS "var_2",1403716230::integer AS "mappingid_2",V_39148."c1" AS "var_3",1673916398::integer AS "mappingid_3"
FROM (SELECT DISTINCT AA."c0" as c0, AA."c1" as c1 FROM  "src_isPartOf" as AA, "src_BasketBallFan" as BA, "src_hasCollegeDiscipline" as CA WHERE AA."c1" = CA."c1" AND AA."c0" = BA."c0" AND BA."c0" = CA."c0" ) V_39148
WHERE V_39148."c1" IS NOT NULL
 ) "V_9146" ON ("V_9514"."var_2" = "V_9146"."var_2")
 ) "V_8672" ON ("V_8490"."var_1" = "V_8672"."var_1")

org.postgresql.util.PSQLException: ERROR: missing FROM-clause entry for table "v_9514"
  Position: 986
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2468)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2211)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:309)
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:446)
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:370)
	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:311)
	at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:297)
	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:274)
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:269)
	at es.upm.fi.dia.oeg.morph.base.DBUtility$.execute(DBUtility.scala:76)
	at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBDataSourceReader.execute(MorphRDBDataSourceReader.scala:19)
	at es.upm.fi.dia.oeg.morph.base.querytranslator.engine.QueryResultTranslator$$anonfun$translateResult$1.apply(QueryResultTranslator.scala:28)
	at es.upm.fi.dia.oeg.morph.base.querytranslator.engine.QueryResultTranslator$$anonfun$translateResult$1.apply(QueryResultTranslator.scala:23)
	at scala.collection.immutable.Map$Map1.foreach(Map.scala:116)
	at es.upm.fi.dia.oeg.morph.base.querytranslator.engine.QueryResultTranslator.translateResult(QueryResultTranslator.scala:23)
	at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner.run(MorphBaseRunner.scala:188)
	at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner$.main(MorphRDBRunner.scala:82)
	at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner.main(MorphRDBRunner.scala)
[main] ERROR es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner$ - Exception occured: ERROR: missing FROM-clause entry for table "v_9514"
  Position: 986
Exception in thread "main" org.postgresql.util.PSQLException: ERROR: missing FROM-clause entry for table "v_9514"
  Position: 986
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2468)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2211)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:309)
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:446)
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:370)
	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:311)
	at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:297)
	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:274)
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:269)
	at es.upm.fi.dia.oeg.morph.base.DBUtility$.execute(DBUtility.scala:76)
	at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBDataSourceReader.execute(MorphRDBDataSourceReader.scala:19)
	at es.upm.fi.dia.oeg.morph.base.querytranslator.engine.QueryResultTranslator$$anonfun$translateResult$1.apply(QueryResultTranslator.scala:28)
	at es.upm.fi.dia.oeg.morph.base.querytranslator.engine.QueryResultTranslator$$anonfun$translateResult$1.apply(QueryResultTranslator.scala:23)
	at scala.collection.immutable.Map$Map1.foreach(Map.scala:116)
	at es.upm.fi.dia.oeg.morph.base.querytranslator.engine.QueryResultTranslator.translateResult(QueryResultTranslator.scala:23)
	at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner.run(MorphBaseRunner.scala:188)
	at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner$.main(MorphRDBRunner.scala:82)
	at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner.main(MorphRDBRunner.scala)

Could you please help to figure out what's the problem?

Uri encoding

Hi,

I’m trying to encode some strange characters from the Uri. I want to encode the "´" symbol to "%C2%B4", but is not working:

uri.encode=(" "->"%20"),,(","->""),,("á"->"a"),,("é"->"e"),,("í"->"i"),,("ó"->"o"),,("ú"->"u"),,("ü"->"u"),,("ñ"->"n"),,("´"->"%C2%B4")

There is any option to encode all the uri characters automatically or I have to define all of them manually? There is a long list… http://www.w3schools.com/tags/ref_urlencode.asp

Heap Space issues

We are using morph-rdb in the context of linked-map project (http://linkedmap.unizar.es/) from Planet Data and we found out a heap space issue that comes out when generating ttl from big tables (more than 300000 rows)

Memory issue happens around java ResultSet so this issue doesn't affect your software directly but there is a coupple of things that could be implemented to solve it:

• First one would be to ensure memory with pagination strategies
• If first is not posible, it would be great to enable sql parser to recognize patterns like:

SELECT tableName.field1 AS f1, tableName.field2 as f2, ... FROM tableName ORDER BY tableName.fieldN OFFSET x LIMIT y

Currently, expresions like that are marked as unparseable:

WARN [main] (R2RMLUnfolder.java:73) - Not able to parse the query, string will be used.
ERROR [main] (R2RMLUtility.java:106) - error parsing query string : 
SELECT buildings.id AS id, buildings.prov AS prov, buildings.proyecto AS proyecto FROM buildings ORDER BY id OFFSET 200000 LIMIT 210000;
ERROR [main] (R2RMLUtility.java:107) - error message = Encountered "OFFSET" at line 1, column 110.
Was expecting one of:
    "ASC" ...
    "DESC" ...
    "FOR" ...
    "," ...
    ";" ...
    "." ...
    "+" ...
    "-" ...
    "*" ...
    "||" ...
    "/" ...
    "**" ...
    "(" ...

 WARN [main] (R2RMLUnfolder.java:73) - Not able to parse the query, string will be used.

Thanks in advance,

add CSV/TSV format for query translation result

Add an option in the properties file to use the SPARQL Query Result CSV/TSV format for query translation results.

See this relevant class:
https://github.com/oeg-upm/morph-rdb/blob/master/morph-base-querytranslator/src/main/scala/es/upm/fi/dia/oeg/morph/base/querytranslator/engine/MorphXMLQueryResultWriter.scala

See this official spec for generating CSV/TSV format: https://www.w3.org/TR/sparql11-results-csv-tsv/

Cannot run any example

I followed the steps from the readme and imported the maven project into eclipse. I ran mvn clean and packages in the main folder. I moved the jar from the target folder to the examples folder. Then I tried to run the example with the CSV file. This is my output.

/DEV/github/morph-rdb/morph-examples/examples-csv$ ./example1-query01-csv.sh
[main] INFO es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphCSVProperties - reading configuration file : examples-csv/example1-query01-csv.morph.properties
[main] INFO es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphCSVRunnerFactory - MorphBaseRunnerFactory running morph-rdb 3.9.12 ...
[main] INFO es.upm.fi.dia.oeg.morph.r2rml.model.R2RMLMappingDocument$ - Parsing mapping document examples-csv/example1-mapping-csv.ttl
[main] WARN es.upm.fi.dia.oeg.morph.r2rml.model.R2RMLMappingDocument$ - Error occured while building metadata.
[main] INFO es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphCSVRunnerFactory - Setting output to examples-csv/example1-query01-result-csv.xml
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.jena.rdf.model.Model.setNsPrefixes(Ljava/util/Map;)Lorg/apache/jena/rdf/model/Model;
at es.upm.fi.dia.oeg.morph.base.materializer.MorphBaseMaterializer.setModelPrefixMap(MorphBaseMaterializer.scala:40)
at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunnerFactory.buildMaterializer(MorphBaseRunnerFactory.scala:320)
at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunnerFactory.createRunner(MorphBaseRunnerFactory.scala:105)
at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunnerFactory.createRunner(MorphBaseRunnerFactory.scala:46)
at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphCSVRunnerFactory.createRunner(MorphCSVRunnerFactory.scala:47)
at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphCSVRunner$.apply(MorphCSVRunner.scala:35)
at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphCSVRunner$.apply(MorphCSVRunner.scala:42)
at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphCSVRunner$.main(MorphCSVRunner.scala:60)
at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphCSVRunner.main(MorphCSVRunner.scala)

What am I doing wrong?

Minor issue in R2RMLDocumentMapping

Hi guys, just a minor issue: when you write a mapping with a reference object map that references a triples that does not exist, the parsing returns an exception rather uneasy to understand.
Instead I've made a change to raise an exception with an explicit message:
at the end of R2RMLDocumentMapping.getParentTriplesMap(refObjectMap: R2RMLRefObjectMap), just replace the last lines as this:

    if (parentTripleMaps.iterator.hasNext) {
        val parentTripleMap = parentTripleMaps.iterator.next;
        parentTripleMap.asInstanceOf[R2RMLTriplesMap]
    } else {
        throw new Exception("Error: referenced parent triples map des not exist: " + parentTripleMapResources)
    }

Cheers,
Franck.

Deal with multiple RefObjectMaps in a predicate-object map

Hi Freddy, very similarly to issue #6, multiple referencing object maps are not taken into account in the unfolder, only the first one is. Actually the fix is very simple: I just added line
for (refObjectMap <- pom.refObjectMaps) { ...}
instead of
val refObjectMap = pom.RefObjectMap(0);

Below I copy the whole code concerning predicate-object maps unfolding. I have added a few comments, debug logs and possibly made a few refactorings, but just take what you like in that ;-)
But you can't copy that right away, as it would not compile: at least, I now use class xR2RMLLogicalSource instead of R2RMLLogicalTable. Thinks that's it but take care anyway.

    // ----- Unfold predicate-object maps
    if (poms != null) {
        for (pom <- poms) {
            // Unfold all predicateMaps of the current predicate-object map 
            if (pom.predicateMaps != null) {
                for (pm <- pom.predicateMaps) {
                    val selectItems = this.unfoldTermMap(pm, logicalTableAlias);
                    result.addSelectItems(selectItems);
                }
            }

            // Unfold all objectMaps of the current predicate-object map 
            if (pom.objectMaps != null) {
                for (om <- pom.objectMaps) {
                    val selectItems = this.unfoldTermMap(om, logicalTableAlias);
                    result.addSelectItems(selectItems);
                }
            }

            // Unfold RefObjectMaps
            if (pom.refObjectMaps != null) {
                for (refObjectMap <- pom.refObjectMaps) {
                    if (refObjectMap != null) {
                        val parentTriplesMap = this.md.getParentTriplesMap(refObjectMap);
                        val parentLogicalTable = parentTriplesMap.logicalSource
                        if (parentLogicalTable == null) {
                            val errorMessage = "Parent logical table is not found for RefObjectMap : " + pom.getMappedPredicateName(0);
                            throw new Exception(errorMessage);
                        }
                        val sqlParentLogicalTable = this.unfoldLogicalSource(parentLogicalTable.asInstanceOf[xR2RMLLogicalSource]);
                        val joinQueryAlias = sqlParentLogicalTable.generateAlias();

                        sqlParentLogicalTable.setAlias(joinQueryAlias);
                        this.mapRefObjectMapAlias += (refObjectMap -> joinQueryAlias);
                        pom.alias = joinQueryAlias;

                        val parentSubjectMap = parentTriplesMap.subjectMap;
                        // Get names of the columns referenced in the parent triples map
                        val refObjectMapColumnsString = parentSubjectMap.getReferencedColumns;
                        if (refObjectMapColumnsString != null) {
                            for (refObjectMapColumnString <- refObjectMapColumnsString) {
                                val selectItem = MorphSQLSelectItem(refObjectMapColumnString, joinQueryAlias, dbType, null);
                                if (selectItem.getAlias() == null) {
                                    val alias = selectItem.getTable() + "_" + selectItem.getColumn();
                                    selectItem.setAlias(alias);
                                    if (this.mapTermMapColumnsAliases.containsKey(refObjectMap)) {
                                        val oldColumnAliases = this.mapTermMapColumnsAliases(refObjectMap);
                                        val newColumnAliases = oldColumnAliases ::: List(alias);
                                        this.mapTermMapColumnsAliases += (refObjectMap -> newColumnAliases);
                                    } else {
                                        this.mapTermMapColumnsAliases += (refObjectMap -> List(alias));
                                    }
                                }
                                result.addSelectItem(selectItem);
                            }
                        }

                        val joinConditions = refObjectMap.getJoinConditions();
                        val onExpression = MorphRDBUnfolder.unfoldJoinConditions(joinConditions, logicalTableAlias, joinQueryAlias, dbType);
                        val joinQuery = new SQLJoinTable(sqlParentLogicalTable, Constants.JOINS_TYPE_LEFT, onExpression);
                        result.addFromItem(joinQuery);
                    }
                }
            }
        }
        logger.debug("Unfolded predicate-object map: " + result.toString.replaceAll("\n", " "))
    }

Conversion errors on doubles

We have been using morph to convert a postgreSQL database to RDF.

In the database we have the values stored as double precision with entries such as 8.8 and 8.6. However, when these are converted by the mappings they become 8.80000000000000071 and 8.59999999999999964 respectively. This is undesirable. We want the values to remain as they appear in the database.

I suspect that this is caused by casting errors in the code. Is it something that can be fixed?

Allow entering parameters through the command line

The properties file includes some parameters that change (almost) every time the program is run.
It would be handy if those parameters could also be inserted through the command line instead of having to edit the properties file in every execution.

Month lost in DATE

I have a problem importing dates from MySQL (MariaDB).

I have a column with the DATE datatype, and the Morph transforms 2019-04-19 into "2019-00-19"^^<http://www.w3.org/2001/XMLSchema#date> (month turned to 00). I tried setting input.date.pattern=yyyy-MM-dd in .properties, but it does not help.

When I convert my MySQL column to DATETIME, everthing works like charm with the same dataset.

example1-mapping-csv.ttl formatted as HTML instead of Turtle, causing jena riot parsing error:"Bad character in IRI"

Following instructions at https://github.com/oeg-upm/morph-rdb/wiki/Installation-using-Git-and-Maven for testing with CSV examples, a parsing error causes a stack trace:

$ ./example1-batch-csv.sh 
22:20:42.935 [main] DEBUG es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphCSVRunner$ - propertiesDirectory = examples-csv
[ . . . ]
22:20:44.232 [main] INFO  es.upm.fi.dia.oeg.morph.r2rml.model.R2RMLMappingDocument$ - Parsing mapping document https://github.com/oeg-upm/morph-rdb/blob/master/morph-examples/examples-csv/example1-mapping-csv.ttl
org.apache.jena.riot.RiotException: [line: 7, col: 11] Bad character in IRI (space): <!DOCTYPE[space]...>
	at org.apache.jena.riot.system.ErrorHandlerFactory$ErrorHandlerStd.fatal(ErrorHandlerFactory.java:136)
	at org.apache.jena.riot.lang.LangEngine.raiseException(LangEngine.java:165)
	[ . . . ]

The mapping document retrieved from that URL appears to be an HTML-formatted rendition of the Turtle document, rather than the Turtle document itself:

$ wget https://github.com/oeg-upm/morph-rdb/blob/master/morph-examples/examples-csv/example1-mapping-csv.ttl
[ . . . ]
$ head example1-mapping-csv.ttl 
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">

Jena riot tries to parse it as Turtle, but barfs on '' because it contains a space. If you look at https://github.com/oeg-upm/morph-rdb/blob/master/morph-examples/examples-csv/example1-mapping-csv.ttl in a browser you can see that it is the github-HTML-formatted rendition.

Perhaps the github "raw" version at https://raw.githubusercontent.com/oeg-upm/morph-rdb/master/morph-examples/examples-csv/example1-mapping-csv.ttl needs to be referenced instead of the HTML-formatted version?

Generation of duplicates in batch mode

Morph batch mode generates duplicates, even when using materialization.distinct=True.

Current behaviour of materialization mode of Morph builds a sql query per each triples map and brings the data to memory. Triples are then generated by iterating over the rows of the in-memory table. One of the drawbacks of this strategy is the generation of duplicates, even when including the DISTINCT clause in the sql queries. This is because the DISTINCT is not applied in subject map or POM level, but at entire triples map level. Example:

This table is retrieved to generate the triples for one triples map. All rows are DISTINCT, but when morph process subjectMap (using trip_id) duplicates are generated because there are rows with repeated values in trip_id. Same logic applies when processing POMs.

Assumptions about type of term maps in query rewriting

Hi again,

By browsing through the code of the query rewriting part (morph-base-querytranslator and morph-r2rml-querytranslator), I've noticed a quite important issue:
in many places you assume that subject maps are always template-based (like when trying to match a URI in subject position of a triple pattern with a subjet map), and object maps are always column-based. Which is wrong.
Imagine a database with URLs in some column: a subject map could simply use this value with a rr;column property, by adding a rr:termType rr:IRI.
Similarly, an object map can use a template, say to concatenate a first name and last name, and then specify a termType rr:Literal.
These examples are not exotic I think.

Hope this helps ;-)

Franck.

Error(?) in blank nodes management in SPARQL query result set

Hi guys,
Just a minor one: in MorphXMLQueryResultWriter.process:
val termTypeElementName = {
if(termType.equalsIgnoreCase(Constants.R2RML_IRI_URI)) {
"uri";
} else if(termType.equalsIgnoreCase(Constants.R2RML_LITERAL_URI)) {
"literal";
} else {
null
}

I guess the null case is when the variable corresponds to a blank node. That may be weird to project a variable with a blank node value, but still when this happens you'll probably get a null pointer exception in the next line.
val termTypeElement = xmlDoc.createElement(termTypeElementName);

Cheers,
Franck.

Deal with multiple predicates and/or objects in a predicate-object map

Hi Freddy,

Another one about an other pb we also discussed by email: the unfolder does not collect all columns in case there are multiple predicates and/or objects in a predicate-object map.
I simply updated the code so that all columns are added to the select items in the sql query, like this:

/**
 * Unfolding a triples map means to progressively build an SQL query by accumulating pieces:
 * (1) create the FROM clause from the logical table,
 * (2) for each column in the subject predicate and object maps, add items to the SELECT clause,
 * (3) for each column in the parent triples map of each referencing object map, add items of the SELECT clause,
 * (4) for each join condition, add an SQL WHERE condition and an alias in the FROM clause for the parent table,
 *
 * @return an SQLQuery (IQuery) describing the actual SQL query to be run against the RDB
 */
def unfoldTriplesMap(
    triplesMapId: String,
    logicalSrc: xR2RMLLogicalSource,
    subjectMap: R2RMLSubjectMap,
    poms: Collection[R2RMLPredicateObjectMap]): IQuery = {

    (...)

    // Unfold predicate-object maps
    if (poms != null) {
        for (pom <- poms) {
            // Unfold all predicateMaps of the current predicate-object map
            val predicateMaps = pom.predicateMaps;
            if (predicateMaps != null && !predicateMaps.isEmpty()) {
                for (pm <- pom.predicateMaps) {
                    val selectItems = this.unfoldTermMap(pm, logicalTableAlias);
                    result.addSelectItems(selectItems);
                }
            }

            // Unfold all objectMaps of the current predicate-object map
            val objectMaps = pom.objectMaps;
            if (objectMaps != null && !objectMaps.isEmpty()) {
                for (om <- pom.objectMaps) {
                    val selectItems = this.unfoldTermMap(om, logicalTableAlias);
                    result.addSelectItems(selectItems);
                }
            }

    (...)

Cheers,
Franck.

output file format

Does Morph actually support multiple output file format? Whatever I set the 'output.rdflanguage' with N-TRIPLE ,TURTLE or N3, the output file always is N-TRIPLE format. And when I set it to RDF/XML, the output file just output prefix lines, no any triples in it. Could you explain a little bit?

Escaping characters in TURTLE exports

Another issue is escaping characters in TURTLE exports.
According to the TURTLE specifications ( https://www.w3.org/TR/turtle/ ) , Turtle supports the following escape sequences:

Escape sequence	Unicode
'\t'	U+0009
'\b'	U+0008
'\n'	U+000A
'\r'	U+000D
'\f'	U+000C
'\"'	U+0022
'\''	U+0027
'\\'	U+005C

However, morph RDB converts all quotation-signs " in the original data into %22 instead of \". This is not supported by other software working with TURTLE. Hence, the data gets corrupted. This happens because GeneralUtility.encodeLiteral is applied to all literals regardless of the export format (both for XML and TURTLE exports)

Error translating Query for aact-query1.sh

I have built the tool using maven but it's failing when I run aact-query1.sh, the generated SQL seems to be missing a table prefix from the inner join;

SELECT T4."id" AS "sponsor",T6."name" AS "label_sponsor_0",2022689531::integer AS "mappingid_label_study",T6."id" AS "label_sponsor_1",'http://www.w3.org/2000/01/rdf-schema#label'::text AS "uri_label1103900028",'http://bio2rdf.org/clinicaltrials_vocabulary:sponsor'::text AS "uri_sponsor2085175700",379121284::integer AS "mappingid_sponsor",T3."nct_id" AS "label_study_1",1919834117::integer AS "mappingid_label_sponsor",T3."nct_id" AS "study",1157943921::integer AS "mappingid_study",T3."brief_title" AS "label_study_0"
FROM ctgov.studies T3 INNER JOIN ctgov.sponsors T4 ON (T3.nct_id = T4.nct_id)
 INNER JOIN sponsors T6 ON ((T4."id" = T6."id") AND (T6."name" IS NOT NULL) AND (T6."id" IS NOT NULL))
WHERE T3."brief_title" IS NOT NULL AND T3."nct_id" IS NOT NULL AND T4."id" IS NOT NULL
LIMIT 5 

It fails with the error:
org.postgresql.util.PSQLException: ERROR: relation "sponsors" does not exist which makes sense as the schema is missing.

Reference to null record created by rr:parentTriplesMap when the foreign key is null

Dear developers of morph-rdb,
many thanks for a great tool.

I have a problem with morph-rdb, which is creating references to null records when the foreign index is null.
Suppose I have two mysql tables:
foo

foo_id	foo	boo
2	aa	bb
3	cc	NULL

boo

boo_id	boo
2	bb
3	dd

and I have a R2RML file:

@prefix rr: <http://www.w3.org/ns/r2rml#>.
@prefix ex: <https://test.de/ontology#>.
<#boo_map>
    rr:logicalTable [ rr:tableName "boo" ];
	   rr:subjectMap [
        rr:template "https://test.de/boo/{boo_id}";
        ];
	    rr:predicateObjectMap [
        rr:predicate ex:boo;
        rr:objectMap [ rr:column "boo" ;];].    
<#foo_map>
    rr:logicalTable [ rr:tableName "foo" ];
	   rr:subjectMap [
        rr:template "https://test.de/foo/{foo_id}";
        ];
	    rr:predicateObjectMap [
        rr:predicate ex:foo;
        rr:objectMap [ rr:column "foo" ;]		;
    ];
			   	    rr:predicateObjectMap [
        rr:predicate ex:has_boo;
		        rr:objectMap [
        rr:parentTriplesMap <#boo_map>;
        rr:joinCondition [
            rr:child "boo";
            rr:parent "boo";
        ];
           ];].

which results in the following RDF:

<https://test.de/boo/2>	<https://test.de/ontology#boo>	"bb" .
<https://test.de/boo/3>	<https://test.de/ontology#boo>	"dd" .
<https://test.de/foo/2>	<https://test.de/ontology#has_boo>	<https://test.de/boo/2> .
<https://test.de/foo/2>	<https://test.de/ontology#foo>	"aa" .
<https://test.de/foo/3>	<https://test.de/ontology#has_boo>	<https://test.de/boo/null> .
<https://test.de/foo/3>	<https://test.de/ontology#foo>	"cc" .

The line <https://test.de/foo/3> <https://test.de/ontology#has_boo> <https://test.de/boo/null> . is an error on the side of morph-rdb, it should not include a non-existent null record from boo when the foreign key in foo has a null value. And other R2RML convertors don't generate this line.
Could you please correct this issue.

test on npd benchmark

Hi,

I am trying to use morph (latest 3.5.3 version) over the npd benchmark (https://github.com/ontop/npd-benchmark).

• The r2rml mapping file is https://github.com/ontop/npd-benchmark/blob/master/mappings/mysql/spatial/npd-v2-ql_a.pretty.ttl
• The SPARQL query is
  https://github.com/ontop/npd-benchmark/blob/master/queries/01.q

And I got the following error message:

2015-01-08 11:28:16 ERROR R2RMLRunner:76 - Exception occured: Illegal repetition near index 41
<http://sws.ifi.uio.no/data/npd-v2/survey/{seaMultiline.seaSurveyName}/multiline>

The complete log is

https://gist.github.com/ghxiao/4db2f62f38a6b853220c

Could you help to figure out what's the problem?

Btw, I also tested the master branch, which reports a different error:

INFO [main] (MorphBaseRunnerFactory.scala:95) - Building query translator...
 WARN [main] (MorphBaseRunnerFactory.scala:104) - Error building query translator!es.upm.fi.dia.oeg.morph.rdb.querytranslator.MorphRDBQueryTranslatorFactory
Exception in thread "main" java.util.NoSuchElementException: None.get
# ...
# ...
2015-01-08 11:41:18 ERROR R2RMLRunner:76 - Exception occured: None.get
    at scala.None$.get(Option.scala:322)
    at scala.None$.get(Option.scala:320)
    at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner$$anonfun$1.apply(MorphBaseRunner.scala:186)
    at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner$$anonfun$1.apply(MorphBaseRunner.scala:184)

Thanks in advance!

Cannot execute a query from Berlin SPARQL Benchmark

Hi,
do you support queries from BSBM? I have tried to execute a query and it failed.

See attached files to see what happens. In general I was not able to query anything where a join is involved. Extension of the files was changed (or added) to .txt so it can be uploaded to github.

example.properties.txt
Query_01.txt
bsbm.txt
log.txt

Exception on extend

Hi,

This is my query:

PREFIX ssn: <http://purl.oclc.org/NET/ssnx/ssn#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX time: <http://www.w3.org/2006/time#>
PREFIX iotsn: <http://iot.soton.ac.uk/smarthome/sensor#>
PREFIX iot: <http://purl.oclc.org/NET/iot#>

SELECT (avg(?val) as ?sval) ?hours
WHERE {
  ?instant time:inXSDDateTime ?date.
  ?obs ssn:observationSamplingTime ?instant;
    ssn:observedBy iotsn:environmental1;
    ssn:observationResult ?snout.
  ?snout ssn:hasValue ?obsval.
  ?obsval a iot:InternalTemperatureValue;
    iot:hasQuantityValue ?val.
  FILTER (?date > "2012-07-20T00:00:00"^^xsd:dateTime && ?date < "2012-07-21T00:00:00"^^xsd:dateTime)
} GROUP BY (hours(xsd:dateTime(?date)) as ?hours)

I suspect somethings gone wrong in between the extend and project ops using Jena. I'm using morph v3.5.16.

This is the error message:

ERROR [main] (MorphRDBRunner.scala:76) - Exception occured: (extend ((?sval ?.0))
  (group ((?hours (hours (<http://www.w3.org/2001/XMLSchema#dateTime> ?date)))) ((?.0 (avg ?val)))
    (filter (&& (> ?date "2012-07-20T00:00:00"^^<http://www.w3.org/2001/XMLSchema#dateTime>) (< ?date "2012-07-21T00:00:00"^^<http://www.w3.org/2001/XMLSchema#dateTime>))
      (bgp
        (triple ?instant <http://www.w3.org/2006/time#inXSDDateTime> ?date)
        (triple ?obs <http://purl.oclc.org/NET/ssnx/ssn#observationSamplingTime> ?instant)
        (triple ?obs <http://purl.oclc.org/NET/ssnx/ssn#observedBy> <http://iot.soton.ac.uk/smarthome/sensor#environmental1>)
        (triple ?obs <http://purl.oclc.org/NET/ssnx/ssn#observationResult> ?snout)
        (triple ?snout <http://purl.oclc.org/NET/ssnx/ssn#hasValue> ?obsval)
        (triple ?obsval <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://purl.oclc.org/NET/iot#InternalTemperatureValue>)
        (triple ?obsval <http://purl.oclc.org/NET/iot#hasQuantityValue> ?val)
      ))))
 (of class com.hp.hpl.jena.sparql.algebra.op.OpExtend)
Exception in thread "main" scala.MatchError: (extend ((?sval ?.0))
  (group ((?hours (hours (<http://www.w3.org/2001/XMLSchema#dateTime> ?date)))) ((?.0 (avg ?val)))
    (filter (&& (> ?date "2012-07-20T00:00:00"^^<http://www.w3.org/2001/XMLSchema#dateTime>) (< ?date "2012-07-21T00:00:00"^^<http://www.w3.org/2001/XMLSchema#dateTime>))
      (bgp
        (triple ?instant <http://www.w3.org/2006/time#inXSDDateTime> ?date)
        (triple ?obs <http://purl.oclc.org/NET/ssnx/ssn#observationSamplingTime> ?instant)
        (triple ?obs <http://purl.oclc.org/NET/ssnx/ssn#observedBy> <http://iot.soton.ac.uk/smarthome/sensor#environmental1>)
        (triple ?obs <http://purl.oclc.org/NET/ssnx/ssn#observationResult> ?snout)
        (triple ?snout <http://purl.oclc.org/NET/ssnx/ssn#hasValue> ?obsval)
        (triple ?obsval <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://purl.oclc.org/NET/iot#InternalTemperatureValue>)
        (triple ?obsval <http://purl.oclc.org/NET/iot#hasQuantityValue> ?val)
      ))))
 (of class com.hp.hpl.jena.sparql.algebra.op.OpExtend)
    at es.upm.fi.dia.oeg.morph.base.querytranslator.engine.MorphMappingInferrer.genericInferBGP(MorphMappingInferrer.scala:51)
    at es.upm.fi.dia.oeg.morph.base.querytranslator.engine.MorphMappingInferrer.genericInferBGP(MorphMappingInferrer.scala:70)
    at es.upm.fi.dia.oeg.morph.base.querytranslator.engine.MorphMappingInferrer.genericInfer(MorphMappingInferrer.scala:85)
    at es.upm.fi.dia.oeg.morph.base.querytranslator.engine.MorphMappingInferrer.inferByRDFType(MorphMappingInferrer.scala:148)
    at es.upm.fi.dia.oeg.morph.base.querytranslator.engine.MorphMappingInferrer.infer(MorphMappingInferrer.scala:100)
    at es.upm.fi.dia.oeg.morph.base.querytranslator.MorphBaseQueryTranslator.translate(MorphBaseQueryTranslator.scala:1366)
    at es.upm.fi.dia.oeg.morph.base.querytranslator.MorphBaseQueryTranslator.translate(MorphBaseQueryTranslator.scala:1429)
    at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner$$anonfun$1.apply(MorphBaseRunner.scala:186)
    at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner$$anonfun$1.apply(MorphBaseRunner.scala:184)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
    at scala.collection.immutable.List.foreach(List.scala:381)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:245)
    at scala.collection.immutable.List.map(List.scala:284)
    at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner.translateSPARQLQueriesIntoSQLQueries(MorphBaseRunner.scala:184)
    at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner.run(MorphBaseRunner.scala:166)
    at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner$.main(MorphRDBRunner.scala:72)
    at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner.main(MorphRDBRunner.scala)

Thanks!

log4j.properties file error

There is a problem related with some missing file, I assume this file is this one https://github.com/oeg-upm/morph-rdb/blob/master/morph-base/src/main/scala/es/upm/fi/dia/oeg/morph/base/MorphProperties.scala around line 87. It needs the configurator file. I don't know if that is the solution, but please have a look at it. Here is the error:

INFO main - reading configuration file : project/run.r2rml.properties
log4j:ERROR Could not read configuration file [log4j.properties].
java.io.FileNotFoundException: log4j.properties (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at java.io.FileInputStream.(FileInputStream.java:93)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:306)
at org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:324)
at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunnerFactory.(MorphBaseRunnerFactory.scala:27)
at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunnerFactory.(MorphRDBRunnerFactory.scala:22)
at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphCSVRunnerFactory.(MorphCSVRunnerFactory.scala:19)
at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphCSVRunner$.apply(MorphCSVRunner.scala:34)
at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphCSVRunner$.apply(MorphCSVRunner.scala:42)
at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphCSVRunner$.main(MorphCSVRunner.scala:60)
at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphCSVRunner.main(MorphCSVRunner.scala)
log4j:ERROR Ignoring configuration file [log4j.properties].

I'm using El Capitan OSX and java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)

I can't generate the output while this error exists.

Thanks.

rdf:type in POMs

If the input query asks about an rdf:type, the engine only search over rr:class property and does not explore the rest of the POMs. Issued by @mchaloupka in #34

Error: Could not find or load main class es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner

Hi, I cannot get the examples to work and it gives the following error on running the batch file: Error: Could not find or load main class es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner

my batch file is untouched and looks like this:

cd ..

java -cp morph-rdb.jar;lib/* es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner examples-mysql example1-batch-mysql.morph.properties

cd examples-mysql

I followed the manual installation for morph-RDB, for windows. I am quite certain that I have followed all the steps as it is quite straightforward.

Provide docker support

create dockerfile and add info about docker image in docker-hub

Next on empty iterator

Hello,
When i try to execute a Sparql query, a error has come
ERROR main - Exception occured: next on empty iterator
Exception in thread "main" java.util.NoSuchElementException: next on empty iterator
at scala.collection.Iterator$$anon$2.next(Iterator.scala:39)
at scala.collection.Iterator$$anon$2.next(Iterator.scala:37)
at es.upm.fi.dia.oeg.morph.base.querytranslator.MorphBaseQueryTranslator$$anonfun$21.apply(MorphBaseQueryTranslator.scala:1393)
at es.upm.fi.dia.oeg.morph.base.querytranslator.MorphBaseQueryTranslator$$anonfun$21.apply(MorphBaseQueryTranslator.scala:1391)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at scala.collection.MapLike$DefaultKeySet.foreach(MapLike.scala:174)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractSet.scala$collection$SetLike$$super$map(Set.scala:47)
at scala.collection.SetLike$class.map(SetLike.scala:93)
at scala.collection.AbstractSet.map(Set.scala:47)
at es.upm.fi.dia.oeg.morph.base.querytranslator.MorphBaseQueryTranslator.translate(MorphBaseQueryTranslator.scala:1391)
at es.upm.fi.dia.oeg.morph.base.querytranslator.MorphBaseQueryTranslator.translate(MorphBaseQueryTranslator.scala:1441)
at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner$$anonfun$1.apply(MorphBaseRunner.scala:182)
at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner$$anonfun$1.apply(MorphBaseRunner.scala:180)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.List.foreach(List.scala:318)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner.translateSPARQLQueriesIntoSQLQueries(MorphBaseRunner.scala:180)
at es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner.run(MorphBaseRunner.scala:162)
at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner$.main(MorphRDBRunner.scala:71)
at es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunner.main(MorphRDBRunner.scala)
at MorphAss.main(MorphAss.java:11)

Could you please fix it? Thanks

R2RML compliance: infer triples maps from properties

Hi Freddy, another minor issue I had told you about: according to R2RML spec, a triples map does not need an explict "rdf:type rr:TriplesMap", but it should be deduced from its properties: a logicalTable and a subjectMap or subject.

I've made it more simple: I only look for one rr:logicalTable, so that at least you get a message in case a triples map has a logicalTable but no subjectMap.

You may want to add this in the R2RMLMappingDocument.scala:

/**
 *  Add triples with rdf:type rr:TriplesMap for resources that have one rr:logicalTable
 */
private def inferTriplesMaps(model: Model) {
    val stmtsLogTab = model.listStatements(null, Constants.R2RML_LOGICALTABLE_PROPERTY, null)
    for (stmt <- stmtsLogTab) {
        val stmtType = model.createStatement(stmt.getSubject, RDF.`type`, Constants.R2RML_TRIPLESMAP_CLASS)
        model.add(stmtType)
    }
}

and call that method in the apply method:
...
model.read(in, null, "TURTLE");
inferTriplesMaps(model)
...

Cheers,
Franck.

Term maps with constant and non-constant properties

Hi Freddy, following our mail exchange I submit an issue (I'm still not sure how pull requests work ;-) ).
This is about method R2RMLTermMap.parse and the way it manage constant properties like.
Initially it could not handle cases where a term map is not defined as a blank node (aka. anonymous in Jena) but as a named resource, for instance:
... rr:predicateMap <#MyPred> ;
...
<#MyPred> rr:constant example:pred .

You removed the if-else from this:
if (rdfNode.isAnon()) {
...
} else {
this.constantValue = rdfNode.toString();
}
but I guess it does not fix everything. The problem is that the parse() method may be called in case of a constant or non-contanst property.

This way it works fine in the following cases:

1. [] objectMap [ rr:column "column" ];
2. [] predicateMap [ rr:constant ex:name ];

but now it fails in a constant property e.g.
[] rr:predicate ex:name
because the term map, in that case, has no property rr:column, rr:constant nor rr:template.
So finally, I did this: in case we find no property on the rdfNode (may it be anonymous or not), then I assume we are in the case of a constant property.
Here is my code :

/**
 * Decide the type of the term map (constant, column, reference, template) based on its properties,
 * and assign the value of the appropriate trait member: IConstantTermMap.constantValue, IColumnTermMap.columnName etc.
 *
 * If the term map resource has no property (constant, column, reference, template) then it means that this is a
 * constant term map like "[] rr:predicate ex:name".
 *
 * If the node passed in not a resource but a literal, then it means that we have a constant property wich object
 * is a literal and not an IRI or blank node, like in: "[] rr:object 'any string';"
 *
 * @param rdfNode the term map Jena resource
 */
def parse(rdfNode: RDFNode) = {
    this.rdfNode = rdfNode;

    if (rdfNode.isLiteral) {
        // We are in the case of a constant property with a literal object, like "[] rr:object 'NAME'",
        this.constantValue = rdfNode.toString()
    } else {
        val resourceNode = rdfNode.asResource();

        val constantStatement = resourceNode.getProperty(Constants.R2RML_CONSTANT_PROPERTY);
        if (constantStatement != null)
            this.constantValue = constantStatement.getObject().toString();
        else {
            val columnStatement = resourceNode.getProperty(Constants.R2RML_COLUMN_PROPERTY);
            if (columnStatement != null)
                this.columnName = columnStatement.getObject().toString();
            else {
                val templateStatement = resourceNode.getProperty(Constants.R2RML_TEMPLATE_PROPERTY);
                if (templateStatement != null)
                    this.templateString = templateStatement.getObject().toString();
                else {
                    // We are in the case of a constant property, like "[] rr:predicate ex:name",
                    this.constantValue = rdfNode.toString()
                }
            }
        }
    }
}

The last case (rr:object "NAME") helped me find another issue in MorphRDBDataTranslator.createLiteral(value: Object, datatype: Option[String], language: Option[String]) :
Line :
val data: String = datatype.get
to be changed into :
val data: String = datatype.getOrElse(null)

Cheers,
Franck.

Publishing a Docker image

Hey there 👋 Thank you for providing a docker-compose setup. Instead of users (like me) building the Docker image manually, what would be wonderful is to build such a Docker image and publish it to the Docker Hub. I see there is one but it looks pretty out-of-date. Is it part of your plan? Would you be interested in a PR which makes it part of the CI?

Security update needed in Log4j

https://securelist.com/cve-2021-44228-vulnerability-in-apache-log4j-library/105210/
https://nvd.nist.gov/vuln/detail/CVE-2021-45046

morph-rdb/pom.xml

Line 35 in a750f84

<log4j.version>2.8</log4j.version>

rr:constant and rr:parentTriplesMap without join conditions

Hi, I am working with LinkedSensorData and morph and have encountered the following problems.

This is my database table schema

describe _4UT01;
+------------------+---------------+------+-----+-------------------+-----------------------------+
| Field            | Type          | Null | Key | Default           | Extra                       |
+------------------+---------------+------+-----+-------------------+-----------------------------+
| time             | timestamp     | NO   |     | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| Precipitation    | decimal(10,0) | YES  |     | NULL              |                             |
| RelativeHumidity | decimal(10,0) | YES  |     | NULL              |                             |
| WindGust         | decimal(10,0) | YES  |     | NULL              |                             |
| WindSpeed        | decimal(10,0) | YES  |     | NULL              |                             |
| AirTemperature   | decimal(10,0) | YES  |     | NULL              |                             |
| DewPoint         | decimal(10,0) | YES  |     | NULL              |                             |
| WindDirection    | decimal(10,0) | YES  |     | NULL              |                             |
+------------------+---------------+------+-----+-------------------+-----------------------------+

+----------+
| count(*) |
+----------+
|     1151 |
+----------+

This is my ttl mapping

<TriplesMapWindSpeed>
  a rr:TriplesMap;
  rr:logicalTable [ 
    rr:tableName  "_4UT01" 
  ];
  rr:subjectMap [    
    a rr:Subject;
    rr:termType rr:IRI;  
    #rr:termType rr:BlankNode;  
    rr:template "http://test/data/{time}";
    #rr:template "1";
    rr:class <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#MeasureData>;
  ];
  rr:predicateObjectMap [ 
      rr:predicateMap [ rr:constant <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#floatValue> ]; 
      rr:objectMap    [ rr:termType rr:Literal; rr:column "WindSpeed"; ]; 
    ];
 rr:predicateObjectMap [ 
      rr:predicateMap [ rr:constant <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#uom> ]; 
      rr:objectMap    [ rr:constant <http://knoesis.wright.edu/ssw/ont/weather.owl#milesPerHour> ]; 
    ];         
.

<TriplesMapWindSpeedObservation>
  a rr:TriplesMap;
  rr:logicalTable [ 
    rr:tableName  "_4UT01" 
  ];
  rr:subjectMap [    
    a rr:Subject;
    rr:termType rr:IRI;  
    rr:template "http://test/obs/{time}";
    rr:class <http://knoesis.wright.edu/ssw/ont/weather.owl#WindSpeedObservation>;
  ];
  rr:predicateObjectMap [ 
      rr:predicateMap [ rr:constant <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#result> ]; 
      rr:objectMap    [ rr:parentTriplesMap <TriplesMapWindSpeed> ]; 
    ];
 rr:predicateObjectMap [ 
      rr:predicateMap [ rr:constant <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#observedProperty> ]; 
      rr:objectMap    [ rr:constant <http://knoesis.wright.edu/ssw/ont/weather.owl#_WindSpeed> ]; 
    ];    
.

So my first issue is with an objectMap rr:constant.

SELECT ?s ?o
WHERE { 
    ?s <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#uom> ?o.
}

This SPARQL query runs into a ERROR [main] (MorphRDBRunner.scala:76) - Exception occured: null exception.

The second issue is a query like this:

SELECT ?val
WHERE { 
    ?s a <http://knoesis.wright.edu/ssw/ont/weather.owl#WindSpeedObservation>;
        <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#result> ?result.
    ?result <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#floatValue> ?val.
}

Should return the values of 1151 results but returns INFO [main] (MorphXMLQueryResultWriter.scala:105) - 1324801 instance(s) retrieved. 1151 x 1151 (a full join on itself?).

According to the R2RML spec,


it seems like rr:constant for objectmap and rr:parentTriplesMap without join conditions are supported by the spec.

Thanks for your time!

I used the version specified in the morph user guide https://github.com/oeg-upm/morph-rdb/wiki (3.5.15) and I am using the https://github.com/AlexDisler/mysql-vagrant box.

+-------------------------+
| version()               |
+-------------------------+
| 5.5.46-0ubuntu0.12.04.2 |
+-------------------------+

Construct sparql queries

Hi,
Does morph-rdb support construct sparql queries? I have a relational database with a mapping into RDF and I would like to extract a subgraph given a starting node x. I found that this could happen using the construct mechanism in a sparql endpoint, but when I tried it with morph-rdb it seems to generate an error.

Invalid link

I am following the steps from the https://github.com/oeg-upm/morph-rdb/wiki page and I am at the part where I run a set of predefined R2RML mappings with MySQL and once I click on this link:
https://github.com/oeg-upm/morph-rdb/raw/master/morph-examples/examples/morph_example.sql
for the SQL example file, it says Page Not Found. Do I need to have this file in order to connect to the database and generate the R2RML mappings?

Thanks!