zazuko / rdf-validate-shacl

Validate RDF data purely in JavaScript. An implementation of the W3C SHACL specification on top of the RDFJS stack.

rdf-validate-shacl

Validate RDF data purely in JavaScript. An implementation of the W3C SHACL specification on top of the RDFJS stack.

We provide a SHACL playground based on this library.

Usage

The library only handles SHACL validation and not data loading/parsing. The following example uses rdf-utils-fs for this purpose. For more information about handling RDF data in JavaScript, check out Get started with RDF in JavaScript.

The validation function returns a ValidationReport object that can be used to inspect conformance and results. The ValidationReport also has a .dataset property, which provides the report as RDF data.

import rdf from '@zazuko/env-node'
import SHACLValidator from 'rdf-validate-shacl'

async function main() {
  const shapes = await rdf.dataset().import(rdf.fromFile('my-shapes.ttl'))
  const data = await rdf.dataset().import(rdf.fromFile('my-data.ttl'))

  const validator = new SHACLValidator(shapes, { factory: rdf })
  const report = await validator.validate(data)

  // Check conformance: `true` or `false`
  console.log(report.conforms)

  for (const result of report.results) {
    // See https://www.w3.org/TR/shacl/#results-validation-result for details
    // about each property
    console.log(result.message)
    console.log(result.path)
    console.log(result.focusNode)
    console.log(result.severity)
    console.log(result.sourceConstraintComponent)
    console.log(result.sourceShape)
  }

  // Validation report as RDF dataset
  console.log(await report.dataset.serialize({ format: 'text/n3' }))
}

main();

Validator options

The SHACLValidator constructor accepts an optional options object as second parameter. The available options are:

• factory: RDF/JS data factory (must have a .dataset() method)
• maxErrors: max number of errors after which the validation process should stop. By default, it only stops after all the errors are found.
• allowNamedNodeInList: SHACL only allows blank nodes in property lists. To allow named nodes to occur in property lists, set this value to true.

Running the tests

$ npm test

Limitations

rdf-validate-shacl does not support SHACL-SPARQL constraints

About

rdf-validate-shacl was originally a fork of shacl-js meant to make it compatible with RDF/JS libraries. Since then, we dropped support for the SHACL-JS extension and adapted the API to suit our needs.

rdf-validate-shacl's Issues

Improve validation message of `sh:xone`

I would propose tweaking the detail validation result message when an XONE constraint is violated

Currently it produces no message at all. Without even knowing which sub shapes are conflicted, it's quite difficult to get a grip on what is wrong.

I have a few ideas

First, could be to enumerate all of the sh:xone and produce a message like

Resource must be exactly one of A, B, C

Where A, B and C would be the rdfs:label or node value if label is not given. This would also apply to a focus node where none of the child shapes are matched

Alternatively, if we can get the info, could be to give the opposite and only mention the shapes which were matched and thus are in conflict

Resource cannot be A and C at the same time

Here B is hypothetically excluded

Function which extracts a JSON based on a SHACL definition

Loosly based on UC42.

Create a function which takes

1. An entry point (root node) term inside a dataset.
2. SHACL description provided with a term inside a dataset.
   and outputs a JSON structure described by the SHACL description.

.asJson(rootTerm, shaclTerm, shaclDataset)

Removing DASH

It turns out that all the validation logic resides in the DASH library and is linked to SHACL constraints in the DASH ontology. There is no way to "just remove DASH" without doing significant changes.

Extracting the validation logic from DASH will make the code much simpler and more maintainable, but it will also make it a lot more work to support SHACL-JS in the future if we decide to.

@bergos What do you think? Should I strip out DASH?

`minExclusive`, `maxExlusive` (etc.) handling of non-numerical literals

sh:minExclusive, sh:maxExlucive, sh.minInclusive and sh.maxInclusive should handle any type of Literal. Currently, the lib only handles numerical values.

API: ValidationReport

According to the spec, the validation process should return a graph that contains a sh:ValidationReport.

To be compliant, we could just return a Dataset that contains the report.

However, the original author of this lib took the decision to return a ValidationReport object that makes it easier to use the report (.conforms(), .results(), etc.). I think that makes sense, but the graph should still be accessible from the returned object.

My first intuition would be to keep the ValidationReport object as return value, but to add a .dataset or .graph property on it that provides access to the graph. But I don't know if it fits usual patterns in the RDF world.

@bergos do you have an opinion on what we should return?

maxErrors = 1 breaks sh:or constraint

From what we have in Cube Creator

<ObservationConstraintProperty> a sh:NodeShape;
  sh:property [
    sh:message "needs a schema:name" ;
    sh:or (
      [
        sh:path schema:name ;
        sh:minCount 1 ;
        sh:datatype xsd:string ;
      ]
      [
        sh:path schema:name ;
        sh:minCount 1 ;
        sh:datatype rdf:langString ;
      ]
      [
        sh:path sh:path ;
        sh:in (rdf:type cube:observedBy) ;
      ]
    ) ;
  ]

A property should pass validation if it satisfy any single one of these constraints. Right now, if one sets maxErrors = 1 (or at least a number lower than the "OR'ed" shapes) the validator will fail.

This happens because there is a single counter for the count of error encountered thus far. Each part of the sh:or list increases that counter, so when maxErrors === 1, the first failure will immediately break the validation.

I'm guessing that a false-negative would also be reported if there were violations found before the sh:or constraint. Thus, any dismissible error from one of the constituents will reach the allowed max and prevent remaining alternatives from being checked.

JS iterable error when using SHACL model

Depending on the structure of your SHACL shape and its list definitions, you may get obscure errors such as this:

Message:
   listNode.list is not a function or its return value is not iterable

Stacktrace:
   at rdfListToArray                 rdf-validate-shacl/src/dataset-utils.js:91            return [...listNode.list()].map(({ term }) => term)                             
   at validateClosed                 rdf-validate-shacl/src/validators.js:43               allowed.addAll(rdfListToArray(context.$shapes.node(ignoredPropertiesNode)))     
   at execute                        rdf-validate-shacl/src/validation-function.js:11      return this.func.apply(globalObject, [this.context, focusNode, valueNode, const…
   at validateNodeAgainstConstraint  rdf-validate-shacl/src/validation-engine.js:217       const obj = validationFunction.execute(focusNode, valueNode, constraint)        
   at validateNodeAgainstShape       rdf-validate-shacl/src/validation-engine.js:177       if (this.validateNodeAgainstConstraint(focusNode, valueNodes, constraint, dataG…
   at validateAll                    rdf-validate-shacl/src/validation-engine.js:154       if (this.validateNodeAgainstShape(focusNode, shape, dataGraph)) {               
   at validate                       rdf-validate-shacl/index.js:36                        this.validationEngine.validateAll(this.$data)                         

This seems to be caused by the clownface list() implementation returning null instead of an iterable (see here

I would have expected no error, or a more informative error instead.

Let me know if you need an MWE, and I'll see whether I can add the M to my WE

Expose `value` in validationReport

Hello,

Currently, most values in a report result are exposed. It would be nice to also have a getter for the value

API

There are still some things in the API that I would like to change, but I would like your opinion before doing it @bergos.

1. The API is very imperative & stateful and I don't see a good reason for it (heritage from Java?)

Currently, the flow looks like:

const validator = SHACLValidator({ factory })
validator.getConfiguration().setSomething(...)
const report = validator.validate(data, shapes)

And the validator also exposes methods like loadDataGraph, loadShapesGraph, showValidationResults, etc...

I would prefer something like:

const validator = SHACLValidator({ factory, ...allTheOptions })
const report = validator.validate(data, shapes)

And to be honest, my favourite version would be:

const validationReport = SHACLValidator.validate(data, shapes, { factory, ...allTheOptions })

I don't see a reason why the validator needs to be an object holding state. Reusing the same validator object to validate multiple datasets doesn't seem useful to me. But maybe I'm missing something?

2. Some functions should be properties, mostly on ValidationReport

• .conforms() -> .conforms
• .results() -> .results
• ...

Infinite loop with cyclic rdfs:subclassOf relationships

Lift syntactic restrictions on property path notation

User story

• [Who] As a user of this SHACL library,
• [What] I want to be able to use IRIs to encode complex property paths,
• [Why] So that I can use SHACL to validate my data in contexts where blank node Skolemization is applied.

Background

SHACL 1.0 puts syntactic restrictions on how complex property paths should be encoded. It is strange that such syntactic restrictions are mandated, since RDF lists may consist of blank nodes or IRIs or combinations of blank nodes and IRIs in any other context. In contexts where blank nodes are Skolemized to well-known IRI (a common practice that is part of the RDF 1.1 standard) complex property paths cannot be used at all.

A more in-depth discussion is over at w3c/data-shapes#137.

Would it be possible to lift this restriction on syntax in this library? Maybe using an option to indicate that this functionality purposefully deviates from the SHACL 1.0 standard in this respect.

Example

The following snippet gives a full example. The complex property path is encoded using IRI i.o. blank nodes. If the list IRI (<list-pq>) is replaced with the Turtle list notation (<p><q>) then the snippet works as intended.

prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix sh: <http://www.w3.org/ns/shacl#>
<shp>
  sh:property <pq-shp>;
  sh:targetSubjectsOf <p>.
<pq-shp>
  sh:class <C>;
  sh:minCount 1;
  sh:path <list-pq>.
<list-pq>
  a rdf:List;
  rdf:first <p>;
  rdf:rest <list-q>.
<list-q>
  a rdf:List;
  rdf:first <q>;
  rdf:rest rdf:nil.
<a> <p> <b>.
<b> <q> <c>.
<c> a <C>.

False negative when validating XoneConstraintComponent

shapes graph

@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix schema: <http://schema.org/> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix ex: <http://example.com/> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix dash: <http://datashapes.org/dash#> .

ex:PersonShape
  a sh:Shape ;
  sh:targetClass schema:Person ;
  rdfs:label "Person" ;
  sh:xone (
    [
      rdfs:label "Full name" ;
      sh:property [
        sh:name "Full name" ;
        sh:path ex:fullName ;
        # sh:minCount 1 ;
        sh:maxCount 1 ;
      ]
    ]
    [
      rdfs:label "First & last name" ;
      sh:property [
        sh:name "First name";
        sh:path ex:firstName ;
        # sh:minCount 1 ;
        sh:maxCount 1 ;
      ] ,
      [
        sh:name "Last name" ;
        sh:path ex:lastName ;
        # sh:minCount 1 ;
        sh:maxCount 1 ;
      ]
    ]
  ) ;
.

data graph

{
  "@context": {
    "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
    "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
    "xsd": "http://www.w3.org/2001/XMLSchema#",
    "schema": "http://schema.org/",
    "foaf": "http://xmlns.com/foaf/0.1/",
    "vcard": "http://www.w3.org/2006/vcard/ns#"
  },
  "@id": "http://example.com/John_Doe",
  "@type": "schema:Person",
  "http://example.com/fullName": "John Doe"
}

expected behaviour

The data should be valid.

actual behaviour

A violation is reported by sh:XoneConstraintComponent. It goes away when sh:minCount is uncommented

other

See this reply by @HolgerKnublauch and this exact example in Shaperone playground

Weird RDF list returned as validation result focusNode

I don't understand but there seems to be a bug in the validation results. Here's an example shapes graph:

@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix schema: <http://schema.org/> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix dash: <http://datashapes.org/dash#> .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
@prefix vcard: <http://www.w3.org/2006/vcard/ns#> .

@prefix ex: <http://example.com/> .
@prefix lexvo: <http://lexvo.org/id/iso639-1/> .

ex:PersonShape
  a sh:Shape ;
  sh:targetClass schema:Person ;
  rdfs:label "Person" ;
  sh:property 
                 ex:KnowsProperty ,
                 ex:AgeProperty ,
                 ex:GenderProperty ,
                 ex:SpokenLanguagesProperty ;
.

ex:KnowsProperty
  sh:path schema:knows ;
  sh:class schema:Person ;
.

ex:AgeProperty
  sh:path schema:age ;
  sh:name "Age" ;
  sh:datatype xsd:integer ;
  sh:maxCount 1 ;
  sh:defaultValue 21 ;
  sh:order 2 ;
  sh:minInclusive 18 ;
.

ex:GenderProperty
  sh:path foaf:gender ;
  sh:name "Gender" ;
  sh:in (
    "Male" "Female" "Other" "Prefer not to tell"
  ) ;
  sh:maxCount 1 ;
  sh:order 3 ;
  sh:message "Please select a valid gender" ;
.

ex:SpokenLanguagesProperty
  sh:path vcard:language ;
  sh:name "Spoken languages" ;
  sh:nodeKind sh:IRI ;
  sh:in (
    lexvo:en lexvo:de lexvo:fr lexvo:pl lexvo:es
  ) ;
  sh:order 5 ;
  sh:minCount 1 ;
.


lexvo:en rdfs:label "English" .
lexvo:de rdfs:label "German" .
lexvo:fr rdfs:label "French" .
lexvo:pl rdfs:label "Polish" .
lexvo:es rdfs:label "Spanish" .

and data graph

{
  "@context": {
    "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
    "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
    "xsd": "http://www.w3.org/2001/XMLSchema#",
    "schema": "http://schema.org/",
    "foaf": "http://xmlns.com/foaf/0.1/",
    "vcard": "http://www.w3.org/2006/vcard/ns#"
  },
  "@graph": [
    {
      "@id": "http://example.com/John_Doe",
      "@type": "schema:Person",
      "schema:age": {
        "@type": "xsd:integer",
        "@value": "17"
      },
      "schema:knows": [
        {
          "@id": "http://example.com/Jane_Doe",
          "@type": "schema:Person",
          "schema:name": "Janet"
        },
        {
          "@type": "schema:Person"
        },
        {
          "@type": "schema:Person"
        },
        {
          "@type": "schema:Person"
        }
      ],
      "schema:name": "John Doe",
      "foaf:gender": "Malee"
    }
  ]
}

When validated together, they return a bunch of results, some of which have the sh:focusNode looking like

[ a sh:ValidationResult ;
    sh:focusNode ( "Male" "Female" "Other" "Prefer not to tell" ) ;
    sh:resultMessage "Less than 1 values" ;
    sh:resultPath <http://www.w3.org/2006/vcard/ns#language> ;
    sh:resultSeverity sh:Violation ;
    sh:sourceConstraintComponent sh:MinCountConstraintComponent ;
    sh:sourceShape <http://example.com/SpokenLanguagesProperty> ],

What's with the list? It is by no means the "focus node that has caused the result" but also not completely random, given that it's clearly the sh:in list from another field.

Is this blank nodes IDs clashing?

Use of '<' character in report messages is confusing.

In the example below, the validator prints Value is not < value of <http://schema.org/deathDate>. The message is correct, however, the use of the character < tends to cause confusion, as the same characters, < and >, are also used to delimit IRIs. Would it be possible to use the words less than or greater than instead?

const n3 = require("n3");
const factory = require('rdf-ext');
const SHACLValidator = require('rdf-validate-shacl');

const shapesTtl = `
@prefix schema: <http://schema.org/> .
@prefix sh: <http://www.w3.org/ns/shacl#> .

schema:PersonShape
    a sh:NodeShape ;
    sh:targetClass schema:Person ;
    sh:property [
        sh:path schema:birthDate ;
        sh:lessThan schema:deathDate ;
    ] .
`;

const dataTtl = `
@prefix ex: <http://example.org/ns#> .
@prefix schema: <http://schema.org/> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

ex:Bob
    a schema:Person ;
    schema:birthDate "1971-07-07"^^xsd:date ;
    schema:deathDate "1968-09-10"^^xsd:date .
`;


async function run() {
  const parser = new n3.Parser();
  const shapes = await factory.dataset(parser.parse(shapesTtl));
  const data = await factory.dataset(parser.parse(dataTtl));

  const validator = new SHACLValidator(shapes, { factory });
  const report = await validator.validate(data);

  for (const result of report.results) {
    console.log(result.message[0].value);
  }
}

run();

Value does not have datatype error with sh:datatype xsd:anyURI

Properties with sh:datatype xsd:anyURI always produce Value does not have datatype <http://www.w3.org/2001/XMLSchema#anyURI> error.

I created a minimal example for the https://shacl-playground.zazuko.com/ playground.

Shapes Graph:

@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix g-validation:  <http://localhost:3001/shapes/validation#> .

@prefix g-framework: <http://localhost:3001/shapes/g-framework#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

g-validation:ServiceShape
	a sh:NodeShape ;
	sh:targetClass g-framework:Service ;
	sh:property [ sh:path g-framework:content ;
	                       sh:datatype xsd:anyURI ] ;
.

Data Graph:

{
    "@context": {
        "g-framework": "http://localhost:3001/shapes/g-framework#",
        "g-validation": "http://localhost:3001/shapes/validation#",
        "sh": "http://www.w3.org/ns/shacl#",
        "xsd": "http://www.w3.org/2001/XMLSchema#"
    },
  "@type": "g-framework:Service",
  "g-framework:content": "http://example.org/"
  
}

Checking for existence of instance of a class not working?

I want to check whether in the Data graph, there is at least one instance of dcat:Catalog. I followed this example from the SHACL wiki.
tl;dr: It does not work in Zazuko, it works in SHACL Play!, but I am not sure which behavior is the correct one.

Data graph - note that there is one instance of dcat:Catalog:

@prefix dcat:    <http://www.w3.org/ns/dcat#> .
@prefix dct:     <http://purl.org/dc/terms/> .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
@prefix vcard2006: <http://www.w3.org/2006/vcard/ns#> .

<https://data.gov.cz/lkod/mdcr/katalog> a dcat:Catalog ;
    dct:title "Katalog otevřených dat Ministerstva dopravy"@cs, "Open Data Catalog of the Ministry of Transport"@en ;
    dct:description "Otevřená data Ministerstva dopravy. Obsahuje datové sady o jízdních řádech a liniových vedeních veřejné dopravy."@cs;
    dct:description "Open data of the Ministry of Transport. It contains datasets regarding timetables of public transport."@en ;
    dct:publisher <https://rpp-opendata.egon.gov.cz/odrpp/zdroj/orgán-veřejné-moci/66003008> ;
    foaf:homepage <https://data.gov.cz/datové-sady?poskytovatel=Ministerstvo%20dopravy> ;
    dcat:contactPoint [
        a vcard2006:Organization ;
        vcard2006:fn "Ministerstvo dopravy, Odbor veřejné dopravy"@cs, "Ministry of Transport"@en ;
        vcard2006:hasEmail <mailto:[email protected]>
    ] ;
    dcat:dataset  <https://data.gov.cz/lkod/mdcr/datové-sady/seznam-zastavky> ,

Shapes graph:

@prefix dcat: <http://www.w3.org/ns/dcat#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix ex: <https://example.org/> .

# Shapes graph
ex:CatalogExistence
    a sh:NodeShape ;
    sh:targetNode dcat:Catalog ;
    sh:property [
      sh:path [ sh:inversePath rdf:type ];
      sh:minCount 1;
    ] .

Like this, it validates in Zazuko. However, if I remove the dcat:Catalog, e.g., by emptying the data graph, it still validates, which is not what I want. According to the example, and also in SHACL Play! it does not validate, which is what I want. However, I am not sure that the example is correct, since the targetNode cannot target anything, when there is no dcat:Dataset in the data graph, and therefore there is nothing to validate, and therefore the data graph seems valid. 🤷🏻‍♂️

Custom Factory implementation not fully supported

I am using a custom factory implementation and I thought, if I can supply the factory as an argument, it would be used throughout the module. The factory uses the standard rdf/js specified interface, but it only supports elements created by itself! That is the reason that my implementation throws an error, because clownface tries to use its own created nodes to access my datasets:

TypeError: Dataset#match - invalid predicate
    at Dataset.match (\Dataset.js)
    at Context.match (\node_modules\clownface\lib\Context.js:142:45)
    at Context.matchProperty (\node_modules\clownface\lib\Context.js:154:17)
    at Context.in (\node_modules\clownface\lib\Context.js:25:17)
    at \node_modules\clownface\lib\Clownface.js:270:81
    at Array.reduce (<anonymous>)
    at Clownface.in (\node_modules\clownface\lib\Clownface.js:270:35)
    at RDFLibGraph.getSubClassesOf (\node_modules\rdf-validate-shacl\src\rdflib-graph.js:38:10)
    at RDFLibGraph.getInstancesOf (\node_modules\rdf-validate-shacl\src\rdflib-graph.js:20:26)
    at new ShapesGraph (\node_modules\rdf-validate-shacl\src\shapes-graph.js:30:44)
    at SHACLValidator.loadShapes (\node_modules\rdf-validate-shacl\index.js:49:24)
    at new SHACLValidator (\node_modules\rdf-validate-shacl\index.js:21:10)
    at Dataset.shaclValidate (\Dataset.js)
    at Context.<anonymous> (\Dataset.test.js)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

The error is thrown, because the method only accepts its own NamedNodes and no arbitrary objects. That is exactly the behaviour I want and I am sad, that I cannot use this magnificient shacl module, because it does not create NamedNodes with the supplied factory :(

I consider this a bug and if it will not get fixed, my only option is to write a shacl validator myself!

(using version 0.2.6 of rdf-validate-shacl)

Unexpected sh:or SHACL violation

SHACL validation incorrectly creates a violation for the sh:or constraint when the NodeShape with the sh:or constraint also has a complex sh:property pointing towards a sh:node (Which is a NodeShape in itself).

Visit https://s.zazuko.com/RJmit for a minimum working example (mwe).
We expect the SHACL report to look like this:

{
    _:report a sh:ValidationReport ;
        sh:result [
            rdf:type sh:ValidationResult ;
            sh:resultSeverity sh:Violation ;
            sh:sourceConstraintComponent sh:NodeConstraintComponent ;
            sh:sourceShape <http://example.com/ns#educationPropShape> ;
            sh:focusNode <http://example.com/ns#Alice> ;
            sh:value <http://example.com/ns#bachelor> ;
            sh:detail [
                rdf:type sh:ValidationResult ;
                sh:resultSeverity sh:Violation ;
                sh:sourceConstraintComponent sh:MinCountConstraintComponent ;
                sh:sourceShape [
                    sh:path <http://example.com/ns#details> ;
                    sh:minCount 1 ;
                ] ;
                sh:focusNode <http://example.com/ns#bachelor> ;
                sh:resultPath <http://example.com/ns#details> ;
                sh:resultMessage "Less than 1 values" ;
            ] ;
            sh:resultPath <http://example.com/ns#education> ;
            sh:resultMessage "Value does not have shape <http://example.com/ns#bachelorNodeShape>" ;
        ] ;
        sh:conforms false .
}

In the mwe the sh:detail contains an additional SHACL violation that we don't expect to see in the SHACL report.

Track which quads are visited during validation

Provide a way to know which quads were visited during the validation process.

SHACL-JS

Hi,

Two questions, would you have an example I can run? I'm having trouble running the code in the readme.md as I'm unsure of the correct way to pass the shapes and data file into the SHACL validator. I tried the following, passing the file name as a string argument but this wont work.

const validator = new SHACLValidator('function.ttl') const report = validator.validate('data.ttl')

Also does your implementation support SHACL-JS extension?

Display sh:message in output

There is support for sh:message in the specification so one can add human readable explanations of the constraint.

It seems like this message is not passed to the report at the moment. Would be great to have that to improve the understanding of the problem.

Validationreport sourceShape is a blank node

When retrieving the sourceShape it returns a blank-node, it would be nice to get the IRI instead of the blank node label

import { runInContext } from "lodash";
import { Parser } from "n3";
const SHACLValidator = require("rdf-validate-shacl");

const dataTtl = `@prefix ex: <http://example.org/ns#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix schema: <http://schema.org/> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

ex:Bob
    a schema:Person ;
    schema:givenName "Robert" ;
    schema:familyName "Junior" ;
    schema:birthDate "1971-07-07"^^xsd:date ;
    schema:deathDate "1968-09-10"^^xsd:date ;
    schema:address ex:BobsAddress .
`;
const shapesTtl = `@prefix dash: <http://datashapes.org/dash#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix schema: <http://schema.org/> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

schema:PersonShape
    a sh:NodeShape ;
    sh:targetClass schema:Person ;
    sh:property [
        sh:path schema:givenName ;
        sh:datatype xsd:string ;
        sh:name "given name" ;
    ] ;
    sh:property [
        sh:path schema:birthDate ;
        sh:lessThan schema:deathDate ;
        sh:maxCount 1 ;
    ] ;
    sh:property [
        sh:path schema:gender ;
        sh:in ( "female" "male" ) ;
    ] ;
    sh:property [
        sh:path schema:address ;
        sh:node schema:AddressShape ;
    ] .
`;
const run = async () => {
  const parser = new Parser();
  const data = parser.parse(dataTtl);
  const shapes = parser.parse(shapesTtl);

  const validator = new SHACLValidator(shapes);

  const report = await validator.validate(data);

  for (const result of report.results) {
    console.log(result.sourceShape); // Returns a BlankNode
  }
};
run()

sh:hasValue does not work inside sh:or

When I specify sh:hasValue inside an sh:or, I have correct data that does not validate.

The following Turtle snippet should validate successfully, but currently results in a violation. My reasoning is as follows:

• id:a has value id:b for property def:p, so
• sh:hasValue id:b should be satisfied for id:a, so
• sh:or ( [ sh:hasValue id:b ] [ sh:hasValue id:c ] ) should be satisfied for id:a.

prefix sh: <http://www.w3.org/ns/shacl#>

prefix def: <https://triplydb.com/Triply/sh/def/>
prefix id: <https://triplydb.com/Triply/sh/id/>
prefix shp: <https://triplydb.com/Triply/sh/model/shp/>

shp:A
  sh:property shp:A_p;
  sh:targetClass def:A.

shp:A_p
  sh:or
    ( [ sh:hasValue id:b ]
      [ sh:hasValue id:c ] );
  sh:path def:p.

id:a
  a def:A;
  def:p id:b, id:d.

Inheritance works only up to one level

Recently, I have been trying to create shapes which inherit properties from other shapes.

Using this library, the validation went fairly well until I hit the second inherited level.

This means that having a construct of Shape C > Shape B and Shape B > Shape A, the properties of Shape B will be validated, the properties of Shape A however won't.

I tried this construct in the https://shacl.org/playground/ and this supports the structure, so I assume there is something not working correctly.

In the following I provide an example shape graph and data graph which validates correctly that the field "schema:name" is missing in the playground, however does not report that issue using this library.

Shapes Graph:

@prefix dash: <http://datashapes.org/dash#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix schema: <http://schema.org/> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix dcterms: <http://purl.org/dc/terms/> .

schema:ToolShape
    a sh:NodeShape , rdfs:Class ;
    sh:targetClass schema:ToolShape ;
    sh:property [
        sh:path schema:name ;
        sh:datatype xsd:string ;
        sh:minCount 1 ;
        sh:maxCount 1 ;
        sh:name "Name"@en, "Name"@de ;
    ] .

schema:SoftwareShape
    a sh:NodeShape , rdfs:Class ;
    rdfs:subClassOf schema:ToolShape ;
    sh:targetClass schema:SoftwareShape ;
    sh:property [
        sh:path schema:version ;
        sh:datatype xsd:string ;
        sh:maxCount 1 ;
        sh:name "Version"@en, "Version"@de ;
    ] ;
    sh:property [
        sh:path schema:creator ;
        sh:datatype xsd:string ;
        sh:minCount 1 ;
        sh:name "Creator"@en, "Ersteller"@de ;
    ] .

schema:AnalysisSoftwareShape
    a sh:NodeShape , rdfs:Class ;
    rdfs:subClassOf schema:SoftwareShape ;
    sh:targetClass schema:AnalysisSoftwareShape .

Data Graph:

[
   {
      "@type":"http://schema.org/AnalysisSoftwareShape",
      "http://schema.org/version":{
         "@value":"ü+",
         "@type":"http://www.w3.org/2001/XMLSchema#string"
      },
      "http://schema.org/creator":{
         "@value":"pop",
         "@type":"http://www.w3.org/2001/XMLSchema#string"
      }
   },
   {
      "@id":"http://schema.org/SoftwareShape",
      "http://www.w3.org/2000/01/rdf-schema#subClassOf":[
         {
            "@id":"http://schema.org/ToolShape"
         }
      ]
   },
   {
      "@id":"http://schema.org/AnalysisSoftwareShape",
      "http://www.w3.org/2000/01/rdf-schema#subClassOf":[
         {
            "@id":"http://schema.org/SoftwareShape"
         }
      ]
   },
   {
      "@id":"http://schema.org/ToolShape"
   }
]

How to validate literals based on their datatype IRI?

I do not understand how literals should be validated based on their datatype IRI. I make the following observations:

1. For some literals specifying the datatype IRI with sh:datatype seems to suffice in order to also check their lexical form. An example of this is xsd:boolean, where lexical form "-false" is currently not accepted because the minus sign is not part of the syntax for Boolean lexical forms.

2. For some literals specifying the datatype IRI with sh:datatype does not seem sufficient, since incorrect lexical forms are still accepted. An example of this is xsd:double for which "--1.1e0" is accepted, even though the double occurrence of the hyphen is not supported by the floating-point syntax.

3. At the same time, it is also not clear how regular expressions could be manually specified in order to fix the absence of lexical form validation (see #44 for generic issues with the way in which regular expressions are currently supported). For example, specifying the regular expression sh:pattern "(\\+|-)?([0-9]+(\\.[0-9]*)?|\\.[0-9]+)" copied from the XSD standard alongside sh:datatype xsd:double still allows validates literals like "--1.1e1"^^xsd:double as ok, even though they violate both the datatype IRI and the regular expression specifications.

At the moment it is difficult for me to determine what is intended behavior and what is a bug. It would be great if SHACL could be used to validate literals, but I am not sure whether (1) such validation is indeed intended by the SHACL standard, and whether (2) it is technologically feasible to implement such validation with contemporary technology.

Validation report message not included when using recursive shapes

Validation report message not included when using recursive shapes. Tested with sh:and

Example:

@prefix dash: <http://datashapes.org/dash#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix schema: <http://schema.org/> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix dataid: <http://dataid.dbpedia.org/ns/core#> .
@prefix dct:   <http://purl.org/dc/terms/> .
@prefix dcat:  <http://www.w3.org/ns/dcat#> .
@prefix dcv: <http://dataid.dbpedia.org/ns/cv#> .

##########
# Group
##########

<#dataid-group-test>   
    a sh:NodeShape ;
    sh:targetClass dataid:Group ;
    sh:and (<#en-title>) .

<#en-title>   
    a sh:PropertyShape ;
    sh:severity sh:Violation ;
    sh:message "Required property dct:title MUST occur at least once AND have one @en " ;
    sh:path dct:title ;
    sh:minCount 1 ;
    sh:languageIn ("en") ;
    sh:uniqueLang true .

Validation failure due to the PropertyShape will not give access to the error message.

`sh:closed` with `sh:alternativePath`

What's the issue?
When using a closed shape and properties with an alternative path, I always get a violation because the predicate is not allowed.

What's expected?
I am expecting predicates schema:ROLE_CAREGIVER to not be violations of the closed shape.

Example SHACL

schema:User a rdfs:Class, sh:NodeShape;
    sh:closed true;
    sh:property [ sh:path rdf:type; sh:hasValue schema:User ];
    sh:property [
      sh:path [ sh:alternativePath ( schema:ROLE_CAREGIVER schema:ROLE_CAREMANAGER ) ];
      sh:node schema:CareRecipient
    ];
.

Playground example
https://s.zazuko.com/JrT3P

---

Is that a real bug or am I misunderstanding SHACL? Either way, would you have any direction as to which part of the code should I change and PR or how might I be able to tackle this (without redeclaring each property one by one, hopefully)? Thank you!

Closed shapes lead to a TypeError

When trying to validate a closed shape (i.e. with sh:closed true), the following error is thrown:

(node:6649) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'equals' of undefined
    at RDFLibGraph.rdfListToArray (/home/nseydoux/dev/sandbox/shapes-hal/node_modules/rdf-validate-shacl/src/rdflib-graph.js:74:22)
    at validateClosed (/home/nseydoux/dev/sandbox/shapes-hal/node_modules/rdf-validate-shacl/src/validators.js:35:36)
    at ValidationFunction.execute (/home/nseydoux/dev/sandbox/shapes-hal/node_modules/rdf-validate-shacl/src/validation-function.js:11:22)
    at ValidationEngine.validateNodeAgainstConstraint (/home/nseydoux/dev/sandbox/shapes-hal/node_modules/rdf-validate-shacl/src/validation-engine.js:213:42)
    at ValidationEngine.validateNodeAgainstShape (/home/nseydoux/dev/sandbox/shapes-hal/node_modules/rdf-validate-shacl/src/validation-engine.js:171:16)
    at ValidationEngine.validateAll (/home/nseydoux/dev/sandbox/shapes-hal/node_modules/rdf-validate-shacl/src/validation-engine.js:144:20)
    at SHACLValidator.validate (/home/nseydoux/dev/sandbox/shapes-hal/node_modules/rdf-validate-shacl/index.js:35:27)
    at main (/home/nseydoux/dev/sandbox/shapes-hal/index.js:51:36)
    at Object.<anonymous> (/home/nseydoux/dev/sandbox/shapes-hal/index.js:68:1)
    at Module._compile (internal/modules/cjs/loader.js:1147:30)

To reproduce

Run the following code snippet:

const SHACLValidator = require("rdf-validate-shacl")
const {Parser} = require("n3")

const ttlData = `
@prefix ex: <http://example.org#>.
@prefix xsd: <http://www.w3.org/2001/XMLSchema#>.

ex:Alice
a ex:Person ;
ex:ssn "987-65-432A" .

ex:Bob
a ex:Person ;
ex:ssn "123-45-6789" ;
ex:ssn "124-35-6789" .

ex:Calvin
a ex:Person ;
ex:birthDate "1971-07-07"^^xsd:date ;
ex:worksFor ex:UntypedCompany .`

const ttlShape = `
@prefix ex: <http://example.org#>.
@prefix xsd: <http://www.w3.org/2001/XMLSchema#>.
@prefix sh: <http://www.w3.org/ns/shacl#>.
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.

ex:PersonShape
a sh:NodeShape ;
sh:targetClass ex:Person ;    # Applies to all persons
sh:property [                 # _:b1
    sh:path ex:ssn ;           # constrains the values of ex:ssn
    sh:maxCount 1 ;
    sh:datatype xsd:string ;
] ;
sh:property [                 # _:b2
    sh:path ex:worksFor ;
    sh:class ex:Company ;
    sh:nodeKind sh:IRI ;
] ;
#sh:closed true ;
sh:ignoredProperties ( rdf:type ) .`

async function main() {
    const parser = new Parser();
    const data = parser.parse(ttlData);
    const shapes = parser.parse(ttlShape);
    
    const validator = new SHACLValidator(shapes)
    console.log("coucou1")
    const report = await validator.validate(data)
     console.log("coucou2")
    // Check conformance: `true` or `false`
    console.log(report.conforms)
     
    for (const result of report.results) {
      // See https://www.w3.org/TR/shacl/#results-validation-result for details
      // about each method
      console.log(result.message)
      console.log(result.path)
      console.log(result.focusNode)
      console.log(result.severity)
      console.log(result.sourceConstraintComponent)
      console.log(result.sourceShape)
    }
} 

main()

The rdf-validate-shacl version is 0.1.1.

SHACL SPARQL constraints support

Is adding SHACL SPARQL constraints support into this library, currently on the roadmap of the development?

ValidationReport: properties should return RDF terms instead of strings

• ValidationReport.conforms -> boolean
• ValidationResult.message -> Literal[] (there can be multiple messages in multiple languages)
• ValidationResult.path -> Term
• ValidationResult.focusNode -> Term
• ValidationResult.severity -> Term?
• ValidationResult.sourceConstraintComponent -> Term
• ValidationResult.sourceShape -> Term

Superfluous validation error when target node doesn't exist in data graph

What's the issue?

A violation is raised when a node that doesn't exist in the data graph is used as an object of sh:targetNode.

What's expected?

I expected the validation report to conform, as the target node doesn't exist in the data graph. This is based off this part of the SHACL spec:

RDF terms produced by targets are not required to exist as nodes in the data graph.

Playground example

In my first example id:c exists as a node in the data graph, while in the second one id:d doesn't.

Feature request: add TypeScript types?

Hi all!

Thanks for this great library! :)

Would it be possible to add a declaration file with TypeScript types?

New release

Hi! I'm really happy with the fix in #49, but cannot use it at the moment because it is not included in a release yet. Would you mind creating one?

Thanks a lot!

Shape inheritance only works when single dataset is used

The validator supports shape hierarchies using rdfs:subClassOf but they are not applied correctly if the data graph and shapes graph are separate instances of Dataset

Consider this example.

The result is false-positive, even though the validated instance should be validated as foaf:Agent.

The correct results is returned if a single dataset is populated, for example using actual named graphs:

const dataset = $rdf.dataset()
const dataGraph = clownface({ dataset, graph: $rdf.namedNode('data-graph') })
const shapesGraph = clownface({ dataset, graph: $rdf.namedNode('shapes-graph') })

debug should be included in dependencies not devDependencies

The debug package is used here as part of the main code and not test or developer tool. Therefore it should be a normal dependency and not a dev dependency.

Export property path matching

Also for other use cases than validation I want to be able, from an array of quads, extract the rdfjs.Terms that match the property path:

matchPropertyPath<RDFJS.Term> (quads: RDFJS.Quad[], subject: RDFJS.Term, path: PropertyPath);

I assume the function on L61 in https://github.com/zazuko/rdf-validate-shacl/blob/master/src/property-path.js#L61 comes pretty close to what I need. Can this functionality be exported separately in this NPM package?

Unsure whether dash and SPARQL selects are implemented

Is targeting via SPARQL implemented? If so, what might I be doing wrong?
My Shapes Graph:

@prefix dash: <http://datashapes.org/dash#> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .

<http://example.org/shapes> owl:imports <http://datashapes.org/dash> .

<http://example.org/shapes#allblanks>
	a sh:NodeShape ;
	sh:target dash:AllObjects ;
	sh:nodeKind sh:BlankNodeOrIRI .

My Data Graph

@prefix dc: <http://purl.org/dc/terms/> .

<https://some.uri.org> dc:title "Hello World"@en .

This yields SUCCESS while I am trying to make this fail - I want to verify that all objects are either blanks or IRIs (no Literals allowed).

I have read that SPARQL-based Constraints are not implemented, was unsure about SPARQL-based Targets

Validation result does not include a sh:resultMessage (and sometimes sh:value)

We encountered two small bugs:

1. the validation result does not include a sh:resultMessage when a ClassConstraintComponent is violated.

2. In some cases, the validation result also does not include a sh:value for violations of ClassConstraintComponent. (Some validation results do emit sh:value; maybe this only works for literals ATM)

We made a minimal working example of the issues. To make it work, download the zip file, and use the following in the Linux terminal:

cd MWE # (given the location where it was extracted)
node script.js

The two most important files in the working example are data.ttl and shapes.ttl.

MWE.zip

Add warning result when maxErrors is reached

I suppose it would be as simple as a top-level result similar to

[] a sh:ValidationReport ;
   sh:result [
      sh:resultSeverity sh:Warning ;
      sh:resultMessage "The maximum number of error (100) was reached. Not all violations may have been reported" ;
   ] ;
.

Unfortunately, according to paragraph 3.6.2 of the spec, sh:focusNode and sh:sourceConstraintComponent are also required. To conform to this requirement we might add some new term to identify the constraint, like ex:MaxErrorConstraintComponent. I have no (good) idea for the focus node other than a new blank node

Implement a SHACL Playground Application

The current playground application should be updated to use this library + standard RDFJS libraries.

Regular expressions with Unicode

Thanks for maintaining this great library!

Observation

I'm unable to properly validate place names using sh:pattern. Place names may include spaces, single quotes, hyphens, and some non-ASCII Unicode characters. Examples of place names that should succeed are 's-Gravenhage, The Hague, and Köln.

If I understand the somewhat cryptic XSD standard (link), then this should be expressible in the following way:

prefix sh: <http://www.w3.org/ns/shacl#>
[ sh:property
    [ sh:pattern "\\p{S}+";
      sh:path <label> ];
  sh:targetClass <C> ].

But the following data does not validate:

[ a <C>;
  <label> "Köln" ].

Since many natural languages include characters that do not occur in simple ASCII ranges like [A-Za-z], and because natural language information is very common in RDF data, support for validating Unicode strings in sh:pattern is useful in many cases.

Expected

The ability to use category escapes in sh:pattern, specifically for natural language content for which simple ranges are difficult/impossible to express.

Add resultMessage for class constraint

Thanks for continuing to improve this library!

Observation

Currently the class constraint component produces no message at all. This is a missed opportunity. Together with issue 63 Adding a result message would tie up all of the default constraints.

Expectation

I would propose tweaking the detail validation result message when an Class constraint is violated.

A proposed message could be of type, :

Value is not of type {$class}

Where $class cuold be the rdfs:label if there is a label and otherwise the node value if there is no label is not given.

Wrong pair constraint comparison using lessThan / lessThanOrEquals

Using property constraint validation with sh:lessThan or sh:lessThanOrEquals I get wrong result in Zazuko SHACL Plaground (https://s.zazuko.com/d56u3).

From my observation, it looks like only the first digit is being compared. For example, if the value of ex:candidates_active is 65, there is no validation error which is obviously fine, but when I change to 8 or 9, I get a validation error.

Thanks in advance.

Provide a CLI

Would it be nice to have a simple CLI provided by this package. I would propose a simple interface which would load data+shapes to memory and offer at least two output "modes"

Standard mode

Writes the validation report to standard output, optionally using type specified in --format

npx rdf-validate-shacl \
  --shapes Person.shape.ttl \
  --data people.nq \
  --format application/ld+json

Without --format it would send n-quads so that the report can be piped easily to other tools

npx rdf-validate-shacl | other-cli-tool

Quiet mode

This would print nothing just return an exit code != 0 when validation failed

npx rdf-validate-shacl \
  --shapes Person.shape.ttl \
  --data people.nq \
  --quiet

Globs for loading the graphs

npx rdf-validate-shacl --shapes shapes/*.ttl --data input/*

Validate a specific target node with a specific shape

Provide a way to specify an entry target node and shape for the validation. The result is similar to injecting a temporary sh:targetNode on a given shape and start the validation for this shape.

Memory issue with nested results

The validation of large datasets can cause memory-problems. The object nestedResult of class ValidationEngine is not cleaned up when validation of a node is successful.
Adding a cleaning action in createResultFromObject() solved the problem for me:

// Validation was successful. No result.
if (!validationResultObj) {

  // 20220806/mv: Clean the nested results of the report-children
  **if (this.nestedResults[this.recordErrorsLevel + 1]?.length)
    this.nestedResults[this.recordErrorsLevel + 1] = [];**

  return null
}

But, thanks for the good work! Really a nice module.

Customization of default failure messages

Is there any opportunity for specifying custom failure messages for different types of failures (texts, that are defined in validators-registry.js)? I want to add localization to my project, so I need to replace these messages with translated ones. Thank you in advance for the answer :)

Validation error when using correct lexical value for xsd:gYear

When instance data contains a xsd:gYear literal in where the year value contains more than 4 digits, and a SHACL PropertyShape uses a DatatypeConstraintComponent', the SHACL validation report incorrectly marks this value as an error: with this message:

Value does not have datatype <http://www.w3.org/2001/XMLSchema#gYear>.

The W3C XML Schema Definition Language (XSD) 1.1 Part 2: Datatypes defines a gYear with this lexical space, so literals like '10000' are valid gYear literal values:

yearFrag ::= '-'? (([1-9] digit digit digit+)) | ('0' digit digit digit))
gYearLexicalRep ::= yearFrag timezoneFrag?

I assume there are similar issues with other XSD date-like datatypes, for sure with gYearMonth (see in this example code), etc.

The issue is caused by a RegExp that only allows 4 digits for a year in the 'ref-validate-datatype' module, see here. I've created a pull request to implement the correct RexExp.

How to reproduce

import fs from 'fs'
import factory from 'rdf-ext'
import ParserN3 from '@rdfjs/parser-n3'
import SHACLValidator from 'rdf-validate-shacl'
import assert from 'assert'

async function loadDataset (filePath) {
  const stream = fs.createReadStream(filePath)
  const parser = new ParserN3({ factory })
  return factory.dataset().import(parser.import(stream))
}

const shapes = await loadDataset('shapes.ttl')
const data = await loadDataset('data.ttl')
const validator = new SHACLValidator(shapes, { factory })
const report = await validator.validate(data)
if (report.conforms === false) {
  console.error('Expected report to validate, it did not:')
  for (const result of report.results) {
    console.error(`'${result.message} on path ${result.path}`)
  }
}

Content of file data.ttl

prefix ex: <http://ex.com/>
prefix xsd: <http://www.w3.org/2001/XMLSchema#> 
ex:Thing a ex:Thing; 
	ex:year "10000"^^xsd:gYear ;
	ex:yearMonth "10000-10"^^xsd:gYearMonth .

Content of file shapes.ttl

prefix sh: <http://www.w3.org/ns/shacl#>
prefix ex: <http://ex.com/>
prefix xsd: <http://www.w3.org/2001/XMLSchema#> 

ex:Shape a sh:NodeShape ;
	sh:targetClass ex:Thing ;
	sh:property ex:gYearProperty, ex:gYearMonthProperty .

ex:gYearProperty a sh:PropertyShape ;
	sh:path ex:year ;
	sh:datatype xsd:gYear .
	
ex:gYearMonthProperty a sh:PropertyShape ;
	sh:path ex:yearMonth ;
	sh:datatype xsd:gYearMonth .

Code Snippet in README

The code snippet in the README throws an error:

SyntaxError: await is only valid in async function

This can easily be solved by putting the code in an async function (e.g., main) and calling it right after.

I could submit a small PR. What do you think?

[Feature request] Extracting shapes from triples

I’m looking for a library that can help me to extract triples adhering to a SHACL shapes from an array of triples.

Could look like this:

const data = await loadDataset('my-data.ttl')
const shapeData = await loadDataset('personShape.ttl')
const objectIterator = extractObjects(data, shapeData, "ex:PersonShape1")
for await (const object of objectIterator) {
    console.log(object); // contains an occurence of array of triples according to this specific shape.
}

Use case issue: TREEcg/event-stream-client#2