comunica / rdf-test-suite-ldf.js Goto Github PK

2.0 3.0 0.0 879 KB

✅ Execute engine-test suites on QUERY-engines

License: MIT License

JavaScript 6.72% TypeScript 93.28%

rdf-test-suite-ldf.js's Introduction

rdf-test-suite-ldf.js

This is a nodejs CLI-tool which executes integration-tests on query-engines such as the comunica-engines and is based on the rdf-test-suite.js written by Ruben Taelman.

It uses test-manifests (you can find examples on the manifest-ldf-tests repository) which are based on the ontology-query-testing to test the integration of your engine.

Installation

Either install it globally:

$ yarn global add rdf-test-suite-ldf

or locally (as a dev dependency):

$ yarn add --dev rdf-test-suite-ldf

Usage

When the rdf-test-suite-ldf script is installed you're good to go. You will need some kind of javascript-engine which follows the ILdfQueryEngine interface as first argument and an engine-test manifest URI as second argument.

Basic execution

The following command executes the sparql-manifest test suite on the engine: myengine.js:

$ rdf-test-suite-ldf myengine.js https://comunica.github.io/manifest-ldf-tests/sparql/sparql-manifest.ttl

This command will output something like this:

✔ SELECT - DBpedia TPF (https://comunica.github.io/manifest-ldf-tests/sparql/sparql-manifest.ttl#directors01)
✔ SELECT - DBpedia TPF (https://comunica.github.io/manifest-ldf-tests/sparql/sparql-manifest.ttl#software02)
✔ SELECT - DBPedia TPF & SPARQL (https://comunica.github.io/manifest-ldf-tests/sparql/sparql-manifest.ttl#simple03)
✔ 3 / 3 tests succeeded!

Start port for mocking

With the -r flag you can decide on which port mock-servers will spawn, e.g. 5000. If an invalid port is given an error will be thrown. If the port is in use rdf-test-suite-ldf will wailt for a specific amount of time if the port becomes available. If it doesn't it will throw an error.

$ rdf-test-suite-ldf myengine.js https://comunica.github.io/manifest-ldf-tests/sparql/sparql-manifest.ttl -r 6000

Extra details on HTTP caching

The -c flag inherited from rdf-test-suite.js does also work in rdf-test-suite-ldf, especially for HDT- and RDFJS-testing. All the files for HDT- and RDFJS-testing should and will be (temporarily) stored in the given local cache folder. These files are deleted by default but when the -c flag is given, all those files will be stored in the directory given with this flag. These files can be used the next time to avoid re-fetching files from servers.

$ rdf-test-suite-ldf myengine.js https://comunica.github.io/manifest-ldf-tests/sparql/sparql-manifest.ttl -c path/to/dir

Delay server termination

With the -v flag you can define the time to wait before stopping the server after each completed test. This is useful in case your engine could perform additional HTTP to the server, even after the query has been done.

$ rdf-test-suite-ldf myengine.js https://comunica.github.io/manifest-ldf-tests/sparql/sparql-manifest.ttl -v 100

Defaults to 10.

Extra options

A list of extra options, inherited from rdf-test-suite.js can be found here.

Some default test suites

I created and published some test-suites for the 4 engines of comunica which can be found here.

License

This software is written by Manu De Buck and based on software written by Ruben Taelman. This code is released under the MIT license.

rdf-test-suite-ldf.js's People

Contributors

Stargazers

Watchers

rdf-test-suite-ldf.js's Issues

Fix bug causing file-testing to break

The mock-class is breaking file testing. There should be some functionality to know which sources should be mocked and which one should not. This will probably need some aid from engine-ontology to define the sourceType of every separate source.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Warning

These dependencies are deprecated:

Datasource	Name	Replacement PR?
npm	`tslint`

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Update dependency fs-extra to v11 (fs-extra, @types/fs-extra)
Update dependency rdf-parse to v3
Update dependency typescript to v5
Click on this checkbox to rebase all open PRs at once

Detected dependencies

npm

package.json

@types/fs-extra ^8.0.0

@types/node-fetch ^2.5.0

fs-extra ^9.0.0

global ^4.4.0

minimist ^1.2.0

n3 ^1.6.3

rdf-parse ^2.0.0

rdf-store-stream ^2.0.0

rdf-test-suite ^1.18.0

tcp-port-used ^1.0.1

winston ^3.2.1

@types/jest ^29.0.0

@types/minimist ^1.2.0

@types/n3 ^1.1.4

jest ^29.0.0

jest-rdf ^1.3.0

manual-git-changelog ^1.0.1

nock ^13.0.0

pre-commit ^1.2.2

ts-jest ^29.0.0

tslint ^6.0.0

tslint-eslint-rules ^5.4.0

typescript ^4.6.2

Check this box to trigger a request for Renovate to run again on this repository

Find a better way to check if something is whitelisted

i.e.: http://ex.org/?s=a&p=b&o=c will be whitelisted (http://ex.org/) if http://ex.org/sparql/ is a one of the datasources...

See: LdfResponseMocker for the code.

Add logging to the CLI

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: Cannot find preset's package (github>rubensworks/renovate-presets:js)

Remove all: const X = require('X')

Fix npm issue

The package is published, but when installing the package is unable to use...

Invalid value "undefined" for header "Content-Type"

When executing SPARQL test simple03 with Comunica, I get the following error:

(node:83950) UnhandledPromiseRejectionWarning: TypeError [ERR_HTTP_INVALID_HEADER_VALUE]: Invalid value "undefined" for header "Content-Type"
    at storeHeader (_http_outgoing.js:411:5)
    at processHeader (_http_outgoing.js:406:3)
    at ServerResponse._storeHeader (_http_outgoing.js:312:9)
    at ServerResponse.writeHead (_http_server.js:271:8)
    at mockFetcher.parseMockedResponse.then (/Users/rtaelman/Documents/UGent/nosync/rdf-test-suite-ldf.js/lib/testcase/ldf/mock/LdfResponseMocker.js:54:38)
    at process._tickCallback (internal/process/next_tick.js:68:7)
(node:83950) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 5)
(node:83950) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

I've done some debugging, and it looks like the content type header is not available in the mocked file.
@ManuDeBuck Is this a problem in the recorder? Or should the test suite runner be able to handle these empty content types?

Add functionality for TPF testing

Add tests for EvaluationHandlers

Add tests for the evaluatianhandlers used for all the TPF testing.

Make sure different source types are tested
Make sure different combinations of source types are tested

Fix bug causing failing tests to count as succeeded test

This seems to be introduced anywhere in my code.

Add documentation for npm package

Add CLI parameters

A user should be able to:

add parameter that selects the portnumber by which the dummyservers start spawning
add parameter to cachefolder for hdt-index files
add parameter that says if the cachefolder should be deleted after the tests

Yarn produces warnings from `url-exists` when installing as a dependency

Yarn produces a number of warnings when installing this package as a dependency in Comunica:

warning rdf-test-suite-ldf > url-exists > [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142
warning rdf-test-suite-ldf > url-exists > request > [email protected]: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
warning rdf-test-suite-ldf > url-exists > request > [email protected]: this library is no longer supported

Maybe the url-exists package could be updated at some point if an update exists.

Create a mockHandler class that handles all mocking

If multiple sources are provided (TPF, File, SPARQL...) one class should stand in for mocking all the sources, if necessary. This avoids code duplication and improves performance and code-readability

Add functionality for RDFJS testing

Servers are sometimes not being closed properly

Error: listen EADDRINUSE: address already in use :::3000

    at Server.setupListenHandle [as _listen2] (net.js:1279:14)

    at listenInCluster (net.js:1327:12)

    at Server.listen (net.js:1414:7)

    at LdfResponseMocker.<anonymous> (/home/travis/build/comunica/comunica/node_modules/rdf-test-suite-ldf/lib/testcase/ldf/mock/LdfResponseMocker.js:31:62)

Required for merging comunica/comunica#501

An in-range update of @types/n3 is breaking the build 🚨

The devDependency @types/n3 was updated from `1.1.3` to `1.1.4`.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@types/n3 is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details

❌ continuous-integration/travis-ci/push: The Travis CI build could not complete due to an error (Details).

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.

Your Greenkeeper Bot 🌴

Add parallel flag

It could be an idea to add a flag which lets the users decide wheter or not to run the tests parallel.

Add flag to enable/ disable info logging

Add a CLI flag that can enable or disable logging, default: disabled.

Improve caching of files

Fix bug in caching, make use of comunica/comunica#537 (comment) instead of current solution.

Add colors to usage description

Add functionality for SPARQL-endpoint testing

Make tpf-recorder cross-platform

Make use of nodejs' path module, instead of hardcoding forward slashes: examples here

cachePath = Path.join(process.cwd(), (args.c === true ? '.rdf-test-suite-cache/' : args.c));

Improve usage of `LdfResponseMocker`

Only use LdfResponseMocker for sourceTypes needing them. This can improve performance for tests that don't make use of mocked servers. (cfr. comunica/comunica#537 (comment))

An in-range update of nock is breaking the build 🚨

The devDependency nock was updated from `11.8.2` to `11.9.0`.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

nock is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details

❌ continuous-integration/travis-ci/push: The Travis CI build could not complete due to an error (Details).
✅ coverage/coveralls: First build on greenkeeper/nock-11.9.0 at 79.167% (Details).

Release Notes for v11.9.0

11.9.0 (2020-02-13)

Features

allow passing a function to enableNetConnect() (#1889) (566461b)

Commits

The new version differs by 11 commits.

a275769 test: Update got to the latest version and fill in missing coverage (#1825)
566461b feat: allow passing a function to enableNetConnect() (#1889)
11c0542 Require Node 10+ (#1895)
e04d61d Update mkdirp to the latest version 🚀 (#1857)
9379f09 Do not return the nock global from cleanAll() (#1872)
6c504c3 Drop support for String constructor (#1873)
1fae725 refactor(test): use Mocha DSL for allow unmocked (#1894)
d0694a7 ci: move semantic-release to GitHub Action
524dd29 test: fix assertRejects usage (#1890)
5d3b270 refactor(test): Chai assert allow unmocked (#1891)
5eaa5f5 Bump lolex and sinon (#1887)

See the full diff