sosna / sdmx-rest4js

JavaScript client for SDMX services

Hi there 👋

I have been working in IT since 1999, with a focus on statistical systems since 2005. During these years, I have developed solutions that handle the ingestion, processing, analysis and dissemination of data.

I strive to develop solutions that are, first of all, correct (no kidding ;)), well-crafted, open to changes, standard-compliant, metadata-driven and well-tested.

I'm the author of the NPM sdmx-rest package, one of the authors of pysdmx and one of the maintainers of the SDMX-REST specification.

A few stats

Favorite IT Motto

In the face of ambiguity, refuse the temptation to guess (PEP 20).

sdmx-rest4js's Issues

Add support for version 1.3.0 of the SDMX RESTful API

A new version of the SDMX RESTful API should be released early October.

Secure default instances

The library offers access to web service instances but these use http. It would be nice to add the option to access the instance over https for the services that support that.

Reduce complexity of isValidQuery

The isValid queries (under data query and availability query) could be simplified.

Add support for the new parameter for hierarchicacodelist queries

The next version of the SDMX RESTful API will add the option to query for hierarchies in an hierarchical codelist (see sdmx-twg/sdmx-rest#33). It would be nice to add support for this.

Programatically list the available/supported services

It would be great for an application using this library to be able to present the user with a drop-down menu of the available/supported services.

If I'm not mistaken, currently the Service class has constants defining the individual services but nowhere is an array of them available.

Doc: wiki service page not up to date

I see on this wiki page that a few of the services defined in the code are not described.

This Where is it used page can help get the descriptive content for the still undefined services.

Review checkStatus

We might need to relax checkStatus() a bit, for example to allow any of the 3** codes.

Decrease complexity of getUrl

Complexity of getUrl can probably be lowered a bit.

Should be possible to use a string as input to execute()

The execute function currently expect a query, a service and (optionally) a header object. It would however be nice to also accept a string representing an SDMX RESTful query.

Add providers

At the moment, the ECB web service is the only predefined instance offered by the library. However, there are other implementers that could be added.

Add function to check the media types

The API offers a method to check the status of a request.

The same should be done for the media types, i.e. is the response type a valid SDMX one and is it the requested format?

Thanks to @stratosn for the idea.

Add support for VTL artefacts

A few maintainable artefacts defined for the VTL will be added to the next version of the SDMX RESTful API.

They should be added to the list of resources supported by the library.

Don't forget to extend the list of resources in api-version!

Update Wiki

Documentation about the new availability API needs to be added to the Wiki

getUrl and schema queries

Since version 2.15.0, it's possible to create schema queries.

Unfortunately, getUrl was forgotten and it does not yet support schema queries, thereby limiting the usefulness of the new feature.

ECB service is no longer working

The preconfigured ECB service is set to use SDMX-JSON. However, the mime type has now changed, following the official release of v1.0.0 of the SDMX-JSON format.

Instead of the generic SDMX-JSON reference, the ECB service should use the specific mime type for the working draft version of SDMX-JSON.

Verify status of services

Some of the supported services may have been upgraded to new versions of the SDMX RESTful API. It would be good to check the status.

Create URLs without default values

The API allows building SDMX RESTful URLs representing queries to be executed against SDMX RESTful web services, using the getUrl function. This function adds the default values defined in the SDMX RESTful API, in case not all query parameters are set.
It would be nice to have an option leaving the choice to the user whether default values should be added to the URL.

ReportingTaxonomy is not recognized as ItemScheme

The file metadata-type offers the possibility to check whether any artefact type (e.g. codelist) is an SDMX item scheme, using the isItemScheme method.

According to the SDMX information model, reporting taxonomies are item schemes but the isItemScheme method states the contrary.

Thanks to @stratosn for reporting this.

Make sure the new features are limited to v1.3.0

This includes the validation and the creation of URLs

Expose checkStatus

The recently introduced request2 function returns the entire HTTP response.

It would be nice to add a helper function allowing a client to check whether the status is one of the expected ones, before processing the response body.

Check out mocha deprecation warning

https://github.com/mochajs/mocha/wiki/compilers-deprecation

Provide http response headers

The library offers the possibility to execute a request.

However, only the response body is returned by the function call.

It should however be possible for a client to get access to the full response, i.e. including headers.

Thanks to @Tzaphkiel for the proposal.

Default options for provided services

It would be nice to have a few default options for provided services such as:

• Preferred data format
• Whether the service supports compression

Fix project setup issues

• Travis build
• Code coverage
• Protect branch

Add support for schema queries

Schema queries are currently not supported in the library. However, they are also part of the SDMX RESTful specification.

Lower complexity of createShortDataQuery

The cyclomatic complexity of createShortDataQuery is too high.

Add support for version 1.4.0 of the SDMX RESTful API

The SDMX-TWG will soon release v1.4.0 of the SDMX RESTful API.

The library should be updated accordingly (i.e. add version, switch reference to latest, etc.)

Non-ItemScheme MD queries not generated correctly

Requestion dataflow metadata for example adds a 4th parameter to the URL that should only be present for ItemSchemes.

var sdmxrest = require('sdmx-rest')
var ws = sdmxrest.getService({url: "http://--someEndPointNotPredefined--/service/"})
var query = {resource: "dataflow", detail: "allstubs"}
console.log("Query: "+ sdmxrest.getUrl(query, ws))

outputs:

http://--someEndPointNotPredefined--/dataflow/all/all/latest/all?detail=allstubs&references=none

Infer the API version of a service

Currently, when a new service is created, its API version will default to the latest one. However, typically, services are based on an older version of the API.

It would be good if the library could infer the API version.

This could be offered either via a dedicated function to be called by the client or transparently via the getService function.

Thanks to @Tzaphkiel for the proposal.

Switch to husky

In the Dev dependencies, ghooks has been deprecated and it is recommended to use husky instead.

Add user-agent

It would be nice to add a user-agent to each request sent

Use arrays of dimension values for the data query key

When creating data queries, a string is currently expected for the key parameter (e.g.: M.NOK+CHF.EUR.SP00.A). However, when clients use select field for dimension filters, it would be convenient to pass an array of values (e.g.: [[M], [NOK, CHF], [EUR], [SP00], [A]). In case there is only one value, strings should be allowed as well ((e.g.: [M, [NOK, CHF], EUR, SP00, A))

Handle space in media types

Some services return the media type with spaces. checkMediaType() implements a string comparison and therefore fail for such cases.

Calling request triggers a TypeError

Executing the following query on a Mac triggers a TypeError

sdmxrest.request("http://sdw-wsrest.ecb.europa.eu/service/data/EXR/A.CHF.EUR.SP00.A")
     .then(function(data) {console.log(data);})
     .catch(function(error) {console.log(error);});

The error is:

url.js:87
    throw new TypeError('Parameter "url" must be a string, not ' + typeof url);
    ^

TypeError: Parameter "url" must be a string, not object
    at Url.parse (url.js:87:11)
    at Object.urlParse [as parse] (url.js:81:5)

Thanks to @stratosn for reporting the issue.

Add support for new SDMX formats

Cf. issue #49 in the sdmx-rest repo.

Add support for new validity API

Cf. issue #40 in the sdmx-rest repo.

SdmxPatterns is undefined

The API offers the possibility to get regular expressions for various SDMX patterns, using sdmxrest.utils.SdmxPatterns. However, this is currently undefined.

Semantic release

Try re-enabling semantic release again

Add support for retrieving multiple items

Cf. issue #39 in the sdmx-rest repo.

Add support for new detail types

Cf. issue #48 in the sdmx-rest repo.

Offer a list of valid resources for an api version

The list of supported resources vary with the version of the SDMX RESTful API. Version 1.3.0 for example adds a few new resources such as actualconstraint and allowedconstraint for example.

It would be nice if the library could offer a function returning the list of valid resources per API version.

Thanks to @Tzaphkiel for the proposal.

Add a Promise polyfill

Some browsers and some older versions of Node.js don't support promises. It would be nice to add a polyfill for these.

Add support for SDMX-JSON Structure format

SDMX-JSON now supports structural metadata.
Therefore, it should be added to the list of valid formats for metadata queries.

Add World Bank to the list of service providers

The World Bank now offers an SDMX RESTful web service (entry point: http://wits.worldbank.org/API/V1/SDMX/V21/rest/) and a dedicated service provider could be added for it.

Lower compexity of createShortMetadataQuery

The cyclomatic complexity of createShortMetadataQuery is too high.

Remove dependency on bithound

The library is depending on bitHound (e.g. for the dependency analysis) but the service has now been closed so an alternative has to be found.

Add support for partial references

Cf. issue #45 in the sdmx-rest repo.

Test ticket from Code Climate

Add support for new content constraints type

Cf. Issue #54 on sdmx-rest repo.

dimensionAtObservation should not have defaults

In case dimensionAtObservation is not set in a data query, the library defaults to TIME_PERIOD.

This is not in line with the SDMX spec, which states: "In case this parameter is not set, the service is expected to: Default to TimeDimension, if the data structure definition has one; If not, default to MeasureDimension, if the data structure definition has one; If none of the above is true, default to AllDimensions."

Therefore, the library should no longer default to TIME_PERIOD when dimensionAtObservation is not set.