sdmx-twg / sdmx-csv Goto Github PK

View Code? Open in Web Editor NEW

13.0 13.0 6.0 261 KB

This repository is used for maintaining the SDMX-CSV message specifications.

csv format sdmx specification standard

sdmx-csv's People

Contributors

Stargazers

Watchers

Forkers

bengraeler nikornnanta dingshutong shikongo-veijo

sdmx-csv's Issues

SDMX-CSV data without series/data

This is somehow related to #5 but the business case here is outside SDMX REST.

Would it be valid to have SDMX-CSV containing only dataset level data attributes ?

SDMX 3.0: implement "feature 008 Enhance the constraints artefacts" for SDMX-CSV messages

https://metadatatechnology.com/sdmx3/designs/008/baseline/%5BApproved%5D%20SDMX3_08-Enhance%20the%20constraints%20artefacts_FeatureSolution-v2.1.1.docx

Handling of missing values

from @sosna
I could not find the SDMX syntax to be used to identify missing values in the SDMX-CSV documentation.

The appropriate approach for missing values in SDMX-CSV should be clarified.
The solution should also distinguish between “values to be set to 'missing'” and “values not to be changed when appending”.

Related ticket in SDMX-XML: sdmx-twg/sdmx-ml#32
Related Ticket in SDMX-JSON: sdmx-twg/sdmx-json#122

SDMX 3.0: implement "feature 028 Simplify DSD dimensions" for SDMX-CSV messages

https://metadatatechnology.com/sdmx3/designs/028/approved/Simplify%20DSD%20Dimension%20definition%20v0.0.2.docx

Why dataflow only?

The SDMX-CSV spec states: "For the first column, the dataflow column, always is the term DATAFLOW".

Typically, SDMX data messages require a reference to structural metadata but this reference can be a data structure, a dataflow or a provision agreement. Considering this, is there a reason why SDMX-CSV accepts one of these 3 types only (i.e. dataflow)?

Also, the minimum required to parse an SDMX data message is a data structure. So forcing a dataflow might seem to push the bar a bit too high, especially in cases where dataflows are not available.

Last, CSV messages have no header and therefore no way to pass the provider ID (unless I’m mistaken). At least, if you could reference a provision agreement instead of a dataflow, there would be a work-around.

A solution would be to:

Rename the 1st column Structure
Specify the artefact type (e.g. Dataflow=BIS:CBS(1.0) instead of BIS:CBS(1.0))

What do you think?

SDMX 3.0: implement "feature 006 Discriminated union of codelists" for SDMX-CSV messages

https://metadatatechnology.com/sdmx3/designs/006/approved/SDMX3%20Discriminated%20union%20of%20codelists%20v29062020.docx

Improve the narrative structure of the SDMX-CSV data specification

The SDMX-CSV data specification should have an overview like this:

Column Name	Description	Order
STRUCTURE	Data structure type (dataflow, datastructure, dataprovision)	1
STRUCTURE_ID	Artefact identification information	2
STRUCTURE_NAME	Localized name of the artefact	3 (Optional)
ACTION	Action code (I - Information, A - Append, R - Replace, D - Delete)	4 (Optional)
SERIES_KEY	Series key for the observation	5 (Optional)
OBS_KEY	Observation key	6 (Optional)
Remaining Columns	Component IDs or localized names
Special cases:
EXAMPLE: text	Extra column with the localized name for the `EXAMPLE` component
EXAMPLE[]	Multi-valued
EXAMPLE[de;fr]	Multi-lingual
PARENT.CHILD	Nested metadata attribute
PARENT[].CHILD	Nested metadata attribute of multi-valued parent attribute

And then discuss them in detail, based on the type of column. The current document is very difficult to follow as it is at the moment.

(In doubt, please handle this as a public review comment on SDMX 3.1 once the comment period begins.)

SDMX-CSV data without observations/data

Dear all,

In SDMX REST data queries it is possible include the query parameter detail=serieskeysonly or detail=nodata.
If SDMX-CSV is requested with detail=serieskeysonly or detail=nodata then what should be the output ?
Could an SDMX-CSV data file exist without any observations ?

Thanks in advance

SDMX 3.0: implement "feature 005 Codelist extension / composition" for SDMX-JSON messages

https://metadatatechnology.com/sdmx3/designs/005/approved/005%20Code%20List%20Extension%20V1.0.0.docx

Escaping the character used as separator between dimension values in a key

The field guide states the following about keys (the highlight is mine):

The column will contain the combination of IDs/values for all the dimensions, order by their order in the data structure definition and separated by a dot character (.), e.g. M.USD.EUR.SP00.2020-01

However, the guide does not specify what to do in case a dimension value contains a dot, i.e. the character to be used as separator. This cannot be the case in case of coded dimensions (as the . is not an allowed character for an SDMX code), but could be the case if:

The dimension is uncoded;
The dimension takes its values from a Valuelist, instead of a Codelist.

So, in case a dimension value contains a dot, what should the service provider do, when building the series and/or observation keys if SDMX-CSV data messages?

Thank you.

Make header mandatory
Make use of IDs mandatory. An allowed variant could be ID+Name to respect the main use case of dissemination
Fix the separator to comma