mdequeljoe / readsdmx

read sdmx data into R dataframes

readsdmx

Read sdmx data into dataframes from either a local SDMX-ML file or from a SDMX web-service:

u <-
  "https://stats.oecd.org/restsdmx/sdmx.ashx/GetData/HH_DASH/..Q"
d <- readsdmx::read_sdmx(u)

The RapidXML C++ library is used to parse the data.

Install

From CRAN:

install.packages("readsdmx")

From GitHub:

devtools::install_github("mdequeljoe/readsdmx")

Data messages

The follow data message types are supported:

• Compact (version 2.0)
• Generic (2.0 & 2.1)
• Structure Specific Data (2.1)
• Utility (2.0)
• Cross-sectional (2.0)
• Structure Code-lists (2.0 & 2.1)

Benchmark

For the above example (locally):

download.file(u, path <- tempfile(fileext = ".xml"), quiet = TRUE)
microbenchmark::microbenchmark(
  readsdmx = readsdmx::read_sdmx(path),
  rsdmx = as.data.frame(rsdmx::readSDMX(path, FALSE)),
  times = 5L,
  unit = "s"
)

#> Unit: seconds
#>      expr    min     lq   mean median     uq    max neval
#>  readsdmx  0.152  0.153  0.159  0.161  0.165  0.165     5
#>     rsdmx 23.955 24.578 24.732 24.937 25.005 25.185     5

Related

rsdmx

pandasdmx (python)

readsdmx's Issues

CodeLists: take 'parentCode' if present

big queries - quiet = FALSE - fatal error

Hi, the package performance is awesome, this is exactly the tool I was looking for !
However, I have spotted something weird happening with some queries.

For example with this big query :

query = "https://bdm.insee.fr/series/sdmx/data/SERIES_BDM/000810674+000810675+000637404+000637405+000810676+000455734+000810680+000810679+000455733+000852054+000810682+001532276+000810685+000495553+001532275+000810687+000455727+000810688+000810657+000810658+000810689+000810690+000455730+000455746+001565202+000455747+001565201+000455744+001565200+001532272+001659203+001659204+001659205+001659206+001659207+001659208+001659209+001659210+001659211+001659212+000455743+000455745+001565198+001565199+001641575+001641576+001642882+001642883+001642884+001642885+000455728+000455729+000455732+000455735+000455738+000455739+000484333+000484334+000484335+000495554+000495557+000495558+000495559+000810635+000810636+000810637+000810638+000810639+000810640+000810641+000810642+000810643+000810644+000810645+000810646+000810647+000810648+000810649+000810650+000810651+000810652+000810653+000810654+000810655+000810656+000810659+000810660+000810661+000810662+000810663+000810664+000810665+000810666+000810667+000810668+000810669+000810670"

if I run this it is working :
data = readsdmx::read_sdmx(query)

but this triggers a fatal error :
data = readsdmx::read_sdmx(query, quiet = FALSE)

and if I run the query with only one series key either with quiet = TRUE or with quiet = FALSE it is working:
query = "https://bdm.insee.fr/series/sdmx/data/SERIES_BDM/000810674"
data = read_sdmx(query, quiet= TRUE)
data = read_sdmx(query, quiet= FALSE)

Maybe the problem comes from my computer but I cannot find why I have a fatal error.

I am using the following :
R version 3.6.1 (2019-07-05)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Do you have any advice?
I am really interested in having a solution as I might use this package as a dependency.

Best regards,
Hadrien Leclerc

character encoding

output should account for proper character encoding (e.g. République instead of République)

include sdmx-json

allow for the possibility to read sdmx-json data-messages as described at https://github.com/sdmx-twg/sdmx-json