bergant / xbrlus Goto Github PK

View Code? Open in Web Editor NEW

21.0 3.0 4.0 26 KB

R interface to XBRL US API

License: Other

R 100.00%

xbrl financial-statements api-client r

xbrlus's Introduction

xbrlus

This package provides an R interface to XBRL US API.

Installation

devtools::install_github("bergant/xbrlus")

Setup

All APIs (except for the CIKLookup) require use of a valid XBRL US API key. You can get the key and read the terms of usage at http://xbrl.us/use/howto/data-analysis-toolkit/.

xbrlus package will read the API key from environment variable XBRLUS_API_KEY. To start R session with the initialized environment variable create a file in your R home with a line like this:

XBRLUS_API_KEY=EnterKeyHere

and name it as .Renviron. To check where your R home is, type normalizePath("~") in your R console.

Usage

Get information about companies and XBRL concepts with xbrlCIKLookup and xbrlBaseElement:

library(xbrlus) 

companies <- xbrlCIKLookup(c(
  "aapl", 
  "goog", 
  "fb"
)) 

elements <- xbrlBaseElement(c(
  "AssetsCurrent",
  "AssetsNoncurrent",
  "Assets",
  "LiabilitiesCurrent",
  "LiabilitiesNoncurrent",
  "Liabilities",
  "StockholdersEquity",
  "MinorityInterest",
  "StockholdersEquityIncludingPortionAttributableToNoncontrollingInterest",
  "LiabilitiesAndStockholdersEquity"
))

Use xbrlValues to get balance sheet values:

values <- xbrlValues( 
  CIK = companies$cik, 
  Element = elements$elementName, 
  DimReqd = FALSE, 
  Period = "Y",
  Year = 2013,
  NoYears = 1,
  Ultimus = TRUE,
  Small = TRUE,
  as_data_frame = TRUE
)

Reshape to wide format and print table:

library(dplyr)
library(tidyr)

balance_sheet <- 
  elements %>% 
  left_join(values, by = "elementName") %>% 
  select(entity, standard.text, amount) %>% 
  mutate(amount = round(amount / 10e6,0)) %>%  
  spread(entity, amount)

balance_sheet <- balance_sheet[
    order(order(elements$elementName)),   
    !is.na(names(balance_sheet))]
row.names(balance_sheet) <- NULL

library(pander)
pandoc.table(
  balance_sheet,
  caption = "Balance Sheet Comparison",
  big.mark = ",",
  split.table = 200,
  style = "rmarkdown",
  justify = c("left", rep("right", 3)))

standard.text	APPLE INC	FACEBOOK INC	Google Inc.
Assets, Current	7,329	1,307	7,289
Assets, Noncurrent	NA	NA	3,803
Assets	20,700	1,790	11,092
Liabilities, Current	4,366	110	1,591
Liabilities, Noncurrent	NA	NA	NA
Liabilities	8,345	242	NA
Stockholders' Equity Attributable to Parent	12,355	1,547	8,731
Stockholders' Equity Attributable to Noncontrolling Interest	NA	NA	NA
Stockholders' Equity, Including Portion Attributable to Noncontrolling Interest	NA	NA	NA
Liabilities and Equity	20,700	1,790	11,092

Table: Balance Sheet Comparison

References

Data Analysis Toolkit and API description on GitHub: https://github.com/xbrlus/data_analysis_toolkit

xbrlus's People

Contributors

Stargazers

Watchers

Forkers

rodmorley ianmadlenya stevebronder deszie

xbrlus's Issues

Error: Get call in xbrlus_get long URI

This my be a non-issue and me just breaking the package. I'm attempting to call xbrlValues with a large amount of elements (951). When I run the function it returns the following error

 values <- xbrlValues(CIK = table_companies[, "CIK"],
                         Element = elements[,"elementName"],
                         DimReqd = FALSE,
                         Ultimus = TRUE,
                         Small = TRUE, 
                         Period = "1Q",
                         Year = 2016,
                         as_data_frame = TRUE)
Space required after the Public Identifier
SystemLiteral " or ' expected
SYSTEM or PUBLIC, the URI is missing
Error: 1: Space required after the Public Identifier
2: SystemLiteral " or ' expected
3: SYSTEM or PUBLIC, the URI is missing

Tracing through the code it appears the error comes from xbrlus_get, specifically in xbrlus_validate(res). It appears that the URI given by httr::GET(xbrlus_url, query = query) brings back something with

<title>414 Request-URI Too Large</title>
</head><body>
<h1>Request-URI Too Large</h1>

So this appears to be an httr problem more than an xbrlus problem. This stackoverflow problem covers the issue and it appears to be a problem with how the php is written.
here.

The only thing I can think of to fix this in xbrlus would be to check whether res status code is 414 and if so, break up the query in multiple queries, run them, then bind them all together at the end.

If you know a workaround so I can get all information from a given companies 10-Q, that would solve my problem and this can be possibly moved over to httr.

xbrlCIKLookup Broken

Hello again! From the tutorial

library(xbrlus)
companies <- xbrlCIKLookup(c(
    "aapl", 
    "goog", 
    "fb"
))
# Error: Returned message is not an xml

Error in xbrlBaseElement: rbind fails due to mismatched columns

The following xbrlBaseElement call returns an error due to their being a balance variable in EarningsPerShareBasicAndDiluted, but not in OtherNonoperatingIncome

xbrlBaseElement(c("OtherNonoperatingIncome","EarningsPerShareBasicAndDiluted" ))
Error in rbind(deparse.level, ...) : 
  numbers of columns of arguments do not match

I'm not sure if this is kosher, but one possible fix is to add a balance variable for the element that does not have one such as the following.

xbrlBaseElement <- function (Element, Namespace = NULL, as_data_frame = TRUE) {
  if (length(Element) > 1) {
    return(do.call(base::rbind, c(lapply(Element, xbrlBaseElement, 
                                   Namespace = Namespace, as_data_frame = as_data_frame), 
                            make.row.names = FALSE)))
  }
  ret <- xbrlus_get("xbrlBaseElement", list(Element = Element, 
                                            Namespace = Namespace))
  if (as_data_frame) {
    ret <- xbrlus_to_data_frame(ret, "baseElement")
   # Add balance if it does not exist
    if (!("balance" %in% colnames(ret))) ret$balance <- NA
  }
  ret
}

Encoding Warning xbrlValues: Default to UTF-8

When running xbrlValues the following warning appears multiple times:

No encoding supplied: defaulting to UTF-8.

This may be due to httr version 1.1.0's content() function (link - line 70) running guesstype() when the type is not specified.

The error shows up with the main example:

companies <- xbrlCIKLookup(c(
  "aapl"
)) 

elements22 <- xbrlBaseElement(c(
  "AssetsCurrent",
  "AssetsNoncurrent",
  "Assets",
  "LiabilitiesCurrent",
  "LiabilitiesNoncurrent",
  "Liabilities",
  "StockholdersEquity",
  "MinorityInterest",
  "StockholdersEquityIncludingPortionAttributableToNoncontrollingInterest",
  "LiabilitiesAndStockholdersEquity"
))

values <- xbrlValues( 
  CIK = companies$cik, 
  Element = elements$elementName, 
  DimReqd = FALSE, 
  Period = "Y",
  Year = 2013,
  NoYears = 1,
  Ultimus = TRUE,
  Small = TRUE,
  as_data_frame = TRUE
)

You may want to change xbrlus_parse to:

xbrlus_parse <- function(res) {
  res_txt <- httr::content(res, as = "text", type = "UTF-8")
  doc <- XML::xmlParse(res_txt, asText = TRUE, options = XML::NOCDATA)
  XML::xmlToList(doc)
}

Or something similar

Session info below

R version 3.2.4 (2016-03-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.4 LTS

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] data.table_1.9.6   magrittr_1.5       tidyr_0.4.1        dplyr_0.4.3        xbrlus_0.1.1.9001 
 [6] rvest_0.3.1        xml2_0.1.2         XBRL_0.99.16       devtools_1.10.0   

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.3     XML_3.98-1.4    digest_0.6.9    assertthat_0.1  chron_2.3-47    R6_2.1.2        DBI_0.3.1      
 [8] httr_1.1.0      stringi_1.0-1   lazyeval_0.1.10 curl_0.9.6      tools_3.2.4     stringr_1.0.0   selectr_0.2-3  
[15] parallel_3.2.4  memoise_1.0.0

Adding XBRLUS enviroment key within R

Just wanted to note that you can use

Sys.setenv(XBRLUS_API_KEY = YOUR_API_KEY)

to create a new environment variable instead of finding home and adding a .Renviron

Question, not an Issue

I am having trouble understanding how to set up the API key. Where do I insert it, how do I reference it, etc? You mention turning it into an .Renviron but then what? What should the file name be? Do we put the API with hyphens? Do we also put the variable and equal sign or just the api key. Please dumb it down and make this user friendly.

CIK to stock ticker

Is there a way to do the inverse of xbrlCIKLookup so that, given a cik, you can look up the ticker?

Warning message: NAs introduced by coercion

Hello again! Hope you are doing well. A bug in parsing or an internal change in XBRL at some point is causing a weird issue in pre-2011 calls. Take for example the company AAR CORP who submitted a 10-Q in the first quarter of 2010 and 2011, respectively. I checked to make sure they submitted for each of these quarters and they submitted on 2010-03-19 and 2011-03-22.

AAR CORP's cik is 0000001750

Here we try to collect the elements given in the toy example for XBRLUS. For 2010 it fails, but for 2011 it works.

company <- "0000001750"

elements <- xbrlBaseElement(c(
  "AssetsCurrent",
  "AssetsNoncurrent",
  "Assets",
  "LiabilitiesCurrent",
  "LiabilitiesNoncurrent",
  "Liabilities",
  "StockholdersEquity",
  "MinorityInterest",
  "StockholdersEquityIncludingPortionAttributableToNoncontrollingInterest",
  "LiabilitiesAndStockholdersEquity"
))


value<- xbrlValues(CIK =  company,#table,
                   Element = elements[,"elementName"],
                   DimReqd = FALSE,
                   Ultimus = TRUE,
                   Small = TRUE, 
                   Period = "1Q",
                   Year = 2010,
                   as_data_frame = TRUE)
Warning message:
NAs introduced by coercion

Now for 2011

value<- xbrlValues(CIK =  company,#table,
                   Element = elements[,"elementName"],
                   DimReqd = FALSE,
                   Ultimus = TRUE,
                   Small = TRUE, 
                   Period = "1Q",
                   Year = 2011,
                   as_data_frame = TRUE)

What is strange is that if you change the years to 2009 and 2012 you get the same effect. Did something change in how XBRL kept data pre-2011?

Session Info

sessionInfo()
R version 3.2.5 (2016-04-14)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.4 LTS

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] xbrlus_0.1.1.9001

loaded via a namespace (and not attached):
[1] httr_1.1.0      R6_2.1.2        rsconnect_0.4.3 tools_3.2.5     curl_0.9.7      XML_3.98-1.4

xml parsing error for xbrlCIKLookup

I'm getting the following when running the example in the readme.md

library(xbrlus)
> companies <- xbrlCIKLookup(c(
+   "aapl", 
+   "goog", 
+   "fb"
+ ))
# Extra content at the end of the document
# Error: 1: Extra content at the end of the document

I'm guessing they switched something up with the doc