walkerke / tidycensus

Load US Census boundary and attribute data as 'tidyverse' and 'sf'-ready data frames in R

tidycensus

tidycensus is an R package that allows users to interface with the US Census Bureau's decennial Census and five-year American Community APIs and return tidyverse-ready data frames, optionally with simple feature geometry included. Install from CRAN with the following command:

install.packages("tidycensus")

For the latest updates, install from GitHub:

remotes::install_github("walkerke/tidycensus")

To learn more about the package, please visit the package documentation at https://walker-data.com/tidycensus/ or read the book Analyzing US Census Data: Methods, Maps, and Models in R at https://walker-data.com/census-r/.

Note: This product uses the Census Bureau Data API but is not endorsed or certified by the Census Bureau.

tidycensus's Issues

ACS5 fails to download for some years(2011 and 2012) but not others.

I get an error running this code

state_income <- get_acs(geography = 'tract',
                          table = c("B19301"),
                          state = 'AZ',
                          year = 2012,
                          survey = 'acs5',
                          geometry = TRUE,
                          cache_table = TRUE)

The error

'''
Please note: get_acs() now defaults to a year or endyear of 2016.
Downloading feature geometry from the Census website. To cache shapefiles for use in future sessions, set options(tigris_use_cache = TRUE).
Using FIPS code '04' for state 'AZ'
trying URL 'http://www2.census.gov/geo/tiger/GENZ2012/cb_2012_04_tract_500k.zip'
Error in download.file(url, tiger_file, mode = "wb") :
cannot open URL 'http://www2.census.gov/geo/tiger/GENZ2012/cb_2012_04_tract_500k.zip'
In addition: Warning message:
In download.file(url, tiger_file, mode = "wb") :
cannot open URL 'http://www2.census.gov/geo/tiger/GENZ2012/cb_2012_04_tract_500k.zip': HTTP status was '404 Not Found'
'''

It doesn't appear to be an internet connection issue, my internet is working fine otherwise and I can download the 2010,2013,2014,2015 data no problem. I can also download the 2012 data through the ACS package with no problems.

Getting population estimates with get_decennial function

Is it possible to get the yearly population estimates using the get_decennial function? It currently limits the year to the actual census year (1990, 2000, 2010)

Link to "popest" dataset: https://www.census.gov/data/developers/data-sets/popest-popproj/popest.Vintage_2015.html

The R package "censusapi" can grab this info with the getCensus function:

censusapi::getCensus(
    name = "pep/population", 
    vintage = 2015, 
    region = "state", 
    vars = c("POP", "GEONAME", "SUMLEV")
)

cannot coerce type 'closure' to vector of type 'character' ... Round 2

I saw a similar issue as a closed issue. I commented as a follow up to the closed issue but I thought it might be helpful to post this as a new issue as well. My apologies if that's too much.

I am experiencing an issue when submitting a census call. I'm getting the following error: Client error Bad Request Client error: (400) Bad Request. I've posted my code below.

Here is what I am running

library(tidycensus)
library(tidyverse)
census_api_key("xxxxx")
age <- get_acs(table = "B01001", state = "SD", county = "Custer", geography = "county") 

And here is what I get back

> age <- get_acs(table = "B01001", state = "SD", county = "Custer", geography = "county")
Using FIPS code '46' for state 'SD'
Using FIPS code '033' for 'Custer County'
[1] "Client error Bad Request Client error: (400) Bad Request"
Error in as.character(x) : 
  cannot coerce type 'closure' to vector of type 'character'

I was able to get
df2 <- get_acs(geography = "county", table = "B01001", survey = "acs5", year = 2015, state = "SD")
to run without issue. But that call returns all counties in the state.

The following which calls for all tracks within a county also seems to work
df2 <- get_acs(geography = "tract", table = "B01001", survey = "acs5", year = 2015, state = "SD")
It seems like the error occurs when trying to pull in data for a single county within a state. Is this a feature or a bug?

can get_acs() pull in 5 year data from 2016?

It seems like the get_acs() function can pull in the "acs5" survey from 2015, but not the most recent 2016 survey. Is this feature going to be available in a future release of tidycensus?

This doesn't seem to work:
state_pop <- get_acs(geography = "state", variables = "B01003_001", survey = "acs5", year = 2016)

But the data for the 1-year survey is available:
state_pop <- get_acs(geography = "state", variables = "B01003_001", survey = "acs1", year = 2016)

Feature Request: Download by Table

I would like to request specifying a table (e.g. "B25036") and getting all the variables associated with that table, instead of having to specify each variable.

At the moment I'm building them with something like:

library(stringr)
housing_vars = paste("B25036", str_pad(1:21, 3, pad = "0"), sep = "_")
x = get_acs("state", housing_vars, ...)

Cannot coerce type 'closure' to vector of type 'character'

Thanks for creating this awesome package. I am using it for the first time and just copy/pasting some code to see how it works and I get an error. Can you help me troubleshoot?

Here is what I pasted in RStudio:

`library(tidycensus)
library(tidyverse)
options(tigris_use_cache = TRUE)

orange <- get_acs(state = "CA", county = "Orange", geography = "tract",
variables = "B19013_001", geometry = TRUE)

head(orange)`

Here is what RStudio gave me in return:

`> orange <- get_acs(state = "CA", county = "Orange", geography = "tract", variables = "B19013_001", geometry = TRUE)
Using FIPS code '06' for state 'CA'
Using FIPS code '059' for 'Orange County'
[1] "Server error Internal Server Error Server error: (500) Internal Server Error"
Error in as.character(x) :
cannot coerce type 'closure' to vector of type 'character'

`
Thanks,
Monica

load_variables error

It seems like my error is unque since I can find no references or it occurring with others. (That's a bad sign for me.)

Using your example: v15 <- load_variables(2015, "acs5", cache = FALSE)

gives this error: Error in bind_rows_(x, .id) : incompatible sizes (1 != 0)

Any ideas?

package or namespace load failed for ‘tidycensus’

I'm getting the following error when I try to load tidycensus.

library(tidyverse)
library(tidycensus)
Error : object ‘modify_depth’ is not exported by 'namespace:purrr'
Error: package or namespace load failed for ‘tidycensus’

Here is my session info:

sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: macOS Sierra 10.12.5

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] dplyr_0.7.2        purrr_0.2.2.2      readr_1.0.0        tidyr_0.6.3       
[5] tibble_1.3.3       ggplot2_2.2.1.9000 tidyverse_1.1.1   

loaded via a namespace (and not attached):
 [1] reshape2_1.4.2      sf_0.5-4            haven_1.0.0         lattice_0.20-34    
 [5] colorspace_1.3-1    htmltools_0.3.6     yaml_2.1.14         rlang_0.1.1        
 [9] foreign_0.8-67      glue_1.1.1          DBI_0.7             rappdirs_0.3.1     
[13] sp_1.2-5            uuid_0.1-2          modelr_0.1.0        readxl_0.1.1       
[17] bindrcpp_0.2        bindr_0.1           plyr_1.8.4          stringr_1.2.0      
[21] rgeos_0.3-22        munsell_0.4.3       gtable_0.2.0        rvest_0.3.2        
[25] tigris_0.5.3        psych_1.6.12        evaluate_0.10       knitr_1.16         
[29] forcats_0.2.0       maptools_0.8-41     parallel_3.3.2      broom_0.4.1        
[33] Rcpp_0.12.12        udunits2_0.13       scales_0.4.1        backports_1.0.4    
[37] jsonlite_1.5        mnormt_1.5-5        hms_0.3             digest_0.6.12      
[41] stringi_1.1.5       grid_3.3.2          rprojroot_1.1       rgdal_1.2-8        
[45] tools_3.3.2         magrittr_1.5        lazyeval_0.2.0.9000 pkgconfig_2.0.1    
[49] xml2_1.0.0          lubridate_1.6.0     assertthat_0.1      rmarkdown_1.6      
[53] httr_1.2.1          R6_2.2.0            units_0.4-6         nlme_3.1-128

Avoid downloading blocks shapefile multiple times in same function call.

When using get_decennial to download block geometries and requesting multiple counties, tidycensus is (a) downloading the much larger state-level blocks shapefile rather than using the county-level shapefiles, and (b) downloading it once for each county.

For example, to grab all blocks for New York City:

nyc = get_decennial("tract", "P0010001", state = "NY", county = c("Bronx", "Kings", "New York", "Queens", "Richmond"), geometry = TRUE, output = "wide")

This will download the New York state blocks shapefile tl_2010_36_tabblock10.zip which is 183MB, and it will download it five times for a total download of ~915MB. It would be better to download the five county shapefiles, tl_2010_36005_tabblock10.zip, tl_2010_36047_tabblock10.zip, etc., for a total download of ~4.1MB.

ACS 1-year survery: error when downloading 2010 or 2011

I received the following error when trying to download 2010 and 2011 data using the ACS 1-year survey. I was able to download 2010-2016 with no problems using the ACS 5-year survey.

Error in get_decennial() when geography is tract

and maybe other levels: 'county' does work

library(tidycensus)
tables <- c("P0030002","P0030003","P0030004","P0030005","P0030006","P0030007","P0030008")
us_tract_pop_2010<-get_decennial(geography = "tract", variables = tables, year = 2010,
                    summary_var = "P0030001",  geometry = FALSE )

Error : lexical error: invalid char in json text.
                                       <html><head><title>Apache Tomca
                     (right here) ------^

Error in overscope_eval_next(overscope, expr) : object 'GEOID' not found

sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

locale:
[1] LC_COLLATE=English_Canada.1252  LC_CTYPE=English_Canada.1252    LC_MONETARY=English_Canada.1252
[4] LC_NUMERIC=C                    LC_TIME=English_Canada.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] tidycensus_0.1.4

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.11     knitr_1.16       bindr_0.1        xml2_1.1.1       magrittr_1.5     hms_0.3          maptools_0.9-2  
 [8] tigris_0.5.3     units_0.4-5      rappdirs_0.3.1   uuid_0.1-2       rvest_0.3.2      lattice_0.20-34  R6_2.2.2        
[15] rlang_0.1.1      udunits2_0.13    stringr_1.2.0    httr_1.2.1       dplyr_0.7.1      tools_3.3.2      rgdal_1.2-8     
[22] grid_3.3.2       DBI_0.7          rgeos_0.3-23     assertthat_0.2.0 tibble_1.3.3     sf_0.5-2         bindrcpp_0.2    
[29] readr_1.1.1      purrr_0.2.2.2    tidyr_0.6.3      curl_2.7         glue_1.1.1       sp_1.2-5         stringi_1.1.5   
[36] jsonlite_1.5     foreign_0.8-69   pkgconfig_2.0.1 

Download one-year estimates

Hi Kyle, I am wondering if tidycensus could be used to download one-year estimates from Census. That would be really helpful. Thank you! - Elmer

get_acs() fails when geometry = TRUE and dplyr 0.6 installed

@walkerke I'm getting the following error when pulling some census data with geometry = TRUE:

Error in select.sf(geom, GEOID, geometry) :
requires dplyr > 0.5.0: install that first, then reinstall sf

It's weird b/c I've got dplyr 0.6 installed

Here's a reproducible example:

census_api_key(api_key)

codes <- c('B15003_002', sprintf('B15003_0%s', 17:25))

cook_acs <- get_acs(geography ="tract", 
                   state = "IL", 
                   county = "Cook", 
                   variables = codes, 
                   summary_var = 'B15003_001', 
                   geometry = TRUE,
                   key = api_key)

Here's the sessionInfo:

R version 3.3.3 (2017-03-06)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: macOS Sierra 10.12.4

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] bindrcpp_0.1       sf_0.4-3           dplyr_0.6.0        ggplot2_2.2.1.9000
[5] tidycensus_0.1.0  

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.11     bindr_0.1        plyr_1.8.4       git2r_0.18.0     tools_3.3.3     
 [6] digest_0.6.12    packrat_0.4.8-1  uuid_0.1-2       jsonlite_1.4     memoise_1.0.0   
[11] tibble_1.3.1     gtable_0.2.0     lattice_0.20-34  pkgconfig_2.0.1  rlang_0.1.1     
[16] DBI_0.6-1        curl_2.3         rgdal_1.2-7      withr_1.0.2      httr_1.2.1      
[21] stringr_1.2.0    xml2_1.1.1       knitr_1.15.1     rgeos_0.3-23     rappdirs_0.3.1  
[26] devtools_1.12.0  tigris_0.5.1     hms_0.3          grid_3.3.3       glue_1.0.0.9000 
[31] R6_2.2.1         foreign_0.8-67   sp_1.2-4         tidyr_0.6.3      purrr_0.2.2.2   
[36] readr_1.1.1      udunits2_0.13    magrittr_1.5     scales_0.4.1     units_0.4-4     
[41] maptools_0.9-2   assertthat_0.2.0 rvest_0.3.2      colorspace_1.3-2 stringi_1.1.5   
[46] lazyeval_0.2.0   munsell_0.4.3   

Class of get_acs depends on value of geometry object

get_acs() appears to return a tibble iff geometry = TRUE:

nc <- get_acs(geography = "county", state = "NC", variables = "B19013_001")
class(nc)
[1] "tbl_df"     "tbl"        "data.frame"

nc_geom <- get_acs(geography = "tract", state = "NC", variables = "B19013_001", geometry = TRUE)
class(nc_geom)
[1] "sf"         "data.frame"

Should the second call above also return a tibble?

Conver to custom variable names

One of the great things that the censusr package does (https://github.com/transportfoundry/censusr) is to allow you to customize the names returned by the API. B19013_001 becomes "hh_size_1" (or whatever it is). In addition to having the variables argument, you also have a names argument that accepts a vector (same length as variables) of custom names.

Further bug related to 2016 5-year data

Thanks for fixing the handling of acs5 vs acs/acs5 (c511445) so quickly! I found a related bug but also a workaround. When I call get_acs(geography = "state", table = "B01001", survey = "acs5", year = 2016), I get the error: Error: lexical error: invalid char in json text. <html><head><title>Error report (right here) ------^

Calling get_acs(geography = "state", table = "B01001", survey = "acs/acs5", year = 2016) works without error.

I know the fix was to replace the survey name with acs/acs5 in the function load_data, but it seems like that fix isn't carrying through to get_acs.

Thanks again for working on this the same day the data came out!

Economic Census and Population Estimates integration

The next step in tidycensus's development will be to add selected datasets from the US Census APIs that are commonly requested. For version 0.5, this includes:

• The Economic Census API: https://www.census.gov/data/developers/data-sets/economic-census.html
• The Population Estimates APIs: https://www.census.gov/data/developers/data-sets/popest-popproj/popest.html

You can track progress on these topics in https://github.com/walkerke/tidycensus/tree/version05. Please note that this branch is likely to be unstable and is not recommended for general use.

More closure issues

For some reason when I try this command, I get the result after:
acstab <- get_acs(geography = "us", variables = c("B00001_001","B00001_002"), year = 2016, output = "wide", survey = "acs5")
[1] "Client error Bad Request Client error: (400) Bad Request"
Error in as.character(x) :
cannot coerce type 'closure' to vector of type 'character'

But this works:
acstab <- get_acs(geography = "us", variables = c("B01001_001","B01001_002"), year = 2016, output = "wide", survey = "acs5")

Both tables are listed in load_variables for 2016, so I'm not sure what is happening

get_acs() error when output = "tidy"

Hi Kyle,

I'm having trouble using "tidy" output from get_acs with version 0.4. I've tried a few different tables, variables, years, etc. and keep getting a similar error when I use tidy output. When I use wide output, everything is working as expected. The error persists with both the CRAN and GitHub versions of tidycensus.

See below for reprex:

library(tidycensus)

# tidy output
get_acs(
  state = "NY",
  geography = "county",
  variables = "B01003_001",
  survey = "acs5",
  year = 2016,
  geometry = FALSE,
  output = "tidy"
)
#> Please note: `get_acs()` now defaults to a year or endyear of 2016.
#> Error: Duplicate identifiers for rows (1, 63), (2, 64), (3, 65), (4, 66), (5, 67), (6, 68), (7, 69), (8, 70), (9, 71), (10, 72), (11, 73), (12, 74), (13, 75), (14, 76), (15, 77), (16, 78), (17, 79), (18, 80), (19, 81), (20, 82), (21, 83), (22, 84), (23, 85), (24, 86), (25, 87), (26, 88), (27, 89), (28, 90), (29, 91), (30, 92), (31, 93), (32, 94), (33, 95), (34, 96), (35, 97), (36, 98), (37, 99), (38, 100), (39, 101), (40, 102), (41, 103), (42, 104), (43, 105), (44, 106), (45, 107), (46, 108), (47, 109), (48, 110), (49, 111), (50, 112), (51, 113), (52, 114), (53, 115), (54, 116), (55, 117), (56, 118), (57, 119), (58, 120), (59, 121), (60, 122), (61, 123), (62, 124)

# wide output
get_acs(
  state = "NY",
  geography = "county",
  variables = "B01003_001",
  survey = "acs5",
  year = 2016,
  geometry = FALSE,
  output = "wide"
)
#> Please note: `get_acs()` now defaults to a year or endyear of 2016.
#> # A tibble: 62 x 4
#>    GEOID NAME                         B01003_001E B01003_001M
#>    <chr> <chr>                              <dbl>       <dbl>
#>  1 36001 Albany County, New York           307891          NA
#>  2 36003 Allegany County, New York          47700          NA
#>  3 36005 Bronx County, New York           1436785          NA
#>  4 36007 Broome County, New York           197381          NA
#>  5 36009 Cattaraugus County, New York       78506          NA
#>  6 36011 Cayuga County, New York            78783          NA
#>  7 36013 Chautauqua County, New York       131748          NA
#>  8 36015 Chemung County, New York           87742          NA
#>  9 36017 Chenango County, New York          49286          NA
#> 10 36019 Clinton County, New York           81505          NA
#> # ... with 52 more rows

Created on 2018-01-08 by the reprex package (v0.1.1.9000).

Session info ------------------------------------------------------------------------------------------
 setting  value                       
 version  R version 3.4.3 (2017-11-30)
 system   x86_64, darwin15.6.0        
 ui       RStudio (1.1.402)           
 language (EN)                        
 collate  en_US.UTF-8                 
 tz       America/New_York            
 date     2018-01-08                  

Packages ----------------------------------------------------------------------------------------------
 package     * version    date       source                               
 assertthat    0.2.0      2017-04-11 CRAN (R 3.4.0)                       
 backports     1.1.2      2017-12-13 CRAN (R 3.4.3)                       
 base        * 3.4.3      2017-12-07 local                                
 bindr         0.1        2016-11-13 cran (@0.1)                          
 bindrcpp    * 0.2        2017-06-17 CRAN (R 3.4.0)                       
 callr         1.0.0.9000 2018-01-07 Github (r-lib/callr@550fa6b)         
 class         7.3-14     2015-08-30 CRAN (R 3.4.3)                       
 classInt      0.1-24     2017-04-16 CRAN (R 3.4.0)                       
 cli           1.0.0      2017-11-05 CRAN (R 3.4.2)                       
 clipr         0.4.0      2017-11-03 CRAN (R 3.4.2)                       
 compiler      3.4.3      2017-12-07 local                                
 crayon        1.3.4      2017-11-15 Github (r-lib/crayon@b5221ab)        
 curl          3.1        2017-12-24 local                                
 datasets    * 3.4.3      2017-12-07 local                                
 DBI           0.7        2017-06-18 CRAN (R 3.4.0)                       
 debugme       1.1.0      2017-10-22 CRAN (R 3.4.2)                       
 devtools      1.13.4     2017-11-09 CRAN (R 3.4.2)                       
 digest        0.6.13     2017-12-14 CRAN (R 3.4.3)                       
 dplyr         0.7.4.9000 2018-01-08 Github (tidyverse/dplyr@e4b34cd)     
 e1071         1.6-8      2017-02-02 CRAN (R 3.4.0)                       
 evaluate      0.10.1     2017-06-24 CRAN (R 3.4.0)                       
 foreign       0.8-69     2017-06-22 CRAN (R 3.4.3)                       
 git2r         0.21.0     2018-01-04 CRAN (R 3.4.3)                       
 glue          1.2.0      2017-10-29 CRAN (R 3.4.2)                       
 graphics    * 3.4.3      2017-12-07 local                                
 grDevices   * 3.4.3      2017-12-07 local                                
 grid          3.4.3      2017-12-07 local                                
 hms           0.4.0      2017-11-23 CRAN (R 3.4.3)                       
 htmltools     0.3.6      2017-04-28 CRAN (R 3.4.0)                       
 htmlwidgets   0.9        2017-07-10 CRAN (R 3.4.0)                       
 httr          1.3.1      2017-12-24 Github (hadley/httr@6b2dadc)         
 jsonlite      1.5        2017-06-01 CRAN (R 3.4.0)                       
 knitr         1.18       2017-12-27 CRAN (R 3.4.3)                       
 lattice       0.20-35    2017-03-25 CRAN (R 3.4.3)                       
 magrittr      1.5        2014-11-22 CRAN (R 3.4.0)                       
 maptools      0.9-2      2017-03-25 CRAN (R 3.4.0)                       
 memoise       1.1.0      2017-04-21 CRAN (R 3.4.0)                       
 methods     * 3.4.3      2017-12-07 local                                
 pillar        1.0.1      2017-11-27 CRAN (R 3.4.3)                       
 pkgconfig     2.0.1      2017-03-21 cran (@2.0.1)                        
 processx      2.0.0.1    2017-07-30 CRAN (R 3.4.1)                       
 purrr         0.2.4.9000 2017-12-09 Github (tidyverse/purrr@62b135a)     
 R6            2.2.2      2017-06-17 CRAN (R 3.4.0)                       
 rappdirs      0.3.1      2016-03-28 CRAN (R 3.4.0)                       
 Rcpp          0.12.14    2017-11-23 CRAN (R 3.4.3)                       
 readr         1.1.1.9000 2017-12-24 Github (tidyverse/readr@5415cc1)     
 reprex        0.1.1.9000 2018-01-07 Github (tidyverse/reprex@6453eff)    
 rgdal         1.2-16     2017-11-21 CRAN (R 3.4.2)                       
 rlang         0.1.6.9002 2018-01-08 Github (tidyverse/rlang@f6e234f)     
 rmarkdown     1.8        2017-11-17 CRAN (R 3.4.2)                       
 rprojroot     1.3-2      2018-01-03 CRAN (R 3.4.3)                       
 rstudioapi    0.7        2017-09-07 CRAN (R 3.4.1)                       
 rvest         0.3.2.9000 2017-12-24 Github (hadley/rvest@9a51a5d)        
 sf            0.6-0      2018-01-06 CRAN (R 3.4.3)                       
 sp            1.2-6      2018-01-05 CRAN (R 3.4.3)                       
 stats       * 3.4.3      2017-12-07 local                                
 stringi       1.1.6      2017-11-17 CRAN (R 3.4.2)                       
 stringr       1.2.0.9000 2018-01-08 Github (tidyverse/stringr@c8bbc0d)   
 tibble        1.4.1.9000 2018-01-06 Github (tidyverse/tibble@f965ac0)    
 tidycensus  * 0.4        2018-01-08 Github (walkerke/tidycensus@4ab9006) 
 tidyr         0.7.2.9000 2018-01-08 Github (tidyverse/tidyr@7e894a3)     
 tidyselect    0.2.3.9000 2018-01-08 Github (tidyverse/tidyselect@d76ca3a)
 tigris        0.6.2      2018-01-04 CRAN (R 3.4.3)                       
 tools         3.4.3      2017-12-07 local                                
 udunits2      0.13       2016-11-17 CRAN (R 3.4.0)                       
 units         0.5-0      2018-01-07 CRAN (R 3.4.3)                       
 utf8          1.1.3      2018-01-03 CRAN (R 3.4.3)                       
 utils       * 3.4.3      2017-12-07 local                                
 uuid          0.1-2      2015-07-28 CRAN (R 3.4.0)                       
 whisker       0.3-2      2013-04-28 CRAN (R 3.4.0)                       
 withr         2.1.1.9000 2017-12-24 Github (jimhester/withr@df18523)     
 xml2          1.1.1      2017-01-24 CRAN (R 3.4.0)                       
 yaml          2.1.16     2017-12-12 CRAN (R 3.4.3)   

tidycensus doesn't install on R 3.2

tidycensus doesn't install on R 3.2. is that intentional? what would be the latest version it will agree with?

Better error handling

The package should get better at handling errors when making invalid HTTP requests - ideally to tell users what they did wrong.

Error in 2000 get_decennial() when geography is tract

st <- c(datasets::state.abb, "DC")
population_2000 <- get_decennial(geography = "tract", endyear = 2000, variables = "P001001", state = st)

Results in:

`Using FIPS code '01' for state 'AL'
Error : lexical error: invalid char in json text.
<title>Apache Tomca
(right here) ------^

Error in gather.default(., key = variable, value = value, -GEOID, -NAME) :
object 'NAME' not found`

When I try it with 2010 data, it works (note, the variable is different):
population_2010 <- get_decennial(geography = "tract", endyear = 2010, variables = "P0010001", state = st)

support for ACS data profiles

currently tidycensus only supports access to summary files from the ACS. It would be great to also include variables from the data profile

Kusilvak Census Area (Alaska) Disappears When Using shift_geo = T

When pulling data and geo files and using the shift_geo = T feature, the Kusilvak Census Area, Alaska disappears (fips code = 02158).

Example:

get_acs(geography = "county", 
                  variables = "S1701_C02_001", geometry = T,
                  shift_geo = T)

Zip code data?

This must be easy but I can't seem to extract zip code data. Is the geography part of get_acs() need to be a specific zip code qualifier?

Received an error - invalid char in json text.

I used R 3.4.0 version, and most recent packages relevant in Windows 64-bit system. When I ran the code below, I received the error message:

Using FIPS code '01' for state 'AL'
No encoding supplied: defaulting to UTF-8.
Error during wrapup: lexical error: invalid char in json text.
<titl
(right here) ------^

The code I ran:
library(tidycensus)
library(purrr)
census_api_key("my key")
us <- unique(fips_codes$state)[1:51]
totalpop <- map_df(us, function(x) {
get_acs(geography = "tract", variables = "B01003_001",
state = x)
})

Thanks,
Ann

Error in bind_rows_(x, .id) : Argument 6 must be length 1, not 0

Command:
v15 <- load_variables(2015, "acs5", cache = FALSE)

as per the instructions on :
https://walkerke.github.io/tidycensus/articles/basic-usage.html#searching-for-variables

Output:
Error in bind_rows_(x, .id) : Argument 6 must be length 1, not 0

Thoughts?

Support the ACS Subject Tables

Add support for the ACS Subject Tables.

NAs returned for tract GEOID

First - this package has saved me tons of time/stress - thank you!

Weird issue here. I am running the current CRAN edition 0.4. When I call get_acs() for tract geography, I see NAs in the GEOID column despite seeing what appear to be correct values for the tract NAME, estimate, and moe. This is a new issue for me after updating to 0.4.

This does not occur for all counties. In a recent pull of tracts for counties in some major urban areas, I had 100% missing GEOIDs for counties in AL, AZ, AR, CA, CO, CT, FL, GA, IL, IN, IA, and nothing missing for any other counties in my state sample. [I did not check every state; summary table to follow.]

`

library(tidycensus)
devtools::session_info()
Session info -------------------------------------------------------------------------------
setting value
version R version 3.4.2 (2017-09-28)
system x86_64, linux-gnu
ui RStudio (1.1.383)
language (EN)
collate en_US.UTF-8
tz Zulu
date 2018-01-20
`

Packages
package * version date source
assertthat 0.2.0 2017-04-11 CRAN (R 3.4.2)
base * 3.4.2 2017-09-29 local
bindr 0.1 2016-11-13 CRAN (R 3.4.2)
bindrcpp 0.2 2017-06-17 CRAN (R 3.4.2)
compiler 3.4.2 2017-09-29 local
datasets * 3.4.2 2017-09-29 local
DBI 0.7 2017-06-18 CRAN (R 3.4.2)
devtools 1.13.3 2017-08-02 CRAN (R 3.4.2)
digest 0.6.12 2017-01-27 CRAN (R 3.4.2)
dplyr 0.7.4 2017-09-28 CRAN (R 3.4.2)
foreign 0.8-69 2017-06-21 CRAN (R 3.4.0)
glue 1.1.1 2017-06-21 CRAN (R 3.4.2)
graphics * 3.4.2 2017-09-29 local
grDevices * 3.4.2 2017-09-29 local
grid 3.4.2 2017-09-29 local
hms 0.3 2016-11-22 CRAN (R 3.4.2)
httr 1.3.1 2017-08-20 CRAN (R 3.4.2)
jsonlite 1.5 2017-06-01 CRAN (R 3.4.2)
lattice 0.20-35 2017-03-25 CRAN (R 3.3.3)
magrittr 1.5 2014-11-22 CRAN (R 3.4.2)
maptools 0.9-2 2017-03-25 CRAN (R 3.4.2)
memoise 1.1.0 2017-04-21 CRAN (R 3.4.2)
methods * 3.4.2 2017-09-29 local
pkgconfig 2.0.1 2017-03-21 CRAN (R 3.4.2)
purrr 0.2.4 2017-10-18 CRAN (R 3.4.2)
R6 2.2.2 2017-06-17 CRAN (R 3.4.2)
rappdirs 0.3.1 2016-03-28 CRAN (R 3.4.2)
Rcpp 0.12.13 2017-09-28 CRAN (R 3.4.2)
readr 1.1.1 2017-05-16 CRAN (R 3.4.2)
rgdal 1.2-13 2017-10-07 CRAN (R 3.4.2)
rgeos 0.3-25 2017-09-25 CRAN (R 3.4.2)
rlang 0.1.2 2017-08-09 CRAN (R 3.4.2)
rvest 0.3.2 2016-06-17 CRAN (R 3.4.2)
sf 0.5-4 2017-08-28 CRAN (R 3.4.2)
sp 1.2-5 2017-06-29 CRAN (R 3.4.2)
stats * 3.4.2 2017-09-29 local
stringi 1.1.5 2017-04-07 CRAN (R 3.4.2)
stringr 1.2.0 2017-02-18 CRAN (R 3.4.2)
tibble 1.3.4 2017-08-22 CRAN (R 3.4.2)
tidycensus * 0.4 2018-01-02 CRAN (R 3.4.2)
tidyr 0.7.2 2017-10-16 CRAN (R 3.4.2)
tigris 0.5.3 2017-05-26 CRAN (R 3.4.2)
tools 3.4.2 2017-09-29 local
udunits2 0.13 2016-11-17 CRAN (R 3.4.2)
units 0.4-6 2017-08-27 CRAN (R 3.4.2)
utils * 3.4.2 2017-09-29 local
uuid 0.1-2 2015-07-28 CRAN (R 3.4.2)
withr 2.0.0 2017-07-28 CRAN (R 3.4.2)
xml2 1.1.1 2017-01-24 CRAN (R 3.4.2)
yaml 2.1.14 2016-11-12 CRAN (R 3.4.2)

`

na_check <- get_acs(geography = "tract",
state = "01",
county = "001",
table = "B01001",
geometry = FALSE,
year = 2015,
survey = "acs5",
cache_table = TRUE,
keep_geo_vars = FALSE,
key = "foo")
Please note: "get_acs()" now defaults to a year or endyear of 2016.
Loading ACS5 variables for 2015 from table B01001 and caching the dataset for faster future access.
sum(is.na(na_check$GEOID))/nrow(na_check)
[1] 1
`

Above I show this result when calling the state/county by FIPS code; I get the same result in GEOID NAs when calling by the text names of state/county. What does look different when I call get_acs() using text names -- the normal message on FIPS translations is repeated 3 times instead of appearing once.

`

na_check <- get_acs(geography = "tract",
state = "AL",
county = "Autauga County",
table = "B01001",
geometry = FALSE,
year = 2014,
survey = "acs5",
cache_table = TRUE,
keep_geo_vars = FALSE,
key = "foo")
Please note: "get_acs()" now defaults to a year or endyear of 2016.
Loading ACS5 variables for 2014 from table B01001 and caching the dataset for faster future access.
Using FIPS code '01' for state 'AL'
Using FIPS code '001' for 'Autauga County'
Using FIPS code '01' for state 'AL'
Using FIPS code '001' for 'Autauga County'
Using FIPS code '01' for state 'AL'
Using FIPS code '001' for 'Autauga County'
sum(is.na(na_check$GEOID))/nrow(na_check)
[1] 1
`

Last thing -- these issues are not unique to the table ID (B01001) and occurred for exactly the same set of states for every table I tried [>40 randomly selected].

Failing states:

Working states:

Rename the package to tidyUScensus

The package should be renamed to something like tidyuscensus since it covers only data about US.

Error: Internal error: expected a vector

When using output = wide

df = get_acs(variables = c("B16008_002E", "B19013_001E", "B07401_001E",
                           "B13002_002E", "B24121_255E"), geometry = TRUE,
             geography = "state",
             output = "wide")

Error: Internal error: expected a vector

Option to choose 1-year or 5-year ACS estimates

I just started exploring tidycensus today and I love it so far. One quick question: is there a way to specify using 1-year or 5-year estimates? It looks like it's defaulting to pulling 5-year estimates which is great for small geographies, but sometimes I'd rather use 1-year estimates for larger/more populated areas. I know the acs package allows this choice.

As much more of an R user than programmer, I can't help with actually implementing this feature, but would certainly use it!

Query data for a list of GEOID (tracts)?

Hi,

Suppose I have a list of tract level GEOID's, and I want to query census data via get_decennial for this list. Can I do this within tidycensus? Or do I have to pull tract data for all states via get_decennial with the geography='tract' option, then subset the GEOID's that I am interested? The latter is inefficient, so would like to know if there is any way around this. Thanks!

Change api calls from http to https

You are probably also on the mailing list for this, but just in case, Census is changing their api from http to https.

To improve security and privacy, and by federal government mandate, the U.S. Census Bureau will stop receiving Application Programming Interface (API) calls athttp://api.census.gov/data/ on August 28, 2017.

HttpsChangeDocument.pdf

Block as invalid geography or geography hierarchy

Following your example from your personal website here, I'm attempting to look at rent rates, but at a block level rather than a census tract level. I used your exact code, but attempted to change 'tract' to 'block', 'block group', or 'blockgroup' to no avail. This might be a simple fix, I just haven't found any examples of how to use block data in your tidycensus package yet.

When I run the code below, I receive the following error message:

Error: You have specified an invalid geography or a geography hierarchy that is currently unavailable in tidycensus. Please visit http://api.census.gov/data/2015/acs5/geography.html for available geographies. tidycensus currently supports geographies nested within states and/or counties.

On your site, it looks as though block group is supported at geography level 150.

library(tidycensus)
library(tidyverse)
library(tigris)
library(sf)
options(tigris_class = "sf")
options(tigris_use_cache = TRUE)
# census_api_key("YOUR KEY HERE")

rent <- get_acs(geography = "block", variables = "DP04_0134", 
                state = c("WA", "OR", "CA"), geometry = TRUE)

Thanks for any and all help!

`Ticycensus` Package Loading Depends on `units` Package

Running devtools::install_github("walkerke/tidycensus") and then library(tidycensus) in a new R session will give you this error:

Error : object ‘set_units’ is not exported by 'namespace:units'
Error: package or namespace load failed for ‘tidycensus’

You can resolve that by simply running library(units) before library(tidycensus). In the future, it might be a good idea for units to be imported by tidycensus, though.

Adjusting margin of error calculations for multiple zero estimates

Hi, I have a question about the MOE functions--have you considered adjusting the derived MOE function for aggregated estimates that include multiple zero estimates?

Some Census documentation recommends using only one MOE from a 0 estimate when there are multiple 0 estimates in order to avoid unnaturally inflating the SE (see Census document here, p.21).

I see in the source code of moe_sum (and in testing the function on my data) multiple zeros aren't accounted for. Might this be something added to the function in the future?

Thanks--and PS, many thanks for creating this package.

Blocks an invalid geography for Census2000?

library(tidycensus)
library(tidyverse)
library(viridis)
library(ggplot2)
library(dplyr)

#------Get Decennial Census Data
ut2000tract <- get_decennial(geography = "tract", variables = "P001001", year = 2000, sumfile = "sf1",
key = mykey, state = "49", county = "035", geometry = FALSE)
ut2000BG <- get_decennial(geography = "block group", variables = "P001001", year = 2000, sumfile = "sf1",
key = mykey, state = "49", county = "035", geometry = FALSE)
ut2000block <- get_decennial(geography = "block", variables = "P001001", year = 2000, sumfile = "sf1",
key = mykey, state = "49", county = "035", geometry = FALSE)
Third case fails.

Meaning of "tidy" option in 'output' argument

Hi there,

according to https://cran.r-project.org/web/packages/tidyr/vignettes/tidy-data.html and my understanding of "tidy" data, variables should always be in their own column.

When downloading data with tidycensus , the "tidy" option in the output argument puts all dowloaded variables in one column which is exactly the non-tidy version.

I guess that output = wide would reflect the "tidy" idea of the tidyverse.
Especially because the package name is leaned on the tidyverse, I think that a change of the name argument name would be appropriate? :)

Thanks for the package, it really simplifies a lot 👍

Suggested approach for aggregating variables

Not an issue per se, just looking for some suggestions for how I'm using tidycensus. Let's say I needed the total female population as well as the female population under 10 years old. I'll use B01001_026 for the total female population and then I need to sum B01001_027 and B01001_028 to get the female population between 0 and 9 years (let's not worry about the MOE for now).

My first approach is to grab the data in "tidy" format and and then when I summarize, I subset the rows I want to be included in each summarized variable.

library(tidyverse)
library(tidycensus)

# define vars
fem_total_v <- "B01001_026"
fem_under10_v <- c("B01001_027", "B01001_028")

# download tidy acs 1 year estimates
state_estimate <- get_acs(
  geography = "state",
  variables = c(fem_total_v, fem_under10_v),
  survey = "acs1",
  year = 2015,
  output = "tidy",
  geometry = FALSE
)
#> The one-year ACS provides data for geographies with populations of 65,000 and greater.
state_estimate
#> # A tibble: 156 x 5
#>    GEOID     NAME   variable estimate   moe
#>    <chr>    <chr>      <chr>    <dbl> <dbl>
#>  1    01  Alabama B01001_026  2506169  5380
#>  2    01  Alabama B01001_027   138715  3426
#>  3    01  Alabama B01001_028   156432  5296
#>  4    02   Alaska B01001_026   349737  1732
#>  5    02   Alaska B01001_027    26878   993
#>  6    02   Alaska B01001_028    25525  1839
#>  7    04  Arizona B01001_026  3440680  2294
#>  8    04  Arizona B01001_027   210782  1481
#>  9    04  Arizona B01001_028   224756  5237
#> 10    05 Arkansas B01001_026  1514628  3921
#> # ... with 146 more rows

# summarize by subsetting
state_summary <- state_estimate %>%
  group_by(NAME) %>%
  summarize(
    fem_total = estimate[variable == fem_total_v],
    fem_under10 = sum(estimate[variable %in% fem_under10_v])
  )
state_summary
#> # A tibble: 52 x 3
#>                    NAME fem_total fem_under10
#>                   <chr>     <dbl>       <dbl>
#>  1              Alabama   2506169      295147
#>  2               Alaska    349737       52403
#>  3              Arizona   3440680      435538
#>  4             Arkansas   1514628      187759
#>  5           California  19704260     2473361
#>  6             Colorado   2715554      340014
#>  7          Connecticut   1838823      193358
#>  8             Delaware    487726       53741
#>  9 District of Columbia    352523       37462
#> 10              Florida  10380034     1095881
#> # ... with 42 more rows

Alternatively, I could get the "wide" data from get_acs() and then do a rowwise summation.

# download wide acs 1 year estimates
state_estimate_wide <- get_acs(
  geography = "state",
  variables = c(fem_total_v, fem_under10_v),
  survey = "acs1",
  year = 2015,
  output = "wide",
  geometry = FALSE
)
#> The one-year ACS provides data for geographies with populations of 65,000 and greater.
state_estimate_wide
#> # A tibble: 52 x 8
#>    B01001_026E B01001_026M B01001_027E B01001_027M B01001_028E B01001_028M
#>          <dbl>       <dbl>       <dbl>       <dbl>       <dbl>       <dbl>
#>  1     2506169        5380      138715        3426      156432        5296
#>  2      349737        1732       26878         993       25525        1839
#>  3     3440680        2294      210782        1481      224756        5237
#>  4     1514628        3921       93788        2299       93971        3715
#>  5    19704260        4443     1225165        2344     1248196       15177
#>  6     2715554        3494      163634        2180      176380        4200
#>  7     1838823        1923       91449         751      101909        3974
#>  8      487726        1047       26996         440       26745        2368
#>  9      352523         699       21104         447       16358        1348
#> 10    10380034        6494      537518        3650      558363       11535
#> # ... with 42 more rows, and 2 more variables: NAME <chr>, GEOID <chr>

# summarize rowwise
state_summary_wide <- state_estimate_wide %>%
  rowwise() %>%
  mutate(
    fem_total = B01001_026E,
    fem_under10 = sum(B01001_027E, B01001_028E)
    ) %>%
  select(NAME, fem_total, fem_under10) %>%
  arrange(NAME) %>%
  ungroup()
state_summary_wide
#> # A tibble: 52 x 3
#>                    NAME fem_total fem_under10
#>                   <chr>     <dbl>       <dbl>
#>  1              Alabama   2506169      295147
#>  2               Alaska    349737       52403
#>  3              Arizona   3440680      435538
#>  4             Arkansas   1514628      187759
#>  5           California  19704260     2473361
#>  6             Colorado   2715554      340014
#>  7          Connecticut   1838823      193358
#>  8             Delaware    487726       53741
#>  9 District of Columbia    352523       37462
#> 10              Florida  10380034     1095881
#> # ... with 42 more rows

identical(state_summary, state_summary_wide)
#> [1] TRUE

Is there an alternative approach you'd suggest or are these the two basic options? Thanks!

lexical error: invalid char in json text.

Running the following:
med_inc_2000 <- get_decennial(geography = "county", variables = "PCT045001", year = 2000, sumfile="sf3", state = "Maryland", output = 'wide')

results in:
Using FIPS code '24' for state 'Maryland'
Error : lexical error: invalid char in json text.
1"], ["31653","Prince George"s County","24","033"], ["38112"
(right here) ------^

Looking at the county names for Maryland shows: "Prince George's County" (without the double quotes). I'm not sure how to fix this error (unless there is a way to make it search by FIPS rather than county name). Any assistance would be appreciated.

Update: this also happens in Iowa for "O'Brien County" Iowa. The apostrophe seems to cause problems in accessing the censusapi

Lexical error: invalid char in json text when accessing 1-year ACS data for 2006-2011

I tried to use TidyCensus to access 1-year ACS data for 2006-2011 using the following:

df <- get_acs(geography = "state", table = "B01001", survey = "acs1", year = 2006)

I received the following error message at the console:

Error: lexical error: invalid char in json text.
                                       <html><head><title>Error report
                     (right here) ------^

The request works for 1-year data from 2012-2016. Possibly an issue with the census API, but I wanted to bring it to your attention.

json error with load_variables()

I used the load_variables example but got this error message: Error: lexical error: invalid char in json text. <html><head><title>Error report (right here) ------^

After looking around online, it looks like it might be an issue with parsing the JSON. Or am I doing something incorrectly?

A function to return smallest available geography

Variables like households by size are available at the block level, while other variables are only available at block group or even tract. It would be incredibly helpful to pass a vector of variable codes into a function that returned a vector of the smallest geography available for each. Because there are so many geographies, it may not be practical (or even possible) to rank them all smallest-to-largest, but the most common ones should be doable.

It could potentially be used as a helper function by your main macros (e.g. "get_acs"). Instead of returning a 400 bad request error, a pre-step could compare the desired geography to the smallest available geographies. If one wasn't available, the function would tell the user.

Anyway, awesome package! Thanks for all your hard work!

relax sf dependency

I wonder if you'd consider relaxing the sf dependency or splitting the mapping and api components of tidycensus into separate packages in the future. The reason I ask is that I'm using rstudio in scientific linux and current gdal libraries are not available (thus no sf).

Ideally we'd work on a different distro here but it also occurred to me that the api functionality could be separated from the geospatial stuff.

Thanks for the package!

Make the variables argument optional

Since this package wraps tigris so nicely, I'd like to be able to use it to download a shapefile without having to include variables. The variables and table arguments should still be required if geometry = FALSE.

Possible to Pull Down National Level Numbers?

I'm trying to pull down national numbers but the highest level of geography seems to be states, where I could get national estimates by adding up state values. I'd rather not do that if possible given the margins of error. Is there a way to do that within the tidycensus package? Thanks for developing this. Its really great stuff!

Ability to handle lots of variables

At the moment, given the package design, users are limited to 24 variables from get_acs and 49 variables from get_decennial. The package currently throws an error and encourages users to split their call up into multiple calls; however tidycensus should be able to handle this for them.

Missing margin of error

library(tidycensus)
census_api_key("2fa891ce3906b073c959075001b6099a4edf7110")
#> To install your API key for use in future sessions, run this function with `install = TRUE`.

geo <- get_acs(
  geography = "metropolitan statistical area/micropolitan statistical area",
  variables = "DP03_0021PE",
  summary_var = "B01003_001",
  survey = "acs1",
  endyear = 2016
)
#> The one-year ACS provides data for geographies with populations of 65,000 and greater.
#> Using the ACS Data Profile
subset(geo, summary_moe < 0 )
#> # A tibble: 394 x 7
#>    GEOID NAME                      variable estim…   moe summary… summary…
#>    <chr> <chr>                     <chr>     <dbl> <dbl>    <dbl>    <dbl>
#>  1 10140 "Aberdeen, WA Micro Area" DP03_00…  1.50  1.10     71628  -5.56e⁸
#>  2 10300 "Adrian, MI Micro Area"   DP03_00…  0.100 0.100    98504  -5.56e⁸
#>  3 10420 "Akron, OH Metro Area"    DP03_00…  1.10  0.300   702221  -5.56e⁸
#>  4 10460 "Alamogordo, NM Micro Ar… DP03_00…  0.700 1.00     65410  -5.56e⁸
#>  5 10540 "Albany, OR Metro Area"   DP03_00…  0     0.100   122849  -5.56e⁸
#>  6 10580 "Albany-Schenectady-Troy… DP03_00…  4.00  0.600   881839  -5.56e⁸
#>  7 10700 "Albertville, AL Micro A… DP03_00…  0.800 0.800    95157  -5.56e⁸
#>  8 10780 "Alexandria, LA Metro Ar… DP03_00…  1.30  0.600   154789  -5.56e⁸
#>  9 10900 "Allentown-Bethlehem-Eas… DP03_00…  2.00  0.400   835652  -5.56e⁸
#> 10 11020 "Altoona, PA Metro Area"  DP03_00…  0.800 0.600   124650  -5.56e⁸
#> # ... with 384 more rows

Looks like missing values are encoded as -555555555 - it'd be nice if this was converted to NA automatically.

error loading acs5 variables

Since today when I am trying to load the ACS table of variables I'm getting an error, and the table is not being loaded. The decennial table is just fine, as well the actual acs5 data sets I use.

I tried few tricks, without success. Just wondering if it is happening due a kind of packages update I may have done, or even because I'm using R on MacOS.

I appreciate your help. Thank you in advance!!

> v15_acs5 <- load_variables(2015, "acs5", cache = TRUE) # ACS variables
Error in bind_rows_(x, .id) : Argument 6 must be length 1, not 0