Interface to batch routing API about cyclestreets-r

[Robinlovelace]

Great news, it now works! Heads-up @mvl22 (and many thanks for implementing this) and @temospena (another potential 'early adopter' of this function) 🎉

[Robinlovelace]

Example inputs and envisioned outputs can be found here: https://github.com/cyclestreets/cyclestreets-r/tree/master/data-raw

Function on the R side could be something like:

journey_batch(od_geo, plan = "balanced")

on the R side.

[Robinlovelace]

Results from quick test below.

# starting from outputs
library(tmap)
tmap_mode("view")
#> tmap mode set to interactive viewing
piggyback::pb_list()
#> Error in libgit2::git_repository_discover: could not find repository from '/tmp/RtmppYpY4d/reprex-71d0d42538d59-flat-ram'
u1 = "https://github.com/cyclestreets/cyclestreets-r/releases/download/v0.5.3/batchtesting.-.with.Include.full.JSON.API.response.option.geojson"
u2 = "https://github.com/cyclestreets/cyclestreets-r/releases/download/v0.5.3/batchtesting.-.with.Include.route.string.column.geojson"
u3 = "https://github.com/cyclestreets/cyclestreets-r/releases/download/v0.5.3/batchtesting.-.with.neither.option.geojson"
route1 = sf::read_sf(u1)
route2 = sf::read_sf(u2)
route3 = sf::read_sf(u3)
route1
#> Simple feature collection with 2 features and 43 fields
#> Geometry type: LINESTRING
#> Dimension:     XY
#> Bounding box:  xmin: -0.24156 ymin: 51.56121 xmax: 0.04791 ymax: 51.58567
#> Geodetic CRS:  WGS 84
#> # A tibble: 2 × 44
#>   start_id start_longitude_request start_latitude_re… finish_id finish_longitud…
#>   <chr>                      <dbl>              <dbl> <chr>                <dbl>
#> 1 0_0_0                     -0.241               51.6 0_0_3              -0.0242
#> 2 0_0_0                     -0.241               51.6 0_0_4               0.0481
#> # … with 39 more variables: finish_latitude_request <dbl>, start <chr>,
#> #   finish <chr>, startBearing <chr>, startSpeed <chr>, start_longitude <chr>,
#> #   start_latitude <chr>, finish_longitude <chr>, finish_latitude <chr>,
#> #   crow_fly_distance <chr>, event <chr>, whence <chr>, speed <chr>,
#> #   itinerary <chr>, clientRouteId <chr>, plan <chr>, note <chr>, length <chr>,
#> #   time <chr>, busynance <chr>, quietness <chr>, signalledJunctions <chr>,
#> #   signalledCrossings <chr>, west <chr>, south <chr>, east <chr>, …
route2
#> Simple feature collection with 2 features and 10 fields
#> Geometry type: LINESTRING
#> Dimension:     XY
#> Bounding box:  xmin: -0.25569 ymin: 51.58447 xmax: -0.03854 ymax: 51.59919
#> Geodetic CRS:  WGS 84
#> # A tibble: 2 × 11
#>   start_id start_longitude_request start_latitude_re… finish_id finish_longitud…
#>   <chr>                      <dbl>              <dbl> <chr>                <dbl>
#> 1 0_0_0                     -0.256               51.6 0_0_4              -0.0820
#> 2 0_0_0                     -0.256               51.6 0_0_5              -0.0386
#> # … with 6 more variables: finish_latitude_request <dbl>, strategy <chr>,
#> #   distance <chr>, time_seconds <chr>, calories <chr>,
#> #   geometry <LINESTRING [°]>
route3
#> Simple feature collection with 2 features and 10 fields
#> Geometry type: LINESTRING
#> Dimension:     XY
#> Bounding box:  xmin: -0.24124 ymin: 51.58359 xmax: 0.04808 ymax: 51.58359
#> Geodetic CRS:  WGS 84
#> # A tibble: 2 × 11
#>   start_id start_longitude_request start_latitude_re… finish_id finish_longitud…
#>   <chr>                      <dbl>              <dbl> <chr>                <dbl>
#> 1 0_0_0                     -0.241               51.6 0_0_3              -0.0242
#> 2 0_0_0                     -0.241               51.6 0_0_4               0.0481
#> # … with 6 more variables: finish_latitude_request <dbl>, strategy <chr>,
#> #   distance <chr>, time_seconds <chr>, calories <chr>,
#> #   geometry <LINESTRING [°]>
qtm(route1, col = names(route1)[1]) # it's a long route!

plot(route1$geometry[1], lwd = 5, col = "red")
plot(route1$geometry[2], add = TRUE)

# Let's take a look at segment level data
route1$distances
#> [1] "15,18,11,12,12,22,15,54,3,48,7,49,27,4,19,3,6,10,9,179,21,51,7,9,18,29,14,18,19,31,15,10,13,47,5,58,20,23,10,4,3,9,12,4,17,11,13,81,26,23,27,31,42,51,1,118,15,41,12,30,33,70,16,20,37,108,28,66,31,5,24,15,10,15,46,6,42,9,5,6,6,3,5,2,13,6,6,5,5,6,2,7,7,7,8,5,5,4,7,7,6,7,13,6,6,7,7,6,7,6,7,5,6,40,32,42,4,38,9,7,9,7,8,10,14,44,10,11,9,8,7,11,12,21,15,8,90,66,10,142,10,31,13,6,7,4,5,14,4,45,234,84,9,14,21,80,5,9,12,42,24,45,82,9,11,37,24,15,37,2,5,2,3,1,4,7,18,6,101,114,3,116,25,15,46,21,103,40,66,48,19,44,293,10,95,44,33,41,21,16,14,20,40,70,75,6,11,18,12,13,37,20,2,24,11,16,10,8,7,14,14,15,36,39,28,23,33,17,27,8,9,3,4,14,16,30,68,32,24,27,44,30,23,20,26,14,10,10,31,20,27,22,11,14,4,3,5,30,35,24,48,15,15,13,11,12,10,9,13,2,53,11,6,8,14,2,6,16,13,3,1,3,4,3,3,3,6,9,9,20,14,23,7,3,12,16,51,50,26,46,21,16,24,62,39,27,33,24,14,19,14,32,19,11,27,30,30,23,31,29,10,20,108,91,56,33,85,30,36,7,17,22,22,4,40,26,14,14,43,16,7,25,36,37,56,48,36,32,27,40,40,7,6,50,49,44,14,24,35,28,7,6,21,41,41,15,4,7,9,10,36,34,60,4,4,13,24,19,3,15,11,11,39,7,10,60,6,15,19,35,53,9,9,21,32,26,3,14,7,8,67,11,8,79,38,23,8,8,55,24,38,12,51,53,21,25,39,27,106,15,14,38,9,56,49,56,10,17,30,11,1,10,8,4,32,32,24,28,24,43,19,17,15,18,18,15,15,12,11,39,26,12,23,8,54,70,71,9,58,17,44,16,58,43,20,25,29,29,4,7,19,26,24,34,5,4,17,22,3,4,4,4,58,4,4,10,1,8,3,5,31,197,20,21,9,27,8,23,34,97,53,21,12,13,20,4,2,4,3,3,5,7,3,4,2,3,8,37,21,19,71,7,34,42,72,75,73,76,16,73,10,20,9,4,3,29,13,12,60,3,99,6,5,4,4,4,5,19,43,3,3,3,4,4,2,2,2,2,4,7,10,6,12,9,8,78,24,105,19,22,25,10,8,12,41,8,9,8,8,27,9,9,8,8,4,3,3,2,3,3,34,38,13,16,26,16,10,5,4,4,10,9,2,15,20,5,6,4,3,9,22,6,34,26,24,14,16,15,11,14,100,80,45,9,7,9,64,7,8,5,10,9,7,4,4,9,12,36,31,12,17,41,44,10,27,146,47,29,9,10,23,32,48,17,33,74,60,51,10,6,7,49,16,44,7,31,13,13,4,14,20,33,14,19,14,21,17,44,77,16,14,28,3,29,62,11,12,6,22,7,16,13,5,5,19,43,33,13,26,41,10,7,6,10,17,36,24,4,2,5,13,87,16,3,4,4,3,7,9,7,16,50,4,4,8,18,43,24,65,25,282,67,36,109,27,2,2,5,33,8,5,3,14,8,21,15,24,8,2,7,10,9,11,10,16,4,3,4,4,4,13,4,13,8,7,5,6,6,4,5,3,15,13,11,14,12,18,21,17,10,3,3,8,10,6,5,4,10,9,4,5,5,5,2,13,99,6,64,4,26,19,14,28,35,21,45,15,8,8,17,52,69,40,7,85,8,59,65,74,75,11,36,13,20,38,58,8,60,13,16,29,42,25,53,8,6,9,7,174,18,92,38,10,21,5,16,8,21,11,7,8,11,37,50,42,39,20,13,23,22,9,4,3,2,3,2,12,13,3,5,17,63,29,23,25,1,14,7,5,9,46,6,5,4,7,39,4,47,3"                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   
#> [2] "15,18,11,12,12,22,15,54,3,48,7,49,27,4,19,3,6,10,9,179,21,51,7,9,18,29,14,18,19,31,15,10,13,47,5,58,20,23,10,4,3,9,12,4,17,11,13,81,26,23,27,31,42,51,1,118,15,41,12,30,33,70,16,20,37,108,28,66,31,5,24,15,10,15,46,6,42,9,5,6,6,3,5,2,13,6,6,5,5,6,2,7,7,7,8,5,5,4,7,7,6,7,13,6,6,7,7,6,7,6,7,5,6,40,32,42,4,38,9,7,9,7,8,10,14,44,10,11,9,8,7,11,12,21,15,8,90,66,10,142,10,31,13,6,7,4,5,14,4,45,234,84,9,14,21,80,5,9,12,42,24,45,82,9,11,37,24,15,37,2,5,2,3,1,4,7,18,6,101,114,3,116,25,15,46,21,103,40,66,48,19,44,293,10,95,44,33,41,21,16,14,20,40,70,75,6,11,18,12,13,37,20,2,24,11,16,10,8,7,14,14,15,36,39,28,23,33,17,27,8,9,3,4,14,16,30,68,32,24,27,44,30,23,20,26,14,10,10,31,20,27,22,11,14,4,3,5,30,35,24,48,15,15,13,11,12,10,9,13,2,53,11,6,8,14,2,6,16,13,3,1,3,4,3,3,3,6,9,9,20,14,23,7,3,12,16,51,50,26,46,21,16,24,62,39,27,33,24,14,19,14,32,19,11,27,30,30,23,31,29,10,20,108,91,56,33,85,30,36,7,17,22,22,4,40,26,14,14,43,16,7,25,36,37,56,48,36,32,27,40,40,7,6,50,49,44,14,24,35,28,7,6,21,41,41,15,4,7,9,10,36,34,60,4,4,13,24,19,3,15,11,11,39,7,10,60,6,15,19,35,53,9,9,21,32,26,3,14,7,8,67,11,8,79,38,23,8,8,55,24,38,12,51,53,21,25,39,27,106,15,14,38,9,56,49,56,10,17,30,11,1,10,8,4,32,32,24,28,24,43,19,17,15,18,18,15,15,12,11,39,26,12,23,8,54,70,71,9,58,17,44,16,58,43,20,25,29,29,4,7,19,26,24,34,5,4,17,22,3,4,4,4,58,4,4,10,1,8,3,5,31,197,20,21,9,27,8,23,34,97,53,21,12,13,20,4,2,4,3,3,5,7,3,4,2,3,8,37,21,19,71,7,34,42,72,75,73,76,16,73,10,20,9,4,3,29,13,12,60,3,99,6,5,4,4,4,5,19,43,3,3,3,4,4,2,2,2,2,4,7,10,6,12,9,8,78,24,105,19,22,25,10,8,12,41,8,9,8,8,27,9,9,8,8,4,3,3,2,3,3,34,38,13,16,26,16,10,5,4,4,10,9,2,15,20,5,6,4,3,9,22,6,34,26,24,14,16,15,11,14,100,80,45,9,7,9,64,7,8,5,10,9,7,4,4,9,12,36,31,12,17,41,44,10,27,146,47,29,9,10,23,32,48,17,33,74,60,51,10,6,7,49,16,44,7,31,13,13,4,14,20,33,14,19,14,21,17,44,77,16,14,28,3,29,62,11,12,6,22,7,16,13,5,5,19,43,33,13,26,41,10,7,6,10,17,36,24,4,2,5,13,87,16,3,4,4,3,7,9,7,16,50,4,4,8,18,43,24,65,25,282,67,36,109,27,2,2,5,33,8,5,3,14,8,21,15,24,8,2,7,10,9,11,10,16,4,3,4,4,4,13,4,13,8,7,5,6,6,4,5,3,15,13,11,14,12,18,21,17,10,3,3,8,10,6,5,4,10,9,4,5,5,5,2,13,99,6,64,4,26,19,14,28,35,21,45,15,8,8,17,52,69,40,7,85,8,59,65,74,75,11,36,13,20,38,58,8,60,13,16,29,42,25,53,8,6,9,7,174,18,92,38,10,21,5,16,8,21,11,7,8,11,37,50,42,39,20,13,23,22,9,4,3,2,3,2,5,6,15,3,6,30,110,23,58,31,66,84,11,51,16,8,28,61,8,15,3,4,4,5,9,11,5,3,2,5,4,2,15,40,92,11,8,58,12,11,27,32,21,29,16,19,28,28,4,90,3,9,11,26,32,4,4,3,3,25,10,6,41,4,6,16,14,41,17,60,33,14,36,10,8,10,9,11,11,14,9,9,7,4,15,56,32,12,9,15,8,5,4,4,36,4,65,72,62,28,5,43,22,2,6,4,4,5,5,6,6,4,32,54,26,10,46,82,18,34,6,27,6,5,13,7,12,16,12,39,20,10,12,8,4,2,4,8,7,8,9,7,9,9,11,8,22,7,15,120,83,7,10,4,4,40,94,6,180,26,12,21,37,28,97,14,21,18,25,37,3,4,16,17,2,19,6,19,12,16,10,30,15,63,41,25,33,4,1,27,22,38,2,5,9,22,34,6,20,11,7,21,28,125,108,61,141,95,131,78,29,8,36,70,16,23,11,29,53,74,21,53,15,7,6,19,21,4,8,11,27,15,8,14,6,26,84,17,30,20,20,8,66,40,65,11,11,63,110,14"

^{Created on 2022-03-22 by the reprex package (v2.0.1)}

[Robinlovelace]

Plan: create a function called batch_journey() that will take arguments TBC in collaboration with @mvl22. Good news: we have example outputs so I can work on this before our meeting next week. See attached.
batchtesting-data.csv.gz

[Robinlovelace]

Planning to work on this today. I may try using httr2 instead of httr: https://github.com/r-lib/httr2

Talk soon @mvl22 !

[Robinlovelace]

Heads-up @mvl22 this is the output from the {geojsonsf} package that I'd like to pass to the batch routing API:

{"type":"LineString","coordinates":[[-1.5187101220052267,53.792230071570809],[-1.5460238195984054,53.79586646502788]]} 
{"type":"LineString","coordinates":[[-1.5460238195984054,53.79586646502788],[-1.5460238195984054,53.79586646502788]]} 

Currently it's failing with the following message:

Response [https://api.cyclestreets.net/v2/batchroutes.createjob?key=xxx]
  Date: 2022-05-14 00:05
  Status: 200
  Content-Type: application/json; charset=UTF-8
  Size: 77 B
{
    "error": "The GeoJSON structure does not contain a features section."

Can you enable it to accept the IMO fairly uncontroversial input above?

For reference the tested example is a featurecollection with an unconventional (as far as I know) ID definition:

{"type": "FeatureCollection", "features": [
      {"type": "Feature", "id": 1, "properties": {}, "geometry": {"type": "Point", "coordinates": [0.14187, 52.20303]}},
      {"type": "Feature", "id": "a", "properties": {}, "geometry": {"type": "Point", "coordinates": [0.14711, 52.20061]}},
      {"type": "Feature", "id": 56, "properties": {}, "geometry": {"type": "Point", "coordinates": [0.11638, 52.20360]}}
    ]}

That's the current blocker on this ATM.

[Robinlovelace]

Another GeoJSON representation that would be ideal:

{
"type": "FeatureCollection",
"name": "coord_example2",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:OGC:1.3:CRS84" } },
"features": [
{ "type": "Feature", "properties": { "id": "S02001616S02001616" }, "geometry": { "type": "LineString", "coordinates": [ [ -3.2235272, 55.9298953 ], [ -3.2101257, 55.9334341 ] ] } },
{ "type": "Feature", "properties": { "id": "S02001616S02001616" }, "geometry": { "type": "LineString", "coordinates": [ [ -3.2101018, 55.9346315 ], [ -3.2203651, 55.930717 ] ] } },
{ "type": "Feature", "properties": { "id": "S02001616S02001620" }, "geometry": { "type": "LineString", "coordinates": [ [ -3.2100342, 55.932279 ], [ -3.2064407, 55.9484088 ] ] } }
]
}

[mvl22]

It does already expect a FeatureCollection with LineStrings.

You will recall I changed this from the original prototype using CSV input to the API release using GeoJSON with LineStrings.

Another GeoJSON representation that would be ideal:

Yes, that is what is now supported, except for the id position which I am working through.

For reference the tested example is a featurecollection with an unconventional (as far as I know) ID definition:

That contained points, so was failing.

[Robinlovelace]

Yes, that is what is now supported, except for the id position which I am working through.

Great to hear, many thanks Martin! Just to clarify, this should now (or soon) work?:

{
"type": "FeatureCollection",
"name": "coord_example2",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:OGC:1.3:CRS84" } },
"features": [
{ "type": "Feature", "properties": { "id": "S02001616S02001616" }, "geometry": { "type": "LineString", "coordinates": [ [ -3.2235272, 55.9298953 ], [ -3.2101257, 55.9334341 ] ] } },
{ "type": "Feature", "properties": { "id": "S02001616S02001616" }, "geometry": { "type": "LineString", "coordinates": [ [ -3.2101018, 55.9346315 ], [ -3.2203651, 55.930717 ] ] } },
{ "type": "Feature", "properties": { "id": "S02001616S02001620" }, "geometry": { "type": "LineString", "coordinates": [ [ -3.2100342, 55.932279 ], [ -3.2064407, 55.9484088 ] ] } }
]
}

[mvl22]

The real download URLs in the output is now implemented.

[mvl22]

Great to hear, many thanks Martin! Just to clarify, this should now (or soon) work?:

Yes, use of IDs in the properties is now supported.

[Robinlovelace]

Edge case bug in attached zip.
test-data-batch.csv.gz

[Robinlovelace]

Reproducible error:

routes = batch_read(file = "test-data-batch.csv.gz")
Reading in the following file:
test-data-batch.csv
NA values detected
NA values detected
NA values detected
NA values detected
NA values detected
NA values detected
NA values detected
Error in `[[<-.data.frame`(`*tmp*`, i, value = c("1", "1", "1", "1", "1",  : 
  replacement has 5912 rows, data has 5940
In addition: Warning messages:
1: In data.frame(..., check.names = FALSE) :
  row names were found from a short variable and have been discarded
2: In data.frame(..., check.names = FALSE) :
  row names were found from a short variable and have been discarded
Called from: `[[<-.data.frame`(`*tmp*`, i, value = c("1", "1", "1", "1", "1", 
"1", "1", "1", "2", "2", "2", "2", "2", "2", "2", "2", "2", "2", 
"2", "2", "2", "2", "2", "2", "2", "2", "2", "2", "2", "2", "2", 
"2", "2", "2", "2", "2", "2", "2", "3", "3", "3", "3", "3", "3", 
"3", "4", "4", "5", "5", "5", "5", "5", "5", "5", "5", "5", "6", 
"6", "7", "7", "7", "7", "8", "8", "8", "8", "8", "8", "8", "8", 
"8", "8", "8", "8", "8", "8", "9", "9", "9", "9", "9", "9", "9", 
"10", "10", "10", "10", "10", "10", "10", "10", "11", "11", "11", 
"12", "12", "12", "12", "13", "13", "13", "13", "13", "13", "13", 
"13", "13", "14", "14", "15", "15", "15", "15", "16", "16", "16", 
"16", "16", "16", "17", "17", "17", "17", "17", "17", "17", "17", 
"18", "18", "19", "19", "19", "19", "19", "20", "20", "20", "20", 
"20", "20", "20", "20", "20", "20", "21", "21", "21", "21", "21", 
"22", "23", "23", "23", "23", "23", "23", "23", "23", "24", "24", 
"24", "24", "24", "24", "24", "24", "24", "24", "25", "25", "25", 
"25", "25", "25", "25", "25", "25", "25", "26", "26", "26", "26", 
"26", "26", "26", "26", "26", "26", "26", "27", "28", "28", "28", 
"28", "28", "28", "29", "29", "29", "29", "29", "29", "29", "29", 
"30", "30", "30", "30", "30", "30", "30", "30", "30", "30", "30", 
"30", "30", "31", "31", "31", "31", "31", "31", "32", "32", "32", 

[Robinlovelace]

This is the failing line:

  res_df$id = elev_df$id

[Robinlovelace]

This is now fixed!

[Robinlovelace]

Code below.

[Robinlovelace]

To set the username try changing...

[Robinlovelace]

You can set the username option in the batch() function as follows:

cyclestreets-r/R/batch.R

Line 45 in 4cbc639

#' routes = batch(desire_lines, username = "robinlovelace", wait_time = 5)