cheeaun / sgbusdata

Singapore Bus data

SG Bus Data

Singapore Bus data

This is a data-only repository to complement BusRouter SG.

The data

It's in the /data folder.

Or, browse through them on the data server: https://data.busrouter.sg/

Terminologies

• Stop - the bus stop itself
• Service - the bus service, with numbers like 133 or 1N
• Route - the route of a bus service, the lines connecting all the stops
• Pattern - the patterns of a bus route, one of the lines connecting the stops
  • If the route is one-way (A->B), there's only one pattern
  • If the route is two-way (A->B, B->A), there're two patterns in a route
  • Each pattern usually pass through different stops, as they're usually on opposite sides of the road.

Scripts

npm run cleanup

Delete files in data/v1/raw/services/*, data/v1/patch/*.cm.json and data/v1/patch/*.om.json, before running the commands below.

npm run fetch

• fetch-bus-stops - Fetch all bus stops from https://www.lta.gov.sg/map/busService/bus_stops.xml
• fetch-bus-services - Fetch bus services from https://www.lta.gov.sg/map/busService/bus_services.xml
• fetch-bus-services-route - Fetch all the XMLs and KMLs e.g.: https://www.lta.gov.sg/map/busService/bus_route_xml/10.xml, https://www.lta.gov.sg/map/busService/bus_route_kml/10-1.kml and https://www.lta.gov.sg/map/busService/bus_route_kml/10-2.kml.

npm run datamall

⚠️ This requires DatamallAccountKey environment variable. Make a copy of example.env file, rename to .env and put in your Datamall API key.

• fetch-bus-stops-datamall - Fetch all bus stops from http://datamall2.mytransport.sg/ltaodataservice/BusStops
• fetch-bus-routes-datamall - Fetch all bus routes from http://datamall2.mytransport.sg/ltaodataservice/BusRoutes

npm run patch

• patch-missing-routes - Patch missing routes (failed requests from fetch-bus-services-route) by getting fresh route data from OneMap/Citymapper.
• patch-multiline-routes - Patch multi-line routes (routes that has more than one line, we only need one line here, per pattern) by getting fresh route data from OneMap (again).
• patch-bus-stop-names - Patch bus stop names (some of them are all-uppercase instead of titlecase) by double-checking them on https://www.transitlink.com.sg/eservice/eguide/bscode_idx.php

npm run generate

• generate-data - Generate all the final clean data
• generate-data-firstlast - Generate the first/last timing data

npm run build

Run all npm commands above.

sgbusdata's Issues

The route LineString is wrong. Doesn't match with Stops.

Hello,

There is some wrong data in this bus service: https://busrouter.sg/?lng=en#/services/965T

502 Bad Gateway error on npm run patch

Running npm run patch results in an error when getting a token or bus route data from developers.gov.sg. Any link in the website just leads to a 502 Gateway Error. This error was first seen on 17 Dec 23 and may have been occurring at most since 2nd Dec 23. I can't seem to find a new endpoint for the /publicapi/publicsessionid or /publicapi/busexp/getBusRoutes

Github Actions Logs:

(From npm run patch)

> [email protected] patch
> node scripts/patch-missing-routes && node scripts/patch-multiline-routes && node scripts/patch-bus-stop-names

📖 Read file: ./data/v1/raw/bus-services.json
📖 Read file: ./data/v1/patch/bus-services-routes.failures.json
📖 Read file: ./data/v1/raw/services/TRUNK/114A.json
📖 Read file: ./data/v1/raw/services/TRUNK/307.json
Missing pattern 2 for service 307
📖 Read file: ./data/v1/raw/services/TRUNK/60T.json
📖 Read file: ./data/v1/raw/services/TRUNK/812T.json
📖 Read file: ./data/v1/raw/services/TRUNK/83T.json
📖 Read file: ./data/v1/raw/services/TRUNK/859T.json
📖 Read file: ./data/v1/raw/services/TRUNK/860T.json
🥏 https://developers.onemap.sg/publicapi/publicsessionid
🚨 Retrying HTTPError: Response code 502 (Bad Gateway) time(s)
🚨 Retrying HTTPError: Response code 502 (Bad Gateway) time(s)
🚨 Retrying HTTPError: Response code 502 (Bad Gateway) time(s)
🚨 Retrying HTTPError: Response code 502 (Bad Gateway) time(s)
🚨 Retrying HTTPError: Response code 502 (Bad Gateway) time(s)
node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

HTTPError: Response code 502 (Bad Gateway)
    at Request.<anonymous> (/home/runner/work/sgbusdata/sgbusdata/node_modules/got/dist/source/as-promise/index.js:117:42)
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  code: 'ERR_NON_2XX_3XX_RESPONSE',
  timings: {
    start: 1704071523771,
    socket: 1704071523772,
    lookup: 1704071523783,
    connect: 1704071524023,
    secureConnect: 1704071524264,
    upload: 1704071524264,
    response: 1704071524547,
    end: 1704071524548,
    error: undefined,
    abort: undefined,
    phases: {
      wait: 1,
      dns: 11,
      tcp: 240,
      tls: 241,
      request: 0,
      firstByte: 283,
      download: 1,
      total: 777
    }
  }
}