cal-itp / graas Goto Github PK

...instead of command line args

Need to define requirements, but they could include

• configure how many entities will query for vehicle-positions.pb at what frequency
• monitor best/avg/worst server response times for vehicle-positions.pb queries
• monitor server load during test run (server auto-scales up to 8 instances under load IIRC)
• specify duration of test run
• schedule recurring tests
• ensure they are not going to GRaaS Report
• re-deploy server after test runs
• Create json or yaml input

Reported by Ali

it references keys/sbc/id_rsa, which either needs to be updated to "clean-air-express", or the file needs to be removed

Folder per agency, with all historic files

Desert Roadrunner 1/11/21

Like Black or something similar

Prevent route-names.json from getting out-of-sync with static-gtfs files. This also streamlines agency onboarding.

1. Add TripListGenerator flags (headsign vs route name, direction) to filter-params.json. Rename it to something more generic, such as "agency-params".
2. Update TripListGenerator to check the gcloud bucket for agency-params.json, and use the flags listed in that file to run TripListGenerator. We'll need to update GCloudStorage.java to include a function for getting an object from the bucket.
3. Update TripListGenerator to save results directly to route-names.json in the gcloud bucket. Rename to trip-names.json while we're at it.
4. Create a daily job on GH actions that checks when route-names.json was last updated, and when the static feed was last updated. If the feed was updated after, re-generate trip-names.json. Include some form of alerting that tells us what was updated.

Comment from @unreasonableman:

Building a system to generate static GTFS feeds isn't really a trivial undertaking, but we may be able to iteratively ease our way into it.

One of the big issues for creating static GTFS is to specify lists of lat/long values for the shapes.txt file. A possible approach is to use the web app as is. Manually create a trips.txt file, and then let the driver/operator/etc. run the web app as usual. Route points will be recorded into our DB, and can subsequently be extracted to create shapes.txt.

There is of course also the issue of creating timetables for routes, which we will need to figure out how to do (OCR of existing paper schedules, export from excel sheets, etc.)

A proposal to free drivers from the grind of having to start and stop trips manually.

• On-device web app: add patrol mode so that app constantly sends location updates without trip ID, but with vehicle ID
• Desktop web client for dispatcher
  • auth through QR code or copy-pasted agency key
  • inactive list of buses that aren't assigned trips
  • active list of buses that are assigned trips
  • switching buses between lists
    • to activate, select inactive vehicle and choose Assign Trip, which brings up trip dropdown
    • to deactivate, select vehicle and choose End Trip
• Server
  • new endpoint /assign-vehicle expects JSON that has agency-id, vehicle-id and trip-id
    • create unique id: <agency-id>-<vehicle-id>
    • write id and trip id to Assignment DB table
    • trip-id can be unassigned
  • regular vehicle position update endpoint:
    • if update without trip-id comes in, create unique id: <agency-id>-<vehicle-id>. Get trip-id value from assignment table. If unassigned add to unassigned feed (see below), otherwise add trip-id field and proceed with normal update flow
• Map
  • to show unassigned buses on map, create new feed /unassigned-vehicle-positions which has the same structure as /vehicle-positions, except trip IDs are all unassigned

Currently a nightly job creates one "dashboard" image per agency, containing info on each trip they took that day using the GRaaS app. Example:

• Create a webpage where current and historic reports can be accessed
• Create an interactive component to the report, where individual trips can be zoomed-in, where tooltips show trip metadata (what currently is shown below the trip title), and where one can use the timeline at the top to filter/highlight the maps below.

Provide billing updates to allow for easy monitoring of cost. Basically, to avoid surprises.

Use gcloud billing java API.

Send email saying "no trips today"

Service may be different than normal on New Years Day, Christmas etc. Include this detail in GRaaS (ie incorporate calendar_dates.txt in addition to calendar.txt.

This has been the root cause for some agency issues, leading to the Calendar value being null on routes_names.json

When I run the web app and try to load the QR scanner, my camera activates (green light turns on), but no camera view appears. It is still able to scan QR codes. If I start a new tab in safari and then go back to the web app, the camera view appears

I recently updated my Macbook Air to MacOS Monterey, and my best guess is this is an issue with the latest version of Safari.

It works fine in Chrome on the same machine, as well as on Safari on my old iPad

This html-qrcode issue captures the same issue (no resolution yet, though).

Copied from GRaaS Staging, notes from @unreasonableman:

Scanned QR code for TCRTA on ipad, then pressed Start button. Dropdowns were empty. Checked filter params and route bucket files, both looked reasonable. Ran web app on desktop safari wit js console open to debug. Routes showed up. Went back to ipad, quit out of safari there and restarted. Issue did not persist past restart.

This may be harder to debug, but it's a bad first impression if it happens.

Additional detail: the dropdowns were completely blank, i.e. didn't have "Select Route" or "Select Vehicle". This suggests either a crash or a stall, and not a filtering issue.

We have a backend for creating agency service alerts - build a front end.

• - In addition to trip name and time, show vehicle ID, device type and last 4 characters of UUID. This will let us track device (type) performance over time even if devices are switched around on vehicles
• - Size the location update dot based on how many reports for that exact position have come in. To show "stuck" GPS.
• - Measure frequency of new GPS updates

As a part of agency onboarding, we set up several json files for each agency, containing vehicle, route & filter parameters. Create a front-end that GRaaS field ops personnel can use to generate, update and modify these values.

The goal would be to streamline/automate the Create service data for agency
and Create keys for agency sections of our onboarding runbook.

If route_name is used for one, it should be used for the other. Same with headsign.

The GRaaS report treats trip_id as a unique identifier. This generally works, as there should only be one "map" per trip, but there are some exceptions, such as:

1. Two drivers select the same trip (presumably one does so erroneously)
2. Two devices are running on the same vehicle (ie babyshark & web app, which we've experienced)

In either of these cases, the GRaaS report will arbitrarily consolidate/overwrite the trips, so that they appear as a single trip.

To address this, we could group data on the GRaaS report by "sessionID" in addition to tripID, so that two maps of the same trip can exist side by side.

SesssionID has been implemented already in the web app, which means the remaining step is to handle trip inference. This can be done one of two ways:
A) Assign a sessionID for trip inference devices (ie new daily unique identifier)
B) In the GRaaS report generation logic, make up a session ID for trip inference devices.

I think B would be sufficient.

Create a website that agencies can use to access various GRaaS tools - such as service alert creation and the GRaaS report.

This could overlap/be shared with existing Cal-ITP infrastructure.

Both the driver + iPad and TripInference appear to have selected a weekday route during Saturday service.

All four trips in the screenshotted saturday GRaaS report are weekday trips, according to both Glenn's static gtfs and route-names.json.

Note that #80 produced a bad solution, with links unable to be previewed

Instructions from here:

1. When deploying, use the command “gcloud app deploy --no-promote”
2. Get the new versionID from the versions page, and access it via the URL: https://<VERSION_ID>-dot-default-dot-lat-long-prototype.wl.r.appspot.com/
3. After testing, use the same versions page to manually migrate traffic from the previous version to the current version

Kay's note on #98:

We are actually pretty close to making this class a very light-weight java API for sendgrid. It's kind of a separate project, but really what's missing is:

• changing name of map back to blobMap
• pass in a list of mime types instead of assuming image\png
• remove hardcoded sender, and instead use SendGrid.Builder approach to create and configure instances

service_account_key's are being deprecated (according to warning that appeared on github). We need to set up Workload Identity Federation instead.

Actions to update:

• pr-test.yaml
• send-graas-report.yaml

First we need a new slack webhook, likely requiring a paid plan. Then we need to create a SlackFailureReporter and set it up in LoadAGencyDataTest and/or in GraphicReport.

Build alerting that notifies us of server issues.