worldbank-transport / open-transit-indicators Goto Github PK

On encountering an error during demographic shapefile import, the error handler fails with
TypeError(u"'source_file' is an invalid keyword argument for this function",)
at the line
self.problemClass.objects.create(**params)

Currently a user is still able to calculate indicators even if some required configuration parameters are missing. One particularly problematic parameter is Distance Buffer, which if not filled in will cause the Scala process to hang indefinitely.

The config file for the Angular app /js/angular/app/scripts/config.js has hardcoded the port for the Windshaft tiler to 8067 -- this should be moved to a separate file for easier provisioning.

Some not-so-specific classes in our main.scss override the input styles which errantly get applied to these elements.

The indicator calculation process currently fails when the instance doesn't have real-time data uploaded.

This one is a little weird. A potential bug was dealt with in a less than ideal way in MR #279 - it may warrant more exploration.

Basically, the less than ideal Map[Period, Map[String, Trip]] which is currently used for 'real time' trip mappings from scheduled trip IDs is used instead of the simpler Map[String, Trip] (which ought to be sufficient) because the latter returns the correct results + 24 hours. Something odd is apparently going on with dates.

Though this workaround might be hiding a more nefarious bug, progress is blocked on this merge and I've been unable to find evidence that such a bug is certain.

A token is passed to scala when creating a scenario. Right now, the oti-admin token is always used, which likely causes the created_by field to be set to oti-admin

Reproduce:

• Navigate directly to: /#/indicators/data
• Click the transit tab
• Some map tiles will not load and the map will be zoomed out

No console errors thrown or map tile 404's. Leaflet scope object looks ok.

It appears that the y axis scale on bar charts defaults to [0,1] if the max indicator value displayed in the chart is less than 1.

Zip files created on Macs currently include a folder called "__MACOSX" which contains Mac-OS-specific metadata. This causes the GTFS import Celery task to fail (the upload will fail with a timeout but the error is visible in the Celery logs). This folder should be ignored or deleted when uploading gtfs feeds (and other file types too, if it makes a difference).

Workaround: Run zip -d <gtfs_feed.zip> "__MACOSX/*" to manually remove the metadata from the zip file.

One of my indicator processing jobs crashed (meaning the status got stuck at 'processing'), and when I tried to kick off another, it remained stuck in the 'queued' status forever until I deleted the existing job that was supposedly still 'processing'.

Perhaps inspecting the celery job status directly would give us more granular info about job statuses?

Currently the app defines the indicator color palettes in two spots.

1. Windshaft, to draw the indicator geometries
2. Angular, to draw the map legends

Ideally, we should pass the geometry to windshaft as a get param, storing the style definitions in only one place.

Reproducable for datasets approx greater than 12mb.
wash-dc.zip, etc

Leaves app in bad state where data partially loaded, should delete partially loaded data on failure

If the scala indicator calculations throw an error, the celery job continues to poll indefinitely for status, and the UI shows the calculations as 'processing', and do not allow the user to cancel or retry the job.

The spray endpoint should return an error response if the job fails, allowing the celery job to update the status appropriately and end the task.

Although there is an exception handler defined in the implicit scope for the spray routes, it doesn't appear to actually catch errors. The handler translates its exception into JSON to send back, but since it's not catching exceptions, celery is getting an unexpected response, resulting in the UI displaying the error message "No JSON object could be decoded: line 1 column 0 (char 0)".

Will need this if we decide to do more advanced tasks with the API

This is likely a trailing slash fail on one of the many endpoints we have.

Attempting to upload stop times for a midsize network (SEPTA bus) results in a timeout error; the celery import task takes tens of minutes.

There is a SQL injection vulnerabililty in Indicator.prototype.getSql of oti-indicator.js.

This url successfully drops the table gtfs_stops:

http://localhost:8067/tiles/transit_indicators/0/%27%29%20as%20hi;%20drop%20table%20gtfs_stops;%20SELECT%20*%20FROM%20%28SELECT%20formatted_value%20as%20value,%20ntile%285%29%20over%20%28order%20by%20value%29%20as%20ntiles_bin,%20the_geom%20FROM%20transit_indicators_indicator%20WHERE%20type=%27/morning/route/10/160/394.png

It looks like we'll have to find a reasonable way to escape or sanitize strings. Stripping out all non alphanumerics should do it.

E.g. Directly navigate via url bar to /settings/overview then switch to /transit.
Voila! Map bounds broken. No console errors.

The recent authentication changes have caused all of the Django tests to break (401 errors on responses to all queries).

The settings field for entering the average fare does not allow entering values with a decimal portion.

PR #355 squashes the timeout bug seen with slightly larger systems of GTFS data and introduces a new issue: without some kind of celery polling (similar to that used in indicator calculation), the client is not going to be able to determine the status of a GTFS ingestion.

A mid-sprint card has been added to add this functionality.

On uploading a GTFS, if it passes validation, but the parser fails to load it, the UI returns a cryptic message that seems to indicate the feed failed validation. All feeds that pass validation should load through the parser, but we should display a more meaningful error message if it doesn't.

Geotrellis tests pass when timezone is left at default (UTC), but fail with timezone set to EST.

Deployment failed to install requirements for me. Two issues: first with ipdb. Though version 0.8 is specified, and pip install ipdb will install version 0.8, attempting to install with the version specified fails. The second issue is with transitfeed. It looks like the most recent version in pypi right now is 1.2.11 (not the latest, 1.2.12, which is specified in requirements.txt).

Creates directories owned by root that cannot be accessed

When the language dropdown is used to set the interface language, the map zooms out to show the whole world.

As mentioned in 728989c, the SEPTA rail length calculation is showing a total of 551 kilometers, where it should actually be somewhere around 450.

Currently, these colors are defined in two places:
js/windshaft/oti-indicators.js
js/angular/app/scripts/config.js

When running the grunt tasks to combine and minify javascript, errors are introduced.

Currently only loads data into memory during an upload

Occasionally, when going to the 'Settings' page after a fresh provisioning, the legend shows in the corner, although the map is not in the background.

When loading the page, first the Demographic check box is activated, followed by Configuration. GTFS, boundary, and real-time aren't activated until I click on them. Clicking on users makes the real-time and GTFS check boxes disappear.

Should also send a DELETE request to somewhere to clear the database gtfs dataset

PR #243 changes values to make tests run and comments out now broken tests - it would be nice if we piggypacked off of testkit and gtfs-test fixtures to use the ad hoc transit system Rob built. It would, in addition to being easy to pop in, allow us to reason in a clearer fashion about what we should expect results to be.

On the Transit Map page, we used to have info with each stop about which routes used it. Did this go away at some point?

When provisioning a fresh machine setup_db.sh and the SQL files it runs against the database fails.

The problem seems to stem from the route type definitions being moved to a migration.

Attempting to upload a file with the .txt extension results in the error:

Upload Failed: {"source_file":["Uploaded filename must end in .txt_new"]}

agency_id is an optional field in the agency.txt and routes.txt file, but the GTFS parser throws an exception unless the field is present