city-of-bloomington / dashboard

Live display of key metrics for city performance

City Performance Dashboard

A performance dashboard that pulls data from CKAN data portals to show key metrics about data sets.

The dashboard gathers, logs, and displays metric values from data sources throughout the city. A metric is some scalar value that reflects something useful about city performance.

We have an instance of this application available here: https://bloomington.in.gov/performance

See also: http://blog.strom.com/wp/?p=4755

Features

Internationalization and Localization ready All words and language used are already pulled out into .po files. Continue using the .po files for all language, and you'll be ready to attract translators for your international audience!

Multi-tenant Host multiple sites using the same codebase. Each site's data, themes, sessions, and other code implementations are contained in their own data directory. The path to the data directory can be set in the apache config for each site. Upgrading the main codebase is simple, with minimal worries about destrorying someone's custom theme.

Themable Speaking of themes. All look and feel is provided via drop-in themes. Any core template or block can be overridden by providing a matching file in your theme directory.

Install

Ansible scripts are available to assist with setting up a new instance. These also provide a description of requirements, in case you need to install using a different mechanism:

Install Documentation

dashboard's Issues

Error when cardLog has no data

When a card does not have any entries in the log, the dashboard and the card log view error out

Error: Call to a member function getStatus() on null in /home/inghamn/Documents/Projects/dashboard/blocks/html/cards/card.inc on line 16

Departmental Report on Staff Performance

HAND is looking for a report that would look a lot like the main report table but be broken down with all HAND staff in uReport to show...

Avg Days Open, Open, Opened Today, Opened This Week, Opened This Month, Closed Today, Closed This Week, Closed This Month

Rosie Beaman would be the point person on this.

A way to manually specify the order of cards

After a new card has been created, we'll want a way to specify an order or position or priority for the card in relation to other cards.

Cache the GoogleService data request

We're making a seperate data request for each calculation of the metric. This is very chatty with Google and results in throttling and blocking from Google.

We should look into downloading and caching the dataset, so we only make one call to Google for a run of the metric log population.

Error while adding new user

After logging in and choosing Admin -> User Accounts (https://bloomington.in.gov/performance/users), I receive the following error message:

Fatal error: Uncaught TypeError: Argument 2 passed to Application\Models\PeopleTable::find() must be of the type array, null given, called in /srv/sites/dashboard/Controllers/UsersController.php on line 19 and defined in /srv/sites/dashboard/Models/PeopleTable.php:24 Stack trace: #0 /srv/sites/dashboard/Controllers/UsersController.php(19): Application\Models\PeopleTable->find(Array, NULL, 20, 1) #1 /srv/sites/dashboard/public/index.php(35): Application\Controllers\UsersController->index(Array) #2 {main} thrown in /srv/sites/dashboard/Models/PeopleTable.php on line 24

Sparklines should be dark color for print CSS

right now they won't show up when printed

Count no longer supports passing null

PHP 7.2 does not allow passing null to count

Warning: count(): Parameter must be an array or an object that implements Countable

Update licensing declaration to match Github style

Github expects the GPL license to be in a file named LICENSE at the root of the project.

We need to rename our license file, and update the license declaration in all our file headers.

Card listing should show status

We implemented card status ( #37 ) but only show the status when viewing a single card. It would be useful to see the status on the full list of cards. To make room on the list, we can probably stop displaying the service method (the service method should be enough info on the list).

CRON script is removing site_config

The cron script has an error. It is doing the purge of old tarballs from the wrong directory. It is instead deleting files from SITE_HOME.

Implement Zabbix service

We've switched from Nagios to Zabbix for our infrastructure monitoring. So, now we need to replace the metric cards with new Zabbix based cards.

Group selection only works on homepage

The links generated to select a group only work on the homepage of the dashboard. We should either remove them from the rest of the screens or figure out how they should work when you're navigating the rest of the site.

Allow for internal cards (not shown to the public)

There are things we want to track that are only relevant to city staff. Rather than install a whole seperate system, it would be nice to keep track of both internal and external metrics in a single application.

We could set up card types, such as "public" and "internal"

Add a metric for CKAN resources updated daily

We want to calculate the percentage of CKAN datasets that get fresh data every night.

What percentage should we shoot for?
Nightly's probably not right for all datasets. We might want to set a keyword tag that describes the expected timeframe for dataset updates. The metric would need to take the expected timeframe into account.

Render sparklines for 90 days of data

The current 10 day sparklines don't go back far enough to show any interesting trends

Implement a card status

We have cards that we are starting to migrate out to new metrics. However we don't want to delete the log data for the card. We need to create a status field for cards, and make sure queries to get the list of cards respect the status.

We'll need to take status into account when:

• displaying public cards
• updating card values

Card detail page design

After clicking on a specific card on the dashboard, the card detail page is displayed. We need to consider the best way to present additional details about the metric to the user.

Example:
https://bloomington.in.gov/performance/cards/2

PHP 7.2 interface mismatch

Warning: Declaration of Application\Models\CardsTable::find(?array $fields = NULL, array $order = Array, ?int $itemsPerPage = NULL, ?int $currentPage = NULL) should be compatible with Blossom\Classes\TableGateway::find(?array $fields = NULL, ?array $order = NULL, ?int $itemsPerPage = NULL, ?int $currentPage = NULL) in CardsTable.php on line 10

Log detailed errors during CRON

The CRON job is currently dumping output to a log file. However, each card is only writing a success message or ERROR on each line.

We should pass detailed error messages all the way through, so they can be written out to the cron log file.

Card parameters do not throw errors for invalid JSON

We need to add a validation check. User-supplied parameters need to be valid JSON

Allow targets to be floating points

So far, we've only been using integer targets. However we need to be able to express floating point targets; especially since so many of our metics involve averages over time.

Cards cannot have a target of zero

Putting a target of zero (0) is getting treated as null.

Render sparklines as SVG on the server side

This would allow them to be displayed responsively on the client

Internal card indicators

WE need a menu/tab for internal cards (like "utiliites", "public safety") in the menu bar when staff is logged in

Cards also need an indicator when they are internal

Ansible does not install site_config.inc

Unify the daily cron scripts

Do backups daily and we do daily updates of data for the dashboard.

We should unify these into a single daily cron script. Also, it should support the new nabbix monitoring system. (It should write to /var/log/cron/dashboard )

Ckan service doesn't handle spaces in parameter values

We're trying to query on tags that have spaces in their names. We need to make sure the URLs are being built correctly.

Cannot create AD user accounts

User account generation is still erroring out, validation is happening before we are reading data from LDAP. We need to defer Person validation until after the user data is loaded from LDAP.

Ability to purge logs for a card

We should provide a way to purge a card's log data.

This can happen when we redefine how the metric is calculated, and we need to clear out the old, invalid metric calculations.

populateCardValues should accept a date range

Currently it is coded to always start from the current date and work back in time. For generating slices of data very far back in time, the script needs to work through all the days in between.

We should, instead, pass in start and end date as a range, and have the script populate values for just the days inside the range.

Cards need to know units

Each card is asking for a number value. When formatting the card for display, we need to know what the units are.

Groups icon implementation broken

The way we've implemented hosting group icons is interfering with the mod_rewrite rules. We have a groups controller that is not being hit, because Apache sees the /public/groups directory.

We should probably rename the groups directory that we're hosting the icons from.

Internal cards do not get updated nightly

The internal cards are not included in the CRON script. This is because of the hard coded permission check in the CardsTable(). CardsTable will only return public cards if you are not logged in. However, the CRON script has no way to log in, and can never see the internal cards, in order to update them with new metric values.

Period should not be required for a card

Different metric types do not necessarily need a range of time. For instance, metrics that are simple counts of something only involve a single point in time.

We should make it so that the time period is not a required field for cards. The period field should still be passed to the Service methods, if it's provided.

Way to display a group of cards

Cards can be assigned to a group (indicated with an icon).
We don't have a way to show all cards belonging to a group (and only those cards).
This issue needs a design for what the interface should look like.

Create a Nagios service type

For ITS, we want to show uptime for various services. We use Nagios for monitoring, which provides a JSON api.

We need to write support for a Nagios service, so we can create cards that show stuff like uptime and performance metrics.

Nightly Cron not working

There's supposed to be a nightly cron job that generates the prior day's metric. It doesn't seem to be working. I've had to generate them by hand for now.

Distinguish between zeros and lack of data

As we've started adding cards, we now have metrics where zero is an actual data point. So far, test for existence of data is a simple truthy/falsy check. This is not enough!

We must expect services to return actual null values in responses in cases of no data. This is very different from zeros in the resonses, as zero can be a valid data point. Status checks must test whether the response is null vs. zero when making a status determination.