jdorn / php-reports

A PHP framework for displaying reports from any data source, including SQL and MongoDB

Php Reports

A reporting framework for managing and displaying nice looking, exportable reports from any data source, including SQL and MongoDB.

Major features include:

• Display a report from any data source that can output tabular data (SQL, MongoDB, PHP, etc.)
• Output reports in HTML, XML, CSV, JSON, or your own custom format
• Add customizable parameters to a report (e.g. start date and end date)
• Add graphs and charts with the Google Data Visualization API
• Supports multiple database environments (e.g. Production, Staging, and Dev)
• Fully extendable and customizable

For installation instructions and documentation, check out http://jdorn.github.io/php-reports/

If you have a question, post on the official forum - http://ost.io/@jdorn/php-reports

Basic Introduction

Reports are organized and grouped in directories. Each report is it's own file.

A report consists of headers containing meta-data (e.g. name and description) and the actual report (SQL queries, javascript, or PHP code).

All reports return rows of data which are then displayed in a sortable/searchable HTML table.

Reports can be exported to a number of formats including CSV, XLS, JSON, and XML.

The Php Reports framework ties together all these different report types, output formats, and meta-data into a consistent interface.

Example Reports

Here's an example SQL report:

-- Products That Cost At Least $X
-- VARIABLE: {"name": "min_price"}

SELECT Name, Price FROM Products WHERE Price > "{{min_price}}"

The set of SQL comments at the top are the report headers. The first row is always the report name.

The VARIABLE header tells the report framework to prompt the user for a value before running the report. Once provided it will be passed into the report body ("{{min_price}}" in this example) and executed.

Here's a MongoDB report:

// List of All Foods
// OPTIONS: {"mongodatabase": "MyDatabase"}
// VARIABLE: {
//   "name": "include_inactive",
//   "display": "Include Inactive?",
//   "type": "select",
//   "options": ["yes","no"]
// }

var query = {'type': 'food'};

if(include_inactive == 'no') {
    query.status = 'active';
}

var result = db.Products.find(query);

printjson(result);

As you can see, the structure is very similar. MongoDB reports use javascript style comments for the headers, but everything else remains the same.

You can populate the 'db' variable by specifying the "mongodatabase" option.

Here's a PHP Report:

<?php
//List of Payment Charges
//This connects to the Stripe Payments api and shows a list of charges
//INCLUDE: /stripe.php
//VARIABLE: {"name": "count", "display": "Number to Display"}

if($count > 100 || $count < 1) throw new Exception("Count must be between 1 and 100");

$charges = Stripe_Charge::all(array("count" => $count));

$rows = array();
foreach($charges as $charge) {
    $rows[] = array(
        'Charge Id'=>$charge->id,
        'Amount'=>number_format($charge->amount/100,2),
        'Date'=>date('Y-m-d',$charge->created)
    );
}

echo json_encode($rows);
?>

Again, the header format is very similar.

The INCLUDE header includes another report within the running one. Below is example content of /stripe.php:

<?php
//Stripe PHP Included Report
//You can have headers here too; even nested INCLUDE headers!
//Some headers will even bubble up to the parent, such as the VARIABLE header

//include the Stripe API client
require_once('lib/Stripe/Stripe.php');

//set the Stripe api key
Stripe::setApiKey("123456");
?>

Hopefully, you can begin to see the power of Php Reports.

For full documentation and information on getting started, check out http://jdorn.github.io/php-reports/

php-reports's Issues

Chart data format bug

Right now, the datatype for a row is determined each row at a time by examining the type of value. There are several cases where this would fail.

Jeremy    10
April         13
John         26

This would product the following chart rows:

["Jeremy",10],
[new Date(...), 13],
["John",26]

We should do a pass over the data first and determine the datatype of each column. If every row is numeric or null, use "number" as the format. If every row is pareseable as a date or is null, use "date" as the format. Otherwise, use "string" as the format.

We have to be careful that "12:00" is not seen as a numeric number. Maybe something like this would work:

<?php
if(is_null($value) || preg_match('/^[0-9,]+(\.[0-9]*)?$/',$value)) {
    //remove any commas from number
    if(strpos($value,',')!==false) $value = str_replace(',','',$value);
}

Add OPTIONS header

The OPTIONS header contains general report options.

They are either flags or name=value format.

The possible values to start are below:

*limit=N - only show first N rows. should probably be done in the actual report instead.
*access=rw - connect to database with a user that has read/write privileges.
*noborder - no table border
*noreport - table hidden (useful if you just want a chart)
*vertical - lays out the table with headers in the first column and rows values in the subsequent columns.

The access=rw option requires restructuring the Database config options slightly. In addition to username and password fields, there should also be username_rw and password_rw fields. If the access option is set, it should use these rw fields instead if available.

Multiple options can be separated by columns or you can just use multiple OPTION headers.

Add inline report table when emailing reports

This should be in addition to attaching a CSV file.

Email This Report button

Add a button to share a report via email.

This requires a new report type that outputs in an email format, including all headers, subject, html version, text version, etc.

The report format will also check for $_GET['email'] and $_GET['message'] to populate the email headers and contents with.

Possible make the email report attach a CSV file with data.

On HTML report format, add an email link that opens up a form to add the person's email and a custom message.

Report Editor

Add an in-browser report editor that lets you create, edit, delete, rename, and move directories and reports in $reportDir.

There will be a manage reports link on the report list page and an edit link on individual reports.

Try to use an existing file browser plugin instead of building a custom one.

Add a ROLLUP Header

The ROLLUP header will display a table footer that rolls up the data in a report. An example is a row that outputs totals for all the numeric columns.

It should have a single param called "columns" which is an object where the keys are the column names and the values are inline Twig template.

Here is an example:

{
    columns: {
        "Order Id": "TOTAL"
        "Revenue": "{{ sum }}",
        "Status": "{% set pending = 0 %}{% set complete = 0 %}{% for value in values %}{% if value.value == 'pending' %}{% pending = pending + 1 %}{% elseif value.value == 'complete' %}{% complete = complete + 1 %}{% endif %}{% endfor %} Pending - {{ pending }}, Complete - {{ complete }}"
    }
}

Any column not specified will be null.

The variables defined in the Twig template are:

• sum - Works with numbers
• mean - Works with numbers or dates
• median - Works with numbers or dates
• count - This doesn't include null or empty rows.
• stdev - Works with numbers only
• min - Works with all data types. For text, strcmp is used.
• max - Works with all data types. For text, strcmp is used.
• values - an array of all the values (including null and empty ones).

This should allow for multiple rollup rows. A common use case may be a Total row and an Average row. Because of this, the Twig variables for each column should be cached and not generated each time.

Report Search

On the list page, have a search box to look for reports based on title, category, and/or description.

When a search is made, filter both the table of contents and the report list to only include matching reports.

Search on keydown, or if it gets too sluggish, on blur.

Instead of just collapsing the non-matching sections, it should actually hide them.

Finish Documentation

Add Documentation for the following:

• Structure of configuration file
• Description of each filter and how to use it
• How to customize templates
• How to make your own report formats
• How to make your own report types
• How to add your own headers

More data types for charts

Google supports "date", "datetime", and "timeofday" data types, but on our end, everything is lumped into "date".

We either need better parsing of values (i.e. don't rely on strtotime) and/or an optional chart parameter for defining data types.

We can use the following rules:

• It is "timeofday" if it matches /^[0-2][0-9]:/
• It is "date" if strtotime returns "00:00:00" for the time
• Otherwise, if strtotime can parse it, it is "datetime"

Better handling of row values

Right now, row values are an array with the following properties:

• raw_value - the original value
• value - the value after filters are applied
• raw - if true, the value contains html
• class - a class to apply to the table cell
• chart_value - a special value to use for charts

I think it would make more sense to have the following:

• original_value - the original value
• filtered_value - the value after filters (not including html)
• html_value - the value after filters (including html)
• chart_value - same as before
• class - same as before
• is_html - if true, use the html_value, otherwise use the filtered_value when displaying

Besides having easier to understand names, this will allow some filters that modify the value but don't add html (like a dateFormat filter) to be used for non-html reports.

Report Pagination

Add 'paginate=#' to OPTIONS header where # is the number of rows per page.

Ability to pass limit and offset into reports in query string.

Cache full report result when limit is specified in query string. When using async html template, generate a random cache key to pass into the async report. The report will store full rows using this cache key.

Paginate links will refresh the table part of the report only, also passing in the generated cache key.

Need new 'table' report type that only displays html table of rows with a "Showing rows X to Y of Z" message at the top.

When pulling rows from cache, don't store report generation time.

Clean up Report Class

• Add variable "header_data" and use that instead of "options" when a header needs to store/retrieve something.
• Move everything else out of options and into their own variables (e.g. "rows", "query", "query_formatted").
• Separate out Info data (e.g. name, description, ...) from the rest of the header data and into its own variable. This is the only part we need to cache and use for the report list. Also every report has a default name/description even if there isn't a header for it.
• Move filter code out of the Report class and into the FilterHeader class.
• Move all other header specific code out of the Report class and into their respective header classes. Add more hooks and callbacks in the Report class where needed.

Variable dropdown options pulled from database table

Support for pulling a list of values for a variable dropdown list from a database table.

VARIABLE: company, Company, Companies.CompanyName

This means use the results from the following query as the options in the drop down list:

SELECT DISTINCT CompanyName FROM Companies

There is an optional WHERE clause too:

VARIABLE: company, Company, Companies.CompanyName, Active=1 AND Status='ENABLED'

This would produce:

SELECT DISTINCT CompanyName FROM Companies WHERE Active=1 AND Status='ENABLED'

In addition, there is an "ALL" modifier at the end that adds the option "ALL" at the top of the list.

VARIABLE: company, Company, Companies.CompanyName, ALL

This is all to support the shorthand version. In JSON, it would be as follows:

VARIABLE: company, {
    "display": "company",
    "values_database": {
        "table": "Companies",
        "column": "CompanyName",
        "where": "Active=1 AND Status='ENABLED'",
        "all"=>true
    }
}

Bug with semicolons in SQL statements

Right now, it splits SQL reports by semicolon, which fails if a semicolon appears in a string.

Finish new documentation

Finish the documentation at http://jdorn.github.com/php-reports and add a link in README.md and the project description.

Also, add screen shots of sample reports in a carousel to the top.

Move environment switcher to top nav

Changing the environment should reload the page. No reason to pass the environment in the variable form.

Move ReportFormat logic into separate classes.

Right now, the logic for outputting different report formats (e.g. html, json, csv) is hard coded in index.php and the PhpReports class. This isn't scalable and can't be customized by the user.

Create a ReportFormatBase class with a single method display.

abstract class ReportFormatBase {
    abstract static function display(Report $report, $request);
}

Also, create classes for HtmlFormat, CsvFormat, JsonFormat, etc.

In index.php, map report/:format to a single displayReport method in PhpReports class that determines the format and sends it off to the right class.

Add DETAIL header

The DETAIL header allows drill-down reports by linking a column of one report to another report, preserving all the macros.

An example is a report that lists the distribution of order sizes. Clicking on the order size would run another report that shows the specific orders of the size.

The format for the header is: column to link (number or name), relative report path, extra macros to set in name=value format. The value in the name=value part can either be a quoted string or a column. If it's a column, that column's value will be used

-- Distribution of Order Sizes (orders/order-size-distribution.sql)
-- DETAIL: 2, orders-with-given-size.sql, size=OrderSize

SELECT OrderSize, COUNT(*) as number FROM OrderSizes GROUP BY OrderSize;

This would make the 'number' column (i.e. the 2nd column) a link to orders/orders-with-a-given-size.sql. The link would contain all of the original report's macros (in this case there are none) along with the macro 'size' set to the value of the OrderSize column for each row.

Multiple name=value declarations can be comma separated. For example size=OrderSize,category="Cars"

Switch all templates over to Twig

Replace all Mustache templates with Twig. Twig is much easier to work with and much more flexible than Mustache, especially when dealing with repeating patterns and recursion.

Consolidate mysql and mongo config

Config should have one 'databases' field that defines all connection info for all report types.

Example:

<?php
return array(
   'databases'=>array(
      'production'=>array(
         'host'=>'www.example.com',
         'mysql'=>array(
            'host'=>'slavedb.example.com',
            'username'=>;'readonly',
            'password'=>'password',
            'host_rw'=>'masterdb.example.com',
            'username_rw'=>'fulluser',
            'password_rw'=>'mypass'
         ),
         'mongo'=>array(
            'host'=>'mongo.example.com:27017'
         )
      ),
      'dev'=>array(
         'host'=>'localhost'
          ...
      )
   )
);

This will simplify the code for switching databases and moving between report types. When you're in 'production' mode, all your reports, whether mysql, mongo, or php will be run on the production environment.

Also, this will potentially allow reports to access other report type's connection info. This would be especially useful for php reports since they could run mysql or mongo queries on the current host.

Add 'bar' filter

The bar filter will replace numeric values with a horizontal bar showing the relative size compared to the other rows.

Bars will be divs with a gradient background.

Report filtering and sorting

This kind of goes along with report pagination (Issue #17).

If we move to a datagrid style HTML report, it will enable us to sort and filter.

We can run the report normally once to create a cached copy and then if there's a sort or filter parameter in the url, we would pull the cached copy and filter or sort the rows in PHP.

The same can be done for pagination.

Make formatted query versions of included reports more standardized

Make MySQL reports and MongoDB reports behave like PHP reports when it comes to formatted queries. They should split out included reports into their own pre tags that are collapsed by default.

Maybe move all of this logic to a method in ReportTypeBase.

Store Recent/Popular Reports

Every time a report is run, add it to a cached recent_reports stack.

Only keep the last X reports (default to 200? configurable?).

Possible Uses:

• Display this data in a recent reports dropdown in the nav bar. It should remove duplicates and only show the last 5-10 reports that were run.
• Display this data in a popular reports dropdown in the nav bar. It should group the list by report and do a count and show the top 5-10 most run reports.
• Use this data to weight search results in favor of popular reports.

Requested URL not found

I followed the install instructions and I am getting a URL not found error. Is there some sort of URL rewriting I need in .htaccess to get this to work? I am able to reach the root php-reports page but clicking on any of the reports results in not found.

Override Report Templates

Ability to override the default report templates.

Add a templates/local/ directory or a local/templates/ directory that will take precedence when rendering templates.

Add this directory to .gitignore

Improve INCLUDE header

In the afterParse method of some headers (VARIABLE, CHART, VALUE, maybe others), import headers from included reports.

For Mongo reports, move entire report file into shell variable and pass as the --eval string. Do the same with PHP reports and eval instead of include statements. This lets us treat them the same was as MySQL reports and standardizes the include process.

Merge Column and Filter Code

Columns and Filters perform the exact same function. They are added to a column and they alter that column's values after the report is run.

An example of a filter is geoip which assumes the value is an ip address and replaces it with city, state country.

An example of a column is link(display) which assumes the value is a url and turns it into an html link with the value as the href and the passed in display value as the link text.

The difference is in how they are defined in the headers. Filters are defined as follows:

FILTER: 2, geoip

This means apply the 'geoip' filter to the 2nd column.

The COLUMN headers is defined like this:

COLUMNS: pre,html,link(display),rpad2

This means map 'pre' to column 1, 'html' to column 2, 'link(display)' to column 3, etc. There is no way currently to only define it for a single column.

To merge the two, I will add a method addFilter($filter, $column, $options) to the Report class. Both the Filter and Column headers will call this method when parsing the header. The Column header will call the addFilter method multiple times, one for each defined column.

I can also get rid of the filterRow method for headers and replace it with a beforeRender method. The only other place the filterRow method is currently used if with charts and that's not actually doing any filtering. a callback like beforeRender would make more sense there.

Update sample reports

Some of the sample reports are still using Mustache instead of Twig.

There should be 3 folders, MySQL Reports, MongoDB Reports, and PHP Reports. At least one of them should have a nested folder. The actual reports should be things like "Best Selling Products" that might represent actual reports.

The sample reports should each demonstrate a different header or style or writing reports.

Add line numbers to formatted SQL and PHP

This should make debugging easier since a lot of the errors reference lines numbers (especially PHP errors).

Improve handling of date variables

Variables declared as type 'date' should be parsed with strtotime and turned into a standard format. This will allow values like "yesterday" and "+3 weeks" in addition to any standard date format to be entered.

There should be a parameter for the VARIABLE header to define what date format should be passed into macros (e.g. "Y-m-d"). It will default to "Y-m-d H:i:s".

The front-end could possibly have a calendar, but that makes the assumption that the dates are close to the current date and that only date and not time are important. It also stops all the parseable dates like "yesterday". To get around this, we can make it optional and only activated when a calendar icon is clicked, not just when the field gains focus.

Replace Mustache templates with PHP

The mustache templating language is too restrictive and requires a lot of restructuring data. Instead, everything will be switched over to use limited PHP instead. The PHP used will be restricted to control statements, loops, and basic formatting functions (e.g. 'htmlentities', 'nl2br', 'ucwords').

For example, instead of doing this:

$rows = array(
    array(
        'key1'=>'value1',
        'key2'=>'value2'
    )
)

We need to structure it like this to use Mustache

$rows = array(
    array(
        'key'=>0,
        'first'=>true,
        'values'=>array(
            array(
                'key'=>'key1',
                'value'=>'value1',
                'first'=>true
            ),
            array(
                'key'=>'key2',
                'value'=>'value2',
                'first'=>false
            )
        )
    )
)

This makes it much harder to debug and makes things much more complicated than they have to be.

Also, if we want to make something uppercase in one place and lowercase in another, we need 2 different macros with Mustache.

Cache Report Data or Headers

At the very least, cache all report headers and options based on the filemtime of the report file. Use the report file, the database, and the macros as the cache key. This should never need to be invalidated since we are basing it on the modification time of the report file. The possible exception is if the config is changed.

Possibly cache result of report as well (i.e. the rows returned). Probably turned off by default and can be activated on a report by report basis with a header. Something like:

CACHE: 3600

This will cache the report output for 1 hour (3600 seconds). Adding a 'nocache' query parameter to the report url will force the report to run, even if it is cached.

Better sorting of report list

Sort the reports and directories by title, not filename.

Also, any directory that doesn't have TITLE.txt in it should be much less prominent that other directories and should float to the bottom of the parent.

Access Control

Ability to restrict access to a specific report or directory.

This may be as simple as a PASSWORD report header or a PASSWORD.txt file in a directory.

It could also incorporate a more complicated user account system with fine grained permissions. It should integrate with existing enterprise systems (LDAP, OAuth, etc.) if possible.

At the very least, there should be the ability to hard code Admin account logins in the config. An admin account would be required to do things like:

• view raw report
• edit report
• view any password protected report

Move all Column Processing to Separate Classes

Just like with the FilterHeader, the ColumnHeader should delegate all processing to different classes for each Column type.

For example, there will be a urlColumn class and a linkColumn class.

The Column classes will all extend ColumnBase, which will have the following structure:

<?php
public class ColumnBase {
    public static function filterValue($key, $value, $options=array()) {
        return $value;
    }
}

The difficult part is figuring out which Column class to use . In order to be backwards compatible with a legacy system, it must support several different formats including:

• rpadN - N is the number of spaces that will be appended to the value. Should map to the rpadColumn class with $options['spaces'] = N
• link(display) - Use the value as the href of a link and use display as the display text. Should map to the linkColumn class with $options['display'] = display
• link - Same as link(display), but with "link" as the display value.
• link_blank(display) - Same as link(display) except target="_blank" is added.
• link_blank - Same as link except target="_blank" is added

The preferred shorthand syntax will be as follows:

//with options
link(display=My Display,blank)

//no options
link

This way, we can easily get the Column class to use and easily construct the $options array. If an option is specified by name only (e.g. "blank" above), $options[name] will be set to true and $options[] will be set to name. For this example, $options = array(0=>"blank", "blank"=>true)

I have to figure out a way to convert the legacy formats to the new format.

link and link_blank are easy, since I can just create a link_blankColumn class that extends the linkColumn class.

The rpadN and lpadN ones are tricky and may just need to be hardcoded to be replaced with rpad(spaces=N) and lpad(spaces=N).

Better ajax loading of reports

Right now, when an Ajax report returns data, it replaces the entire body of the page. It should only replace the report content section and leave the header and variable form section untouched.

Improve styling of HTML reports and the Report List

Improve the look and feel of the report framework.

Break out styles into their own css files under the public/ directory.

MongoDB Support

Reports will be run through the Mongo shell. The report file will be passed in and macros will be added with the --eval flag

mongo  localhost:27017/MyDatabase --quiet --eval "var macro1='value1'; var macro2='value 2';" path/to/report.js

The expected return value is a JSON array of report rows. Each row should be a simple object with column names as the keys and strings as the values. Each row should have the same keys. Example output:

[{
    "column 1": "value 1",
    "column 2": "value 2"
},
{
    "column 1": "another value",
    "column 2": "blah blah blah"
}]

All of the headers for MySQL reports are supported. In addition, there is a 'Database' header that specifies the Mongo database the report should be run against.

Running the report should be as simple as calling the shell command and json decoding the response.

Add INCLUDE header

The INCLUDE header includes a file inside a report. This is useful for multiple reports that share common report code.

Example:

-- This is the included file (cars/includes/includes.sql)
-- VARIABLE: limit, Max Results

SET @product_category = 'cars';

{{#limit}}
SET @limit = {{limit}};
{{/limit}}
{{^limit}}
SET @limit = 100;
{{/limit}}

-- This is the main report (cars/list_of_products.sql)
-- INCLUDE: includes/includes.sql

SELECT * FROM Products WHERE ProductCategory = @product_category LIMIT @limit;

The include path is relative to the current report. The file extension is optional and will default to the file extension of the current report. All headers in the included report will be parsed and added to the existing headers of the current report. It is possible to have nested includes, but the include path is always relative to the current report.

The included report contents will be inserted at the beginning of the current report contents. If more than one INCLUDE header is present, the included report contents will be inserted in the order they appear in the headers.

Header inheritance issue

When using included reports, some headers bubble up to the original report (right not just VARIABLE and VALUE).

The problem is the header only bubbles up if the original report has the same header. The following case will fail:

-- Main report
-- VARIABLE: startdate, Start Date
-- INCLUDE: included_report
...

-- Included report
-- VARIABLE: enddate, End Date
-- VALUE: enddate, today

In this case, the VARIABLE header will bubble up correctly, but the VALUE header will not.

Move charts into their own report format

Make a chart report format that just outputs a chart of the data. The main report would load this chart report in an iframe.

Benefits:

• Caching will be enabled by default for chart reports, so there will only be a small performance hit
• We can take out all the google api code from the main report page
• Charts will work fine in both asynchronous and non-asynchronous modes (broken now in non-asynchronous mode)
• It will be easier to implement an image version of charts that can be exported to a pdf for example

Breadcrumb for report template

Instead of just a Reports Home link at the top of html reports, we should have a full breadcrumb trail.

For example, if the report is in Shopping Cart -> Orders -> My Report

The breadcrumb should be:

<a href='?'>Reports Home</a>
&gt;
<a href='?#report_orders'>Orders</a>
&gt;
<strong>My Report</strong>

Also, javascript on the page should check on page load if there is a hash tag in the url and expand that section so deep links will work.

Get Rid of Submodules

Submodules are messing up the zip and tar downloads from github, which makes it hard to provide download links anywhere.

It's currently using 3 submodules - flight, SqlFormatter, and FileSystemCache. SqlFormatter and FileSystemCache are both single files, so that's not a problem. Flight has a bunch of files, but that shouldn't be too bad either.

Going along with this, we should delete all the tests and documentation from 3rd party included libraries. Php Reports is starting to get bloated.

Support for PHP Reports

A PHP report will look like the following:

<?php
//NAME: This is my report
//DESCRIPTION: This is a description

$rows = array();

...

echo json_encode($rows);

Macro values will be available in the $_REQUEST object.

<?php
//My Report
//VARIABLE: startdate, Start Date

$start_date = $_REQUEST['startdate'];

Like MongoDB, these reports will output JSON.

The report will either be run from the shell or with an include. I haven't decided yet.

Handle exceptions better when displaying reports

Right now, if there is an exception when instantiating a report object (file not found, header parse error, etc.), it falls back to Flight's exception handler that prints out a stack trace.

It should render the exception just like ones that happen when running a report.

Move support for legacy macros into the MysqlReportFormat class.

Right now, the legacy macro format of {varname} and {(varname)} are converted to the proper format in the VariableHeader class. This is only relevant for Mysql reports and should be moved to that class instead.

Make header syntax consistent

All headers should support the format:

HEADERNAME: {
    "param": "value"
}

Headers can also accept one or more non-JSON shortcuts that map to an equivalent JSON declaration. For example:

VARIABLE: varname, Display Name

VARIABLE: {
    "name": "varname",
    "display", "Display Name",
    "type": "text"
}

Headers should change to have the following methods instead of parse:

• init($params) - takes an array of parameters and initializes the header (sets report options, etc.)
• parseShortcut($value) - takes a string and returns an array of params

Documentation and examples should try to use the JSON format whenever possible. To make things easier, the JSON part of headers should be less strict (i.e. allow single quotes, allow not quoting keys, etc.)

In the debug report format, it should output the original headers and the JSON header params. This will make spotting improper shortcut usage easier.

Because we are switching to mostly JSON params, some of the param names should change to be more readable. For example, the VARIABLE header has a param called "values_database" which is hard to remember and tough to tell its purpose just by the name. If we rename it to "database_options", then it becomes more clear that it means it's pulling select options from a database.

Clean up Headers

Create a new INFO header to hold all the meta data about the report.

Deprecate the following headers by moving them to "classes/headers/deprecated". Also trigger a deprecated error when using one of these headers.

• Column (duplicate of Columns)
• Options (duplicate of Options)
• Plot (duplicate of Chart)
• Total (duplicate of Totals)
• Totals (replaced with Rollup)
• Columns (just a wrapper for Filter)
• Value (just a wrapper for Varible -> default)
• Detail (merge into Filter)
• Mongodatabase (merge into Options)
• Cache (merge into Options)
• Database (merge into Options)
• Status (merge into Info)
• Caution (merge into Info)
• Name (merge into Info)
• Description (merge into Info)
• Note (merge into Info)
• Created (merge into Info)

This should leave us with the following 7 headers:

• Chart - Add a chart/graph to a report
• Filter - Alter the values and/or the formatting of a column
• Include - Include another report
• Info - Meta data about the report (name, description, create date, etc.)
• Options - Various options that change how the report runs
• Rollup - Add rows to the report footer
• Variable - Parameterize the report

This should make things much cleaner and easier to document and explain.

Report Caching Possibly Not Working

I need to dig into this further, but it seems like report results are not being cached some of the time. It may just be a file permissions problem.

It should cache the result of every report for 10 minutes. By default, the CSV report type (and maybe JSON) should pull from cache if available. If a CACHE report header is specified, every report type (including HTML) should also pull from cache if available.