keen / explorer

Data Explorer by Keen - point-and-click interface for analyzing and visualizing event data.

Keen

The Complete Event Data Management Solution

This project contains mono-repository for Keen.io Front-End projects and applications.

Documentation

All documentation, including charts, widgets and dataviz can be found at Storybook

Contribution guide

This is an open source project and we love involvement from the community! Hit us up with pull requests and issues.

Examples

Example dashboards build with keen.io visualization libraries.

Smart Home Monitoring

Smart Home Dashboard

Marketing performance dashboard

Marketing Dashboard

Packages

@keen.io/charts

Set of reusable plots components built with React and D3.

@keen.io/colors

Color palette used by Keen.

@keen.io/dataviz

The DataViz is abstraction layer that connects Keen Analytics API with visualization components. It allows to seamlessly embed and deliver metrics within your user interface with minimal effort.

@keen.io/embedded-registration

Embeddable registration component used by keen.io for HubSpot integration.

@keen.io/forms

Set of form helpers used in keen.io projects.

@keen.io/icons

Set of icons used in keen.io projects.

@keen.io/parser

Set of parsers that are used internally to transform Keen API responses into @keen.io/charts friendly format.

@keen.io/pubsub

Implementation of PubSub messaging service.

@keen.io/react-hooks

Custom hooks used in keen.io projects.

@keen.io/storybook-utils

Helper functions used internally in Keen Storybook.

@keen.io/toast-notifications

keen.io wrapper for react-toast-notifications package.

@keen.io/tracking-lite

Light and simple version on Keen Tracking library - used for integration with Google Tag Manager.

@keen.io/ui-core

Set of reusable visual React components used in keen.io Front-End projects.

@keen.io/widgets

Higher visualization layer that connects @keen.io/charts with other React components that could be used to interact with charts plots.

npm scripts

List of useful commands that could be used by developers. Execution in the command-line interface should be prefixed with yarn package manager.

Command	Description
storybook	run storybook for development on port 3000
build-storybook	build storybook production distribution.
lint	run linter against current application codebase.
test	run unit tests.
test:cov	run unit tests with coverage reporter.
prettier	run code formatter process against current codebase.

commit

This project uses Conventional Commits to enforce common commit standards.

Command	Description
npx git-cz	run commit command line interface.

build packages

Command	Description
npx lerna run build	run build script from package.json for each package

add dependency to package

Command	Description
npx lerna add <name>[@ver] packages/<name> [--dev]	Add dependency to specific package in mono-repository.

updating dependencies

Command	Description
update-packages	run update wizard command line interface.

deployments

The all commits pushed into master branch will be picked by CircleCI workflow that perform npm packages version and publish.

license

MIT

Copyright (c) 2019-2020 Keen.io.

explorer's Issues

The CI setup looks broken

installing deps is failing: https://circleci.com/gh/keen/explorer/327

"Can't get my saved query to work" (explorer re-named it in all lowercase)

I want to setup a dashboard that uses a saved query. So, I saved a new one in the Explorer and now I'm following the js-guide for how to embed it in my dashboard. However, I keep getting an error that the resource does not exist!? I'm being super careful, double-checking everything, even copy & pasting the name of the saved query from the explorer so that I get it exactly right. I'm trying every API key and variation on a saved query URL that I find in the API reference. What am I doing wrong!? Do saved queries even work!? Is this API reference incorrect? What is the deal here!? Finally, I'm inspecting the explorer to try to see what network request is actually being sent when it loads the chart.... ahh.....

After a lottttt of troubleshooting by no less than 3 Keen folks, we finally realized that saved query URL is case sensitive, and that the explorer had named the query in all lowercase.

Steps to repro:

1. In the explorer, create a saved query and name it "myTshirtRequests"
2. Now try to embed a saved query with the name "myTshirtRequests"
   (it only works if you rename it to "mytshirtrequests")

Turns out there is a lower-case version of this query displayed here, but it's not clear how/why that is there or that I should use that when embedding saved queries.

Optional Steps in Funnel Query are "forgotten" by Saved Queries in the Explorer

"I'm creating a three-step funnel. The second and third step I check the "Optional Step" flag and then save the query, but when it saves it clears the optional flags. So the next time I run it I have to remember to go in and check those flags again."

Repeatable steps:

• step 1: access any saved funnel query
• step 2: check optional for step 3, and then click 'update' to save
• step 3: immediately the optional box becomes unchecked
• step 4: returning to the saved query reveals the box is still unchecked

Ideal outcome: Ideally the flag stays after saving a query from the Explorer!

Filters on saved queries with property_value strings can be interpreted as dates

If a filter's property_value can be parsed into a valid Date object, that filter value will get handled as such. Eg: "2015" becomes "2015-11-19T00:00:00". This occurs prior to saving a given query, meaning the saved query parameters are also incorrect.

1. Create a filter for a property where operator equals a string value of 2015
2. Close filter modal
3. Name and save the query
4. Reload the query and re-open the filter modal
5. That filter's property value is now "2015-11-19T00:00:00".

Timeframe text description does not pluralize to match dropdown

When running a query with a timeframe, you can change the unit amongst minutes/hours/days/etc. You can also change the number of units. When I change the number of units to be > 1, the description updates.

Actual: The last 3 day including the current day.
Expected: The last 3 days including the current day.

Note: I took this screenshot on the production version running on keen.io.

(I recognize this is an exceedingly tiny issue, but still wanted to document it for completeness.)

Relative Reference of rest.js in client > js > app > modules > persistence > persistence.js

Was setting up the keen data explorer using the instructions in readme and encountered an issue while serving the project with gulp.. Below is the trace

[00:38:08] Using gulpfile /Users/shabeermothi/projects/keen-data-explorer/explorer/gulpfile.js [00:38:08] Starting 'build-scripts'... [00:38:08] Starting 'images'... [00:38:08] Starting 'fonts'... [00:38:08] Starting 'build-styles'... [00:38:08] Starting 'connect'... [00:38:08] Server started http://localhost:8082 [00:38:08] Finished 'connect' after 238 ms [00:38:08] Starting 'watch'... [00:38:08] Finished 'watch' after 135 ms Autoprefixer's process() method is deprecated and will removed in next major release. Use postcss([autoprefixer]).process() instead [00:38:08] Finished 'build-styles' after 820 ms [00:38:08] Finished 'images' after 834 ms [00:38:10] Finished 'fonts' after 2.2 s events.js:85 throw er; // Unhandled 'error' event ^ Error: module "./rest.js" not found from "/Users/shabeermothi/projects/keen-data-explorer/explorer/client/js/app/modules/persistence/persistence.js" at notFound (/Users/shabeermothi/projects/keen-data-explorer/explorer/node_modules/browserify/index.js:812:15) at /Users/shabeermothi/projects/keen-data-explorer/explorer/node_modules/browserify/index.js:762:23 at /Users/shabeermothi/projects/keen-data-explorer/explorer/node_modules/browserify/node_modules/browser-resolve/index.js:250:24 at /Users/shabeermothi/projects/keen-data-explorer/explorer/node_modules/browserify/node_modules/browser-resolve/node_modules/resolve/lib/async.js:55:18 at load (/Users/shabeermothi/projects/keen-data-explorer/explorer/node_modules/browserify/node_modules/browser-resolve/node_modules/resolve/lib/async.js:69:43) at onex (/Users/shabeermothi/projects/keen-data-explorer/explorer/node_modules/browserify/node_modules/browser-resolve/node_modules/resolve/lib/async.js:92:31) at /Users/shabeermothi/projects/keen-data-explorer/explorer/node_modules/browserify/node_modules/browser-resolve/node_modules/resolve/lib/async.js:22:47 at FSReqWrap.oncomplete (fs.js:95:15)

Found the rest.js module is required from persistence.js (client > js > app > modules > persistence > persistence.js) as a reference from the same path whereas it should be a folder up in the hierarchy.

Current reference in persistence.js
REST: require('./rest.js')

Actual reference in persistence.js which would make it work
REST: require('../rest.js')

The data explorer is cool btw 👍 and the readme is very much descriptive 👍

Save and Update buttons no longer toggle to "Saving..." and "Updating..." when a request is in flight

If you save or update a query, the save and update buttons no longer switch to "Saving..." and "Updating..." - this makes for a laggy/weird experience when interacting with the save operations.

Explorer is meant to do this and it was working before, so this is a bug not an enhancement.

Repro steps:
Just save or update an query and check out the buttons + button text.

Invalid time JS error in funnel analysis

When performing a funnel analysis it is not possible to select an actor property due to an Uncaught Error: Invalid time value (see screenshot).

Table view is missing

As a user, I want to be able to easily copy & paste the results of my aggregations for use in excel, google sheets, keynote, etc.

I used to use the "table" visualization type for this, but recently I noticed it is missing from the explorer.

Local explorer models do not update when you update a saved query

It seems that when you try to update an attribute of a saved query, the update gets sent to the API, but the local explorer model does not get updated.

This propagates in the UI as hitting the "Update" button seems to revert your changes. However, upon refresh, your changes will be there.

getChartTypeOptions is CPU intensive

The getChartTypeOptions function in ExplorerUtils initializes a new keen.js Dataviz object, passes it data which has to get parsed (sometimes quite large query responses) and then gets the array of supported chart types given the query and data. Having Dataviz parse the query response each time this function gets run is very CPU intensive. This should be changed.

Possible solution:
Keep our own map of which chart types are supported for the given query without needing to initialize a new Dataviz object.

Timeframes added twice bug, query returns but results are incorrect.

There is a bug with absolute timeframes which creates a working query with results that are somewhat "off". It looks like the explorer is adding two timeframe parameters to the API query which may or may not be related to the problem. This one may be easier to show in a screenshare with a specific query that I know it affects.

Steps to repro:

1. Create a query with absolute timeframes, for example Nov 1 2015 to Dec 1 2015
2. Run the query and it looks good.
3. Copy & paste the API query URL from bottom left.
4. Now change the timeframe from Dec 1 2015 to Jan 1 2016
5. Run the query again and it may not look like you expect.
6. Copy & paste the API query URL from the bottom right of explorer. Notice that this query has two timeframes.
7. If you press Run Query yet again, it corrects the issue.

example generated query URL which has two timeframes:

https://api.keen.io/3.0/projects/PROJECTID/queries/maximum?api_key=KEY&event_collection=Stripe_Events&target_property=data.object.amount&group_by=%5B%22data.object.receipt_email%22%2C%22data.object.customer%22%5D&timeframe%5Bstart%5D=2015-12-01T00%3A00%3A00.000%2B00%3A00&timeframe%5Bend%5D=2016-01-01T00%3A00%3A00.000%2B00%3A00&timeframe=%7B%22start%22%3A%222015-12-01T00%3A00%3A00.000%2B00%3A00%22%2C%22end%22%3A%222016-01-01T00%3A00%3A00.000%2B00%3A00%22%7D&filters=%5B%7B%22property_name%22%3A%22type%22%2C%22operator%22%3A%22eq%22%2C%22property_value%22%3A%22charge.succeeded%22%7D%5D

Disclaimer: I'm not 100% the above steps will reproduce this issue, or if my issue is related to the timeframe being duplicated in the query URL. But there is definitely something fishy going on!

Remove internet connection requirement for running tests

Tests make use of CDN hosted versions of jQuery and Keen.js. Ideally, you shouldn't need an internet connection to run tests.

Geo filter "radius in miles" input doesn't work

Can't seem to update the name of my query

It says that it updated my query, but why hasn't the name changed, then?

queries do not run when clicked on http://keen.github.io/explorer

The queries do not automatically run when they're clicked.

http://keen.github.io/explorer/?query%5Bevent_collection%5D=page_load&query%5Banalysis_type%5D=count&query%5Btimezone%5D=UTC&query%5Btimeframe%5D=this_1_weeks&visualization%5Bchart_type%5D=metric#demo

Analyses with only 1 group-by field return results with an "undefined" label attached to each group

For all analyses which only have 1 group-by field, the returned result now shows up with the label "undefined" after every field value.

By adding a 2nd group-by, the "undefined" field is replaced by the property category of the 2nd group-by.

Error reported through Keen Support Pool & the error is reproducible.

Feature request - batched extracts

Not sure which repo to raise this in, so raising it here.

I need to extract all events of a certain type for the last 6 months. Doing this as a single extraction fails as I guess keen can't handle it, so I have selected some date ranges and eventually successfully extracted all the data I need like that.

However this is also far from ideal. It's meant a lot of trial and error and tedious form manipulation in order to find date ranges that are small enough to be exported. Once exported I also have the problem that some of the exports are so large that they cannot be converted into json as they exceed the memory and/or language limitations of javascript and my laptop.

What I really need is a proper batched extract - I send a request for a certain collection of data, but am able to stipulate the maximum size of a single extract, and keen batches up the data into files no bigger than these limits (specified in MB or row count) and sends them to me.

Is this a feature you'd be open to developing?

Can't group by more than one property. "Uncaught TypeError"

When I try to run a query grouped by two properties, an error occurs and the visualization window is blank. In console, I can see the error: Uncaught TypeError: Cannot read property 'initialize' of undefined

Steps to repro:

1. Create a small query like "count" pageViews this_1_day, group_by "ip_geo_info.country"
   This query executes quickly & successfully.
2. Now add a second group_by property like "ip_address" and run the query again. No visualization or error appears, the visualization area is empty. In console, you can see the error Uncaught TypeError: Cannot read property 'initialize' of undefined

Steps to repro:

Notes:

1. Once this error occurs, you can't correct the query by deleting the second group_by. You're kind of "stuck" once you get this error.
2. The API query URL produced by the explorer works great and the query returns in seconds.
3. This issue is not related to having too many groups as the example I tested only had 11 unique groups in the double-group-by.

Link to demo in README.md points to http://localhost:8081/explorer

I'm guessing there is a hosted instance of this somewhere you intended to point to. If not no big deal.

Can't update saved query

As a user of Keen's explorer, I want to be able to open a saved query, modify it, and then save my changes.

When updating & renaming my saved query, I got the following error:

"Problem updating: The specified value is invalid for the specified property for this cached query. Property: target_property. Value: None"

The error doesn't really make sense since the query runs fine and I have a target property specified.

I don't have time to confirm the repro on another saved query, but the steps I took were something like this:

1. Open a saved query.
2. Modify the timeframe & add an interval.
3. Change the query type from count to count_unique & add a target property.
4. Run the query and results look good.
5. Rename the query.
6. Click update.
7. Get the error "Problem updating: The specified value is invalid for the specified property for this cached query. Property: target_property. Value: None"

Remove incorrect query attributes when the analysis type changes

When the analysis type is changed from count_unique to count, the target_property does not get cleared from the query and remains both in the URL and in the data that gets sent to Keen when trying to save a query.

Saving a query breaks explorer: Cannot read property 'length' of null

I tried to create a simple query for testing purposes. Saving it seems to break the explorer.

Steps to repro:

1. Go to the project.
2. Select a count for the collection "data"
3. Enter timeframe "this 14 years"
4. Run Query.
5. Enter title "Simple Query 2"
6. Click Save query.

Result => Error in console: "cannot read property 'length' of null".
The explorer sort of breaks and I can't even create new queries.

I will send you the link to try it out yourself.

I clicked the green button and got stuck

If you build a query and accidentally click the green button "save query" instead of "run query", you get an error about how you must name the saved query. This unfortunately is really easy to do because Save Query is the rightmost button and it's green. So instinctively I want to press it more than the blue "run query" on the left.

Unfortunately, now you are completely stuck because it won't let you click "run query" after that error appears.

You have to reload the explorer :(

Steps to repro:

1. Create a query like count last_14_days
2. Click save query instead of run query.
3. Get error.
4. Be stuck.

Schema preview gets cut off

Here's an example:

_.cloneDeep is very CPU intensive

There are quite a number of _.cloneDeep (lodash function) calls in Explorer. While I understand this approach is used rather than modifying underlying properties of store objects, the cloning of entire models is very CPU intensive, particularly for queries with large data sets. This function is the most CPU intensive piece of code running in Explorer by quite a bit based on my last CPU profiling.

There should be a new approach used to handle updating models and _.cloneDeep should ideally be used sparingly or not at all.

npm install fails on mocha-phantomjs

npm ERR! peerinvalid The package phantomjs does not satisfy its siblings' peerDependencies requirements!
npm ERR! peerinvalid Peer [email protected] wants [email protected] - 1.9.7-15

npm ERR! System Linux 3.13.0-58-generic
npm ERR! command "/usr/local/bin/node" "/usr/local/bin/npm" "install"
npm ERR! cwd /home/ubuntu/dash
npm ERR! node -v v0.10.25
npm ERR! npm -v 1.3.24
npm ERR! code EPEERINVALID
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /home/ubuntu/dash/npm-debug.log
npm ERR! not ok code 0

Filters - 'not in'

I'm working to exclude a small number of outliers from a dataset.

The most preferable way to do this would be a a 'not in' filter against an array of values.

The documentation is a little ambiguous on whether this is possible;
'in: Whether or not the property value is in a given set of values. The value passed in must be a JSON array of values. Example: [1,2,4,5].'

Would be great to have this as an enhancement.

Queries do not automatically run on pageload

It used to be the case that queries would automatically run if you opened an Explorer page that had query attributes in the URL. This appears to have stopped happening.

The fix for this is likely simple, but I'm also going to see if I can get a unit test in for this. That may be more involved, since this happens on app initialization.

Internal Ticket: https://trello.com/c/Kg89AbeM/123-explorer-not-auto-loading-queries-from-url-on-pageload

"Interval" doesn't clear properly and makes it so you can't save funnel queries.

At least 2-3 times today I've had a interval of "minutely" mysteriously appear in the explorer. One time it was when updating a saved query, it changed from monthly to minutely.

Another time, it was after I clicked "new query" that the new query started with a default interval of "minutely".

Another time, it appeared after I clicked "clear".

Sorry I don't have more details yet!

Boolean filter values handled as strings

Filter values of true|false fail internal validation, resulting in an error message of:

The specified value is invalid for the given operator. Operator: exists. Value: false

Rationale for self-hosting this project

What scenarios would I want to self-host this explorer? Is it recommended to do that in general, or to use your hosted version?

Using Arrayed Query Results as Filters in Related Queries

Thing that would be awesome: If you could create explorer queries using the results of other queries as filters.

Something that would look like this:

http://content.screencast.com/users/tannerjjones/folders/Jing/media/30678b98-f954-4c75-89c8-d4b8ffa1a129/00000138.png

Table view is wonky

When I switch from JSON to table view, part of my table is cropped. I have to keep scrolling left and right to see the values, even though there is plenty of space for the table to be bigger.

It makes it really difficult to copy & paste the table.

Autoprefixer's process() method is deprecated

With the release of Autoprefixer 5.2.0 you get the following warning after running gulp:

Autoprefixer's process() method is deprecated and will removed in next major release. Use postcss([autoprefixer]).process() instead

“group_by” param in the API Query URL does not group

There is an issue w/ the “group_by” param in the API Query URL that you generate in the Explorer that, in effect, does not group the query results :

api.keen.io/3.0/projects/project/queries/count?api_key=key&timezone=US%2FEastern&timeframe=this_30_days&group_by%5B0%5D=app.name&event_collection=Viewed%20photo&filters=%5B%5D

I bolded the text — but here is what it does: &group_by%5B0%5D=

Realized this an hour into futzing with it and trying to figure out why I couldn’t import the JSON into a Google Doc. Removing that invalid character fixed it immediately.

Am giving you a heads up, since I assume this is a widespread issue and not unique to our account.

Capitalization of rest.js still throws error

I also found a similar error to #5

Error: module "./rest.js" not found from "./explorer/client/js/app/modules/persistence/persistence.js"
As per in the reported bug, I lowercased REST.js to rest.js and problem solved, but I'm just reporting that the bug still exists in a post-fix HEAD.

I'm running Ubuntu 14.10 node v0.12.7

explorer doesn't properly handle queries with result: 0

Queries which return the API response of {result: 0} are incorrectly displayed in the explorer as "Your query returned no results".

That might make sense on some queries e.g. Count the number of events where myproperty = "feather monkey brain", but it doesn't make sense on mathematical operations where the result is 0.

For example, a user is running a percentile query on a dataset where the value is often zero. For percentiles less than 38, they should get "0" as the result. Instead they get "Your query returned no results".

Another point is that there is no way to toggle to see the JSON response from the query. If you could, it would be easier troubleshoot and see the result of 0.

Update hosted assets for multiple group-by support

Currently the linked assets here https://github.com/keen/explorer#option-1-use-the-hosted-assets-from-our-cdn point to an old version of keen.js and keen-explorer.js. Can these be updated?

Explorer ought to display the timezone of a time displayed in charts or tables

Explorer displays dates without timezone information in chart and table views so when you mouse over a point in a chart you get something like "Nov 30, 2015 4:00PM" with no information about what timezone that is being used to interpret that date and time. It would be quite helpful if the date and time was displayed with the current timezone in use in the chart or table, e.g. "Nov 30, 2015 4:00PM (PST)" for Pacific Standard Time.

Datetime filters do not work (shows invalid date)

If you try to create a Datetime filter, the inputs show "invalid date" and you cannot edit them.

When you delete a saved query, it prevents you from opening other saved queries.

To repro:

1. Create a saved query.
2. Delete it.
3. Try to select another saved query - nothing happens you click it. There is an error in the console.

Feature request: share reports with non-Keen users

Hey folks,

Sometimes, we'd like to share reports with our users and/or investors who'd like to get an update on our metrics.

Currently, we take screenshots of the charts and send them over – but it would be lovely to somehow enable the ability to send a unique link that gives the user access to a specific query/saved-query.

This would be relatively simple and possible if the read key was embedded in the query and the explorer used it to perform its query (and the parameters of the query would of course have to passed as well).

Issue with renaming a saved query

I tried to rename my query, but when I did the query name flipped back to the old name, and no visualization was shown.

I think my steps were like this:

1. Load a saved count query
2. Modify the timeframe from "previous month" to an absolute timeframe with a monthly interval.
3. Run the query and it looks good.
4. Rename the query from "New paying customers this month" to "New paying customers monthly"
5. Press update.
6. Notice that the name reverted back to "New paying customers this month".
7. Notice that the graph is now missing when I run the query, and there is an error.
8. Notice that the interval changed to "minutely".
9. Notice that when I run the old saved query URL, "https://keen.io/project/PROJECT/explorer?saved_query=New-Paying-Customers-This-Month", it says "could not be found".

Sorry this isn't a very good bug report. It will take some time to nail down exactly what the problem is.

Grammar: "There was a problem: Choose an Target Property."

it should say "a" instead of "an"

Can't load saved queries

Step to reproduce:

1. Select a saved query

Error when sharing explorer link with absolute timeframe

Explorer URL with absolute timeframe gives an error "value was not able to be parsed as DateTime" and datepicker shows "Invalid Date".

Steps to repro:

1. Create a count query with date range March 1st to March 8th and run it (works fine)
2. Copy explorer link and paste it in a new tab (should also work fine)

Here is my explorer URL which didn't work:
https://keen.io/project//explorer?query%5Bevent_collection%5D=create_organization&query%5Banalysis_type%5D=count&query%5Bgroup_by%5D%5B0%5D=organization.first_internal_referrer.full&query%5Bfilters%5D%5B0%5D%5Bproperty_name%5D=organization.first_external_referrer.full&query%5Bfilters%5D%5B0%5D%5Boperator%5D=exists&query%5Bfilters%5D%5B0%5D%5Bproperty_value%5D=false&query%5Btimeframe%5D%5Bstart%5D=2016-03-01T00%3A00%3A00.000-08%3A00&query%5Btimeframe%5D%5Bend%5D=2016-03-08T00%3A00%3A00.000-08%3A00

Allow filters to equate to an empty string.

Right now, you can't create a filter to check for the equalness or non-equalness of an empty string (it just removes the filter). There are cases where you would want to check for an empty string or when that field is not empty.

I'd love a switch to get some larger chart sizes

The Explorer is super useful. I'd really like a way to get some graphs in larger sizes, for sharing in reports and making it easier to decipher crowded charts.

Feature request: export to CSV from Table view

Hey folks,

I think it's an obvious extension to the table view of the data to have an Download as CSV button of some sort.

This will make it easy to do further analysis in Excel/Spreadsheets outside of Keen (for example, when you want to perform rate of change and/or accumulation).