berkmancenter / dotplot

Telling a story through dots

Dotplot - Telling a story through dots

DotPlot is a data visualisation tool built on top of D3.js to help non-programmers create interactive data visualisations using easy to use drag and drop interface.

Create View (Create or Edit DotPlot Projects)

Presentation View (Share or Present DotPlot Projects)

Prerequisites

To contribute to this project, you will need the following things properly installed on your computer.

• Git
• Node.js (with NPM)
• Bower
• Ember CLI
• PhantomJS

Installation (Frontend)

• git clone https://github.com/berkmancenter/dotplot.git this repository
• change into the new directory
• npm install
• bower install

Running / Development (Frontend)

• ember server
• Visit your app at http://localhost:4200/create.
• Public View: http://localhost:4200/project?id=<Your Project Id>.

Installation (API Server)

• Change into the API directory cd api/
• npm install

Running / Development (API Server)

• node index.js
• Make HTTP requests at http://locahost:3000/api/<API Endpoint>

Code Generators

Make use of the many generators for code, try ember help generate for more details

Running Tests

• ember test
• ember test --server

Building

• ember build (development)
• ember build --environment production (production)

DotPlot Guide

Further Reading / Useful Links

• ember.js
• ember-cli
• Development Browser Extensions
  • ember inspector for chrome
  • ember inspector for firefox

Ember Actions Help

loadProject (Function, Observer)

Accepts: None
Use: Calls importJSONData if a valid Project ID is provided.

labelToggle (Function, Observer)

Accepts: None
Use: Observes showLabel property and calls showLabels or removeLabels.

getNodes (Function, Promise)

Accepts: Type of frame (String)
Returns: An array of node objects.

getFoci (Function)

Accepts: Number of choices (Array)
Returns: An array of foci coordinates.

createFrame (Ember Action)

Accepts: None
Action: Calls another action based on the type of frame.

createSingleChoice (Ember Action)

Accepts: None
Action: Creates a new record for a single choice frame and calls d3Init (Run force layout).

createMultipleChoice (Ember Action)

Accepts: None
Action: Creates a new record for a multiple choice frame and calls d3Init (Run force layout).

createDialog (Ember Action)

Accepts: None
Action: Displays createFrame model if CSV file is loaded.

deleteFrame (Ember Action)

Accepts: Frame (Object)
Action: Deletes the frame record based on the provided frame object.

showModel (Ember Action)

Accepts: Element Id (String)
Action: Displays dialog based on the provided id.

hideModel (Ember Action)

Accepts: Element Id (String)
Action: Hides dialog based on the provided id.

fileUpload (Ember Action)

Accepts: File (Object)
Action: Calls importJSONData or importCSVData based on the file type.

importJSONData (Ember Action)

Accepts: File (Object)
Action: Creates new frame records using the data in the provided JSON file.

importCSVData (Ember Action)

Accepts: File (Object)
Action: Extracts different columns from the CSV file and creates node objects. Allows to create frames from scratch.

selectColumn (Ember Action)

Accepts: Column (Object)
Action: Updates the selectedColumn (Question) and highlights the selection.

d3Init (Ember Action)

Accepts: Frame (Object)
Action: Updated nodes data, removes the nodes that are not in the data exit() creates nodes that are not on the SVG enter() and calls force layout action d3Plot.

nodeClick (Ember Action)

Accepts: Node Selection (Object) and Frame (Object)
Action: Finds all the nodes using nodeId (Including Duplicate Nodes), increases the node radius and calls the nodeInfo action.

nodeInfo (Ember Action)

Accepts: Node Selection (Object)
Action: Displays node information DOM Element (Question Answer Sets).

hideNodeInfo (Ember Action)

Accepts: None
Action: Hides the node info DOM Element.

d3Plot (Ember Action)

Accepts: Frame (Object)
Action: Updates nodes data and runs force layout force.start().

removeLabels (Ember Action)

Accepts: None
Action: Selects all the labels on the SVG and removes them.

showLabels (Ember Action)

Accepts: Frame (Object) and updatePosition (Boolean)
Action: Updated label data and creates labels 'enter()'.

updateLabels (Ember Action)

Accepts: None
Action: Hides editLabel dialog, calls showLabels action after removeLabels.

saveNodePositions (Ember Action)

Accepts: Frame (Object)
Action: Iterates over nodes on the SVG and updates the X and Y coordinates of the nodes in the frame object.

changeGravity (Ember Action)

Accepts: Change Event (Object)
Action: Sets new gravity value and calls d3Plot action after removeLabels action.

changeCharge (Ember Action)

Accepts: Change Event (Object)
Action: Sets new charge value and calls d3Plot action after removeLabels action.

changeRadius (Ember Action)

Accepts: Change Event (Object)
Action: Transitions into new radius value transition().

selectFrame (Ember Action)

Accepts: Frame (Object)
Action: Updates the nodes data, removes nodes that are not in the data exit() creates nodes that are not on the SVG enter() and transitions into the specified node positions.

showNotification (Ember Action)

Accepts: Type of notification [Error, Warning, Info, Success] (String) and Notification Message (String)
Action: Displays a notification at the bottom of the view which automatically disappears after 2200ms.

sendToServer (Ember Action)

Accepts: File (Blob) and File Name (String)
Action: Posts the data to /api/project and displays a notification with Project ID Hash.

exportData (Ember Action)

Accepts: Type (String)
Action: Iterates over all the frame records and creates a Blob, calls sendToServer if the type is Publish or else downloads a json file DotPlot.json.

License

Dotplot is licensed under the GPL

Copyright

2017 President and Fellows of Harvard College

dotplot's Issues

Change distance between foci using slider.

Save node position after force layout ends.

Save all the calculated positions when the force layout ends to reduce computation.

Support time based dynamic node positioning.

Manual colour selection for node clusters.

Ability to select cluster colour manually.

Upload window popup animation is too slow

Can we make this animation duration about a quarter of what it currently is?

Supervised data parsing.

Provide a supervised CSV parsing support to support all kinds of CSV export files.

Reverse Foci Distance slider

If I move the slider to the right, I expect the Foci to get further apart. Instead, they get closer. Can we switch that?

CSS for smaller screens

It looks like the creation view is currently optimized for screens 1333 pixels wide or wider. If I make my browser window narrower than that, the resulting scrollbars allow me to put the top and bottom panels on top of the left side panel.

Add and edit foci labels.

Text based search with node highlighting.

Manual text-based search for finding (highlighting) nodes in different clusters.

Organize nodes by color

When running the force layout to generate node locations, nodes should first move to cluster by color before switching part way through the transition to cluster by foci. This ensures nodes of the same color will be close to one another in the resulting foci. It both makes the graph appear more organized and makes each foci into a rough little pie chart.

Invalid file type or uploaded file

I installed dotplot in my pc (windows 10) and when I try to upload either csv or json file the message "invalid file type or uploaded file" appears and I cannot make anything then.
Any help please?
Yannis

Update build process.

Including Webpack in the workflow.

Add Design Files

Sketch UI Files

Move node info box in public view

When highlighting a node in the public view, the node detail box pops up in the center of the screen, which is often on top of a good portion on the graph.

Design workflow of turning survey data into graphable JSON

Right now, we're focusing on turning CSVs exported from Qualtrics into JSON that we can visualize.

We need a tool that a non-programmer can use that allows them to upload the CSV exported from Qualtrics and get back a basic visualization in return.

Maybe we need auth, maybe we need additional info from the user that's not in the CSV, maybe the user needs to pick some settings. This is the workflow that needs to be designed before we can start coding anything.

JSHint Errors.

Following errors in the latest pull request.

controllers/create.js: line 12, col 47, 'reject' is defined but never used.
controllers/create.js: line 390, col 39, 'd' is defined but never used.
controllers/create.js: line 393, col 39, 'd' is defined but never used.
controllers/create.js: line 401, col 47, 'i' is defined but never used.
controllers/create.js: line 12, col 20, 'Promise' is not defined.

Change handling of frame titles

Let's make frame titles required. Then, in the frame cards on the left, let's remove "title:", "count:" and "type:" and just show the frame title.

Migration to D3 version 4.

Fix slider values.

Fix maximum and minimum values for Charge, Gravity and Radius sliders.

NProgress integration.

Display process progress using NProgress.js in DotPlot.

Publish Project

Save model data and CSV file on an Express.js server.

Update node position on frame selection.

Update the current visualisation to selected frame on click (Reposition nodes).

Add and update documentation.

1. Update app screenshots.
2. Code Comments.
3. Add contribution guidelines.
4. Add demo video.

Transitioning between frames is buggy

I don't think storing the node locations for each frame is working correctly. If I create one frame, it looks good. If I create a second frame, that also looks good. If I now click on the first frame, all the nodes jump up to the top left corner.

If I click reset the node positions using the little grid button, I can get positions back, but the nodes often end up in the locations for the other frame if I keep clicking around.

Support for multiple choice questions.

Warnings

Display warnings and errors for invalid actions or data.

Show only single field for multiple choice questions in the column list.

Save node position button.

Add a button to save the position of nodes. This is help retain the changes after changing the charge between the nodes and distance between the foci.

Make "Add Frame" step more obvious

After uploading a CSV, it's unclear what to do next. Maybe we should just automatically open the Add Frame dialog. What do you think?

Manually re-locating foci.

Ability to drag and change foci location on the SVG.

Public View

Public view to demonstrate the project.

Keep highlighted node large when transitioning frames

When a user clicks on a node, the node info box pops up and the node grows in size. If the user then transitions to another frame, the node info pop stays up, but the selected node decreases in size. It should stay large until the node info box is closed to aid in visually following the node through the various frames.

Change Icons

Frame customisations stops working for multiple choice questions.

#20 Issue in latest pull request.

Text alignment issue when re-organising nodes on a saved project.

When using the sliders to organise nodes (Running the force layout again) after updating the text of the labels, the alignment is not proper. This is caused because of the repetitive multiplication of the scale value with the coordinates. It's better to update and store the node and label coordinates instead of using the scale value for each calculation.

Make node stroke colors consistent

I'm not sure what the logic behind the stroke colors (the color of the edge of the circle) is right now, but it's confusing. Can we make the stroke color just a few shades darker than the fill color? My old code did this, if you want an example.

Update Berkman logo

We just renamed ourselves to the Berkman Klein Center and we got a new logo in the process. You can grab it from here: https://cyber.law.harvard.edu/sites/cyber.law.harvard.edu/themes/berkman/images/logo.png

Change "Save / Publish" text

In the left panel on the create page, above the buttons "Save" and "Publish", it says, "Looks good enough? You can either export the nodes or publish it on our platform." I think we can remove this text and change the "Save" button to say "Export".

Add help text to frame buttons

In the creation view, it's hard to tell from the icons on the frame panel buttons what the buttons do. Adding a title attribute with some descriptive text would help a lot.

Export Project

Export project in a reusable JSON file.

Highlight selected column in the list.

Change opacity and background color of the selected column in the list.

Build fails

I think you're missing nprogress in the bower requirements. I get the following error when serving:

The Broccoli Plugin: [SourceMapConcat: Concat: Vendor /assets/vendor.js] failed with: Error: ENOENT: no such file or directory, stat '/home/jclark/projects/dotplot/tmp/source_map_concat-input_base_path-2aRCImAD.tmp/0/bower_components/nprogress/nprogress.js'

Fix deprecation and build errors. [Dependency Update]

1. sassOptions (Should be defined in ember-cli-build.js)
2. Overriding init without calling this._super is deprecated.
3. Overriding Date.parse with Ember.Date.parse is deprecated.
4. Update dependencies.

Show node details on click.

Allow updating already published graphs

If I publish a graph, but then go back and edit it, it's unclear how I can save my changes and retain the same public ID.

Change charge between nodes using slider.

Add Frame modal text changes

The title text currently reads: "Create a new frame. Start by selecting single or multiple columns from the list bellow." Let's change it to: "Create a new frame. Start by selecting a survey question below."

Let's change "Column Id" to "Question".

Let's change "Tell us something about the nodes" to "Frame title".

Not text, but let's also change the color of the Create button to something that distinguishes it from the Cancel button.

Drag to reposition frames.

Drag frames in the sidebar to change the order.

Dependency Update.