ccali / a2jauthor

CALI A2JAuthor document automation authoring and viewing platform

A2J Author 11 (previously know as CAJA)

This repo is part of the A2J Author Project which consists of four repos...

1. A2JViewer - https://github.com/CCALI/a2jviewer

2. A2J Author - https://github.com/CCALI/a2jauthor

3. A2J Document Automation Tool - https://github.com/CCALI/a2jdat

4. A2J Dependencies - https://github.com/CCALI/a2jdeps

====

This the public repo for the browser-based versions of A2J Author

IF YOU ARE ATTEMPTING TO SELF HOST AND RUN THE A2JAUHTOR SUITE IT IS HIGHLY UNLIKELY THAT THIS IS THE REPO YOU ARE LOOKING FOR!!! YOU MOST LIKELY NEED THE A2JVIEWER AND A2J DOCUMENT AUTOMATION TOOL. THIS REPO IS FOR THE AUTHORING TOOL.

Notes and pre-requisites

Before following the instructions for setup below. It is assumed that you have npm and nodejs already installed.

• Currently only node 16 is supported. You must have nodejs and npm installed on your system for these instructions to work
• According to John, you cannot run this on Windows machines at all due to the path length limitation of 256 characters. This is actually a limitation of the Windows API, not the NTFS file system, but it makes it almost impossible to work on reasonably complex javascript projects on Windows.

A2J Author currently requires an *AMP stack (Apache, MySQL, PHP) to be running. For local development on non-linux environments CALI uses MAMP. Make sure to update MAMP's webroot setting to the proper directory. See the [wiki](./wiki/Dev Environment MAMP.md) document for more details on setting up a MAMP environment.

For production, in addition to the above, A2J Author requires a functional drupal 7 environment. A basic installation can be performed by following the directions here

To setup:

wkhtmltopdf

Before installing the server side dependencies, you need to install wkhtmltopdf on your system first, wkhtmltopdf is a command line tool that renders HTML into PDF. The easiest way to do this is to download a prebuilt version for your system

Once wkhtmltopdf command line tool is available on your system, take note of the path as this will be needed for the config file.

Setup the database

1.) open mysql and create a user for the app. CREATE USER 'a2j'@'localhost' IDENTIFIED BY 'password';

2.) create the database for the app. This can be any available name. caja is a legacy name.

CREATE DATABASE caja;

FLUSH PRIVILEGES;

3.) grant all privileges to the above user GRANT ALL PRIVILEGES ON caja TO 'a2j'@'localhost' WITH GRANT OPTION;

FLUSH PRIVILEGES;

4.) Seed the db with the command below mysql -u a2j -p caja < wiki/resources/caja_default_2021-03-23.sql

To build the main application:

1.) clone the repo as a subfolder in the root of the drupal install if this is production or webfolder if this is development.

2.) From the root folder (a2jauthor/) run

$ npm run deploy

Server setup:

There are two configuration files necessary: config.json and config_env.ini

Server Configuration: config.json

In production mode, the server uses a configuration file called config.json that is expected to be in the parent directory of the folder where the git repo is cloned. A sample is located at wiki/resources/config.json.sample.md. This file should have the following structure:

{
  "isProductionServer": true,
  "LOCAL_USER": "45",
  "SERVER_URL": "http://my.server.org/",
  "CAJA_WS_URL": "https:/my.server.org/a2jauthor/CAJA_WS.php",
  "GUIDES_DIR": "/www/my.server.org/a2jauthor/userfiles/",
  "VIEWER_PATH": "/path/to/viewer/a2j-viewer/viewer",
  "GUIDES_URL": "../userfiles/",
  "SQL_HOST": "localhost",
  "SQL_USERNAME": "a2j",
  "SQL_PASSWD": "PASSWD",
  "SQL_DBNAME": "caja",
  "SQL_PORT": 3356,
  "DRUPAL_HOST": "localhost",
  "DRUPAL_USERNAME": "DRUPAL USERNAME",
  "DRUPAL_PASSWD": "DRUPAL PASSWD",
  "DRUPAL_DBNAME": "DRUPAL DBNAME",
  "DRUPAL_PORT": 3356
  "WKHTMLTOPDF_PATH": "/usr/bin/local/wkhtmltopdf",
  "WKHTMLTOPDF_DPI": 300,
  "WKHTMLTOPDF_ZOOM": 1.6711
}

isProductionServer is optional for production

LOCAL_USER is used for development to assign an id for authorid. For CALI environments typically the dev user is 45.

SERVER_URL is the base URL for the server hosting the app

CAJA_WS_URL is the url path for CAJA_WS.php. This is used by the DAT.

GUIDES_DIR is the system path location of the guide files. Must be web accessible. If you are in dev mode create the directory dev in this directory now.

VIEWER_PATH is identical to GUIDES_DIR in production but is the location of the viewer when setup for standalone viewer and DAT

GUIDES_URL is the relative url of guides with respect to CAJA_WS.php

SQL_HOST is the address of the mysql server

SQL_USERNAME is the mysql username for the app

SQL_PASSWD is the mysql username for the app

SQL_DBNAME is the mysql database for the app

SQL_PORT is the mysql port where the apps database lives

DRUPAL_HOST is the address of the mysql server for Drupal

DRUPAL_USERNAME is the mysql username for Drupal

DRUPAL_PASSWD is the mysql username for Drupal

DRUPAL_DBNAME is the mysql database for Drupal

DRUPAL_PORT is the mysql port where the Drupal database lives

WKHTMLTOPDF_PATH is the system path for wkhtmltopdf

WKHTMLTOPDF_DPI is the DAT property to control how wkhtmltopdf renders documents. Usually this should be set to 300

WKHTMLTOPDF_ZOOMis the DAT property to control how wkhtmltopdf renders documents. Usually this should be set to 1.6711 on linux but this might need to be tested and tweaked for your environment to render properly.

The SERVER_URL and GUIDES_DIR properties are used by the Node server, but this file will also be used by CONFIG.PHP, which also uses the database connection information.

Server Configuration: config_env.ini

a second configuration file is necessary called config_env.ini. This is used to setup allowed file types, analytics, and a2j.org. This file is expected to be in the parent directory of the folder where the git repo, i.e. the same folder as config.json. A sample config is located here sample-configs/config_env.ini.sample

Launch the app

To launch the app simply open a broswser and navigate to the a2jauthor folder e.g. http://a2jauthor.loc/a2jauthor

To run tests:

From the root folder (a2jauthor/) run

$ npm test

Debugging the server:

Prepend any of the npm commands above with DEBUG=A2J:* For example, to debug the server running locally:

$ DEBUG=A2J:* npm start

Then any debug(...) messages in the code will be displayed in the console.

To run client tests:

$ npm test

for questions contact [email protected]

a2jauthor's Issues

All Files not refreshing after new templates

When creating a new interview, and adding templates, the All Files tab does not refresh to show the new templates.json index file or any new template files. Switching to a different GI and reloading the original does show the template files. Need to make sure that list is listening for changes/updates and refreshing the All Files view.

Outline button does nothing

I'm unsure what it's supposed to actually do (being that the DAT is currently not working in Production - I will send an email about that), but the "Outline" button does nothing when clicked. Nothing displays in the browser developer Console.

Tested in staging v 6.0 2020-07-29

Stop sharing app-footer component

Author and Viewer share the footer component currently, but as one of the functions provided by the footer is showing the current version, they should each have their own. It's also not unlikely that they would diverge in the future, so no point sharing that code via a2jdeps.

Use viewer app as preview app

Should be able to import the stand-alone viewer top level component into Author as the preview tool.

Find usage is returning values for a blank variable

"Find Usage" button in Variable design editor is returning "used by" locations for a blank, unnamed variable (shows up when you go to create a new variable without inputting a name).

Should be blank if var name is blank.

Put components in a 'src' folder to clean up root

In splitting the A2J mono repo into it's separate apps, it was easier to have a flat folder structure. Now that things are working, we should move components into a src directory to make the root folder cleaner as per @tobiasnteireho suggestion. Should also do this for CCALI/A2JViewer and CCALI/A2JDAT

Interview start page should auto update on name change

Will at NKU changed the name of his starting page, but it didn't update it for the Starting point under Steps tab. Start and Exit point, if set, should auto update when Page Editor changes the name without the Author having to go into Steps Tab and reselect it.

Set start point/set button destination not working

The list of page targets for set start point in steps and set button destination is not loading/working.

Some preview styles not loading/applying

The standalone viewer looks fine, but when used as the preview app for author, some styles are either not being imported, being imported in the wrong order, or have different selector targets. Need to resolve those issues, and clean up the Author 'styles' folder to reduce style 'noise'.

side note: does not seem to affect any functionality in preview, just styling.

Add a new destination for buttons that just displays "Need to close browser"

Author request to add a final destination option that would display to the end user a warning that they need to close their browser window.

Use case: Interview is not intended to produce a document, just triage the user or provide information alone. Last screen isn't intended to "Submit" or "Assemble" and none of our current destinations fit. Closest is [Exit - User Doesn't Qualify] which pops up a warning for user to close browser if no forwarding URL Is included by the author. However, that warning tells the user they don't qualify to use this A2J Guided Interview, which wouldn't work for desired scenario.

Suggestion - add an addition destination option to buttons called [Exit - Display Message]
It won't close the user's browser window, won't post, and won't redirect anywhere. Instead it'll display a default message that says "You have completed this A2J Guided Interview. Please close your browser window to exit."

Also if this option is selected by author, provide author with an additional field in buttons section to add a custom display message. Similar to the interaction that happens with default warning for required questions, but provided field for overriding default language.

null button cleanup causing issue with add page

needs to check for button before button.next

Saving a file remove canjs `guide` map briefly

During a save of the legacy global window.gGuide file, there is a custom listener that update the appState level guide map for canjs. That listener actually replaces the map, which causes any stache files that do something like {{#if(this.guide)}} to re-render. Up until now, that has not cause an issue for the end user, but it is the root cause of the Mapper to go blank on auto-cleanup as per this issue: #59.

The long term fix is to change the canjs guide update to not replace the whole guide, but just update the property values. This code update will do it, but needs solid QA testing to make sure the global gGuide and canjs guide map stay in sync.

  // when window.gGuide is saved to the server successfully,
  // sync the map reference in the appState.
  $authorApp.on('author:guide-updated', function () {
    // this is work around for binding error in can-map
    const gGuideMapData = {}
    Object.keys(window.gGuide).forEach((key) => {
      gGuideMapData[key] = window.gGuide[key]
    })
    // update current CanMap with global gGuide state
    // 2nd param `true` removes props not in gGuideMapData from appState.guide
    appState.attr('guide').attr(gGuideMapData, true)
  })

the above replaces these code lines:

a2jauthor/src/utils/bind-custom-events.js

Lines 71 to 81 in 9a1bd3a

	// when window.gGuide is saved to the server successfully,
	// sync the map reference in the appState.
	$authorApp.on('author:guide-updated', function () {
	// this is work around for binding error in can-map
	const gGuideMapData = {}
	Object.keys(window.gGuide).forEach((key) => {
	gGuideMapData[key] = window.gGuide[key]
	})
	const gGuideMap = new CanMap(gGuideMapData)
	appState.attr('guide', gGuideMap)
	})

Look at Miro for ideas for mapper https://miro.com/

LoneStar legal aid uses it for outlining their entire process

Moving multiple questions within Map doesn't work like 4.0

In 4.0, you could move several Questions at once in the Map by using the CTRL function on your keyboard along with mouse clicks.

This isn't possible in 5.0.

Add overlay option for loading component

The Mapper tool has an issue where the jointjs package 'blocks' the UI, preventing the loading component from showing until the last split second. Add an 'overlay' option that will appear similar to a modal, over the top of the locked UI to show the 'loading' message.

Adding a new page no longer uses prior page's name as a base

In production v 6.0 2020-05-11, adding a new page will use the last selected page's name as a starting point. For example, If page "1-Intro" was the last page selected, a new page's default name will be "1-Intro 1".

In staging v 6.0 2020-07-29, any new page will simply be named "New Page [X]".

The existing functionality is helpful and should be retained for a couple reasons:

1. When building an interview, a new page often will have a similar naming convention since it will be collecting or presenting similar information
2. Sorting: because pages are sorted by name, a new page that starts with 'N' may go somewhere completely different in the list of a steps questions and be difficult to find.

Popups can only be created on pages tab

In 4.0, you could create a pop up on the fly within the question design window (when you clicked the P you could create new popup and attach it to a word/section of a question).

In 5.0, you have to create the pop up first in the pages tab, then associate it with a question in the question design window. This isn't an intuitive workflow for authors.

Add metadata to generated pdfs

@johnmayer posted this in the a2jbitovi slack panel

https://github.com/smucclaw/complaw/tree/master/doc/ex-xmp-20200808

Makes sense to me that for any PDF that A2J produces, that we include structured metadata inside the Word and PDF docs. Might be super useful in the future - if we want to re-import the data later. Could also include things like the version of A2J Author that generated the document and maybe the A2J Author GI ID that generated it.

Map disappears when you do post it nightmare or auto clean up

Opened GI that hadn't been in new mapper yet, hit auto clean up and map disappeared. No error in the console. Tried post it nightmare, flashed map, then disappeared. Finally, clicking auto clean up twice made the map "stick". Still no errors in the console.

Browser: Chrome

Add ability to delete multiple interviews in Interviews list

This is mostly a feature for me and other advanced authors with multiple old or test interviews.

Add the ability to bulk delete interviews from the interviews tab.
Add an "are you sure" warning before permanent delete.

Empty string named variable created when author forgets to put a variable in field

Author creates a field then forgets to associate a variable with it.

A2J 4 ignored that field when creating the answer file.

A2J 6 creates a variable with the name (" ") and stores the user's input in it.

HotDocs freaks out with that empty string named variable, throws an error called "answer name missing" and causes the document to fail to assemble.

We need to remove the code that creates that blank string in A2J 6 AND do author outreach to ensure authors have assigned variables to all fields.

Posted a message to the list serves detailing the author fix to this problem and how they can easily look for variable fields with missing variables by running a full report.

Need to add Graphic Alt-Text and Video Transcript to Full Report

Added new fields to the Learn More info section of the questions. Need to add these to the Full Report. May need to add to text report as well -- assuming accessibility applies to other languages as well.

• Graphic Alt-Text

• Video Transcript

• Media label

• Change "Help" to "Response"

Split up pdf/ shared between deps and author

Most of the shared a2jdeps/pdf folder is used for editing in a2jauthor, and only the assemble module is really shared between a2jauthor and a2jdat - need to refine what stays in a2jauthor, and what should live in the shared a2jdeps (most likely just the assemble.js file and it's tests)

add version script

For branding and continuity, the plan is to make the separate repo/apps start with version 7.0.0 I think it would still be nice to add the date for quick reference when working with folks who self host. We can rewrite the npm release script to assemble a version name like:

A2J Author 7.0.1 - 2020-05-15

Update $path for a2jorg

distance to CAJA_WS.php has changed from 4 to 2, so this line needs an update:

a2jauthor/CAJA_WS.php

Line 657 in fb16179

$path = dirname(__FILE__, 4);

Changing Starting/Exit point page name breaks link

If you change the name of the page listed as the Starting Point, the Preview function results in a 'page does not exist' error. Further, publishing the interview to LHI and attempting to run it also results in a 'page does not exist' error.

Likewise, changing the name of a page listed as the Exit Point will result in a 'page does not exist' error when clicking the SAVE AND EXIT button, both in Preview mode and on LHI.

Tested on staging v 6.0 2020-07-29, but also happens in production v 6.0 2020-05-11.

Character limit hint of remaining characters/over limit doesn't show with text long field type when using expanded text pop out

Author sets character limit on text (long) field.

Character limit displays and character count down/overage shows if using text field. If you expand the text field to the pop out additional space, the character count down doesn't show and if you minimize the expanded field, the count down/overage doesn't display any more.

The end user can't move on if they've typed more than the character limit allows, but they aren't told anymore how many characters they have gone over.

Author reported not being able to type anymore once limit is hit, but I couldn't replicate that.

Here's a test GI:
Text long with character limits.zip

Add new mapper

Need to integrate the new mapper component into a2jauthor.

Interviews without templates.json file error on templates tab in Author

Previous code generated a templates.json file on loading the Templates tab, if it didn't already exist. This caused odd bugs trying to write to the file system while in the midst of an ajax call for data. Currently the code adds a templates.json file on publish, so that is the current 'fix' for this problem. Most older GIs are already built with hotdocs templates, so this edge case would only come up when an Author would want to migrate from hotdocs to A2J DAT templates. Currently we have had no one run into this issue on production, but should we start to get more migrations than publish/re upload can fix, we should re-address this.

Names of pages cut off prematurely in Map

Names of pages cut off when testing Map with 1920x1080 resolution laptop monitor at 100%

At 90%, the page names appear and they appear when on larger external monitor.

Should give the left panel a bit more room. @mikemitchel suggested 3:9 split instead of current 2:10 split of screen.

Config standardjs linting to local package.json

Need to remove old traces of jshint configs, and setup standardjs + possibly an eslintrc config file so that linting works natively after a normal npm install without and global install requirements

move styles back to local directory

The styles dependency is causing too many issues across the separate repos/apps. Moving them back to local files in a2jauthor, a2jviewer, and a2jdeps, as well as any and all shared images. This will allow for easier trouble shooting in the short term, and also the ability to trim them down in each local repo/app to what that particular app needs. Will copy this issue to CCALI/a2jdeps and CCALI/A2JViewer

Clone on Pages tab doesn't clone number of buttons in question

When using Clone button in Pages tab to create a duplicate copy of a question, it clones the page name (and appends a "2"/"3", etc) to the clone to differentiate it for the question naming purposes.

It also clones the fields and the variables associated with those fields. Including repeat status.

Does not clone buttons. So if author wants to clone a question with 3 buttons, in the clone only one button is there.

Should clone everything the author adds to the questions- fields, learn mores, pop up links, and buttons. Probably logic too.

Unknown/Null value for destination results in map location off the side of the grid

Destination of page = [Unknown id [null]: [null]

Likely user changed name of page and it didn't update properly in the interview.

Leads to a weird map experience though. Results in a line that goes off to the top left corner and author can't drag/reposition to see where it leads. Looks like there's more to the map than is being shown. In reality, the unknown page is leading to literally nowhere.

If the destination is "nowhere" the button color is gray. Perhaps "unknown" destinations should get this treatment as well. Or a visual clue to the author that their button is set to go somewhere it can't possibly go.

If an end user hits that button, they are going to get an error within the interview and won't be able to move on.

Cleanup mapper pages list and make collapsable

Should allow a User to collapse the steps, and have a cleaner interface, almost like a mini toolbar for the add button

Special characters in alt-text for graphic breaks learn more

Adding a quote inside the alt-text for graphic field results in a broken learn more. Image won't display at all.

Error in console:

Ex. used "quote" as the alt-text

This error also results in the interview basically breaking all together. When you go back to "Edit this" preview mode and authoring mode are overlapped. Requires a refresh of the page to clear the issue.

set destination style regression

After cleaning up the Author styles, and removing the unused viewer styles director, have a regression where the steps and button destination picker is 'flat' again

Change Author app-state to a DefineMap

Eventually all CanMaps should be converted to DefineMap, but it's nice to at least have the app-state level model be easier to work with.

New pages aren't immediately showing up in Map

Blank GI with new pages added or changed steps doesn't immediately show up in Map

Steps to replicate:

• Create new GI

• Go to Map, see 4 default questions there (3 in Step 0, 1 in Step 1)

• Go to Pages tab, make a couple new pages

• Add one of the new pages to a different step

• Go back to the Map. New pages aren't shown.

• Go back to Pages tab and force a Save or open Preview. Then pages show.

Links within A2J Author 5 "Help" button go to author.a2jauthor.org (not a2jauthor.org)

I am updating the Help sections on the website. I wanted to make sure the links within the authoring tool were going to the places I was updating. They aren't. They go to author.a2jauthor.org

For example, clicking the Help Button on the About Tab within A2J 5.0 takes you to http://author.a2jauthor.org/csh5/About

not http://www.a2jauthor.org/csh5/About

Does it matter?

Phone number field type is missing sample value option

No sample value field on field type: Number Phone

Update repo README.md

This README contains older info for the mono repo found at CCALI/CAJA, but should be updated to reflect the singular /a2jauthor repo.

CSS: Larger field editors leave less screen real estate

Field editors in staging v 6.0 2020-07-29 are 'taller' than in production v 6.0 2020-05-11, resulting in less screen real estate when editing pages (less screen real estate means more scrolling and time involved for finding and editing content).

Bottom of buttons overlap toolbar bottom now that button heights have increased under WCAG improvements

On Variables, Pages, All Files, All Logic, All Text, and Template tabs, the bottom of the buttons in the tool bar overlap with the bottom of the tool bar and look cut off.

Likely caused by increase in pixel size of buttons per WCAG improvements.

Example:

Map issues

• new interview - all the questions aren't auto cleaned up. They overlap each other and author has to click "Auto Cleanup" button. Default blank interview should come cleaned up already.

• Zoom to width cuts off final step in the map. No bottom scroll bar given, so can't actually move over to see it.
  

• When adding new page via "add" button in pages list on Map, new page is dropped to bottom of the screen. Need to resize screen when a new page is added so that it can be easily seen by author.
  

• Auto clean up doesn't persist. This was seen in a GI from production that was uploaded to hydra for purposes of testing map.
  

• No symbol for repeat loop on new map
  

Get Travis CI working

Should get automated Travis test runner working for a2jauthor, a2jviewer, a2jdat, & a2jdeps for branches and PRs. This repo has a .travis.yml we can use as a template

"Used by" in Variables Tab is pulling erroneous information

Author reported variable of [Employer] in variables tab showing 15 'used by' places. When looking at those used by references, the word Employer is there, but not a variable.

Talked with @mikemitchel and he thinks this may have been an old variable that was added to the variables dictionary in the code and when the variable was deleted, it wasn't deleted from this dictionary. So the 'Used by' code is looking in there and giving erroneous used by values.

[Employer] variable can be found in this interview:
TEST - Employment discrimination complaint - IDHR (1).zip

Reported by Steve Simon working on ILAO interview

A2J Author should save between tabs/pages

Anytime a tab on the vertical-navbar is selected and leave one tab for another, it should fire a save on any open guide and then switch the tabs, ideally with a spinner showing saving.

Styles issues within Author

• Buttons on variables tab are cut off - related to 44 pixel increase for button styles overall
  

• Name field is significantly larger than type and comment fields.
  

• Buttons in debug panel are cut off
  

• Styles different for Preview.js than for viewer (grass missing, step 2 bubble huge, missing step 4 bubble)
  

• In text templates, list for variables is overlapping with other text.
  

A2J GI starter templates

As per Alexandra's presentation: The idea to grab the low hanging fruit of non-fillable pdf docs that are not automated yet, and make a state neutral set of starter templates with markers for custom content that states would fill in. The ideal would be to 'meta' the ability to get a source answer file from the state that would fill in that custom content for them and generate the now Automated GI that would then be consumed via the A2J Viewer to get the automated document.

@JessicaFrank viability of this? The same tool could be used for the translation tool as well.