ccali / a2jdat Goto Github PK

When in author creating new, valid guides not always creating templates.json resulting in blank screen in templates tab and ENOENT in error log. Failure to create templates.json is not being recorded in logs

Adding even a couple of lines to a custom footer adds large space at the bottom of a rendered text pdf

DAT kept some docker files for an alternate dev setup that should really be part of the Author repo. These are not related the docker version that @tobiasnteireho made during the hackathon.

If there is an issue creating a template file, or if there is a reference to a template ID number in the templates.json file that doesn't exist, A2J should handle that cleanup, or prompt the author to remove the reference. Currently if the templates.json index file points to a template that doesn't exist, it causes the templates page to not load at all, and requires an Author to publish the interview and re-upload it to fix those references.

Should handle:

• templates.json file doe not exist, but template files do (template1.json, template2.json, template2.pdf)
• templates.json references a template ID # that no longer exists
• moved to #69

As a developer, I want to have automated accessibility testing set up for the preview page, so that I can catch errors faster and avoid regressions.

We can use axe-core for integration testing the generated html page, in addition to unit tests in the individual repos.

If we use axe-core, then I think we might need to:

• Set up Testee with Firefox: https://github.com/CCALI/a2jdeps/blob/344f10c539c8419130a4b8f65e2e1f918a095862/package.json#L9
• Set up a test that calls the DAT server to get a preview (or somehow get the preview HTML into the test; this could also be a script that generates the preview HTML that the tests then fetch, e.g. how the actual app works)
• Include axe-core in the test page and run it

Feature request to be able to show the current date with a simple setting/tick in the template(s). I think currently we can display a variable, but that will only grab the current date via logic for the day the GI was taken, not the date the form was printed.

Found a bug where unanswered variables in text templates were rendering the variable name instead of being blank as expected. Turned out to be that the done-css/steal-css compiled css stylesheet was not being read/loaded and so the final html to be rendered via wkhtmltopdf did not have the styles need to hide that unanswered variable name.

Could be one of these problems:

1. the wrong path is generated - unlikely as there is also a manually created link that has the right path
2. it's silently failing to read/load that css file sheet
3. the path and ability to read are there, but some race condition prevents the styles from being inserted in time

Currently we are running the development version of the DAT in production, which is working but not ideal.

current good zoom on linux looks to be closer to 1.600 based on cal. this has changed before #37 . Is this due to deps?

lsnj reports that Changing title of GI to unicode characters (arabic observed) causes DAT to fail to assemble. This is related but separate to #71. If the title only contains roman chars but the content is in unicode assembly works

unoconv can convert from pdf to html which may allow for implementing pdf accessibility for combined document

https://github.com/unoconv/unoconv

It looks like CKEditor allows display of checkboxes and radio buttons, and we can likely hook into that with a custom plugin to allow a GI variable to check or select those boxes on render.

node 12 goes EOL 2022-04-30

observed on production a2j.org that mixed templates were occasionaly failing to properly assemble. Errors are generated but not useful with some examples below occurring on multiple lines. Unreproduceable after updating pm2 and restarting process set.

The most salient likely being

(node:13244) Warning: Accessing non-existent property 'current' of module exports inside circular dependency

with occasional appearances of below which would also accompany good documents

can-view-scope::attr is deprecated, please use peek, get or set

done-autorender didn't receive route definitions.For Server-Side-Rendering you have to provide an initialized can-route.See the guide for information. https
://canjs.com/doc/can-route.html and https://donejs.com/Apis.html#can-route

The DAT viewer path is derived from Author code that expects it and the guides folder to be in a certain relative space.
https://github.com/CCALI/CAJA/blob/bitovi/src/util/paths.js#L21

Update the code so that it can be explicitly defined in config.json as we did for the location of wkhtmltopdf.

Manju D J from marlabs reports pm2-windows-service has issues with node 12 on azure. including:

1.) Earlier installed pm2 windows service was pointing to older version of nvm folder which was not valid.

2.) System variable “PM2_SERVICE_PM2_DIR” was pointing to wrong nvm location.

3.) Inquirer module was not updated to newer version to support latest npm version.

3 is resolved with jon-hall/pm2-windows-service#56

Need to update docs.

The Template Options do not seem to be applying correctly when a document is generated.

In the below examples, there is a header/footer, section heading, and some regular text in the body. The header, footer, and body are all set to use "Normal" paragraph format and the "(Default)" font and size (which I presume should come from the Template Options)

Example 1: Set Font Family to Times New Roman and size 14.

• The resulting PDF is all in Times New Roman (yay!)
• The header/footer are 9.60 and the body is 8.45.

Example 2: Set Font Family to Arial and size 10.

• The header is Times New Roman size 9.6
• The section title and body are in Arial, but size 5.95

The header/footer seem to be 'stuck' at Times New Roman size 9.6, and the body is coming out around 60% the expected size.

Being that I am not terribly familiar with the DAT, this could be user error or some settings I am missing, but if so, those are not obvious to me and may be easily missed by other users.

Tested in staging v 6.0 2020-07-29. Interview/templates and examples are attached.

Example 1.pdf
Brett's component split test 1.zip
Example 2.pdf

Feature request from an author -

Allow authors to set the font size for a single variable to be different from the entirety of the template. The author wants one variable to be larger than the rest to fit with a court form.

As a screen reader user, I want the page to have the lang attribute, so that the page is read with the correct accent.

Currently, no lang attribute is provided for the preview page. This should probably be set to en-US.

Slack thread for context: https://cali.slack.com/archives/C069V9TTN/p1639757244068800

Steps to reproduce:

1. Remove the .skip from a preview test, e.g.

   a2jdat/test/routes/preview.js

   Lines 29 to 30 in 6f59824

   	// TODO: this test is skipped due to the issue described in https://github.com/CCALI/a2jdat/issues/100
   	it.skip('previews basic templates correctly', function (done) {

2. Run the tests (npm test)

Expected results: tests would pass.

Actual results: all the tests pass except this preview test.

Note that if you skip this test, then the preview test will pass:

What I observed with @tobiasnteireho is that when a document is rendered with SSR for assembly or preview, it works the first time, but subsequent calls fail because the server has been torn down.

I am happy to pair with whoever picks this up so I can help them reproduce it.

Have a GI with a conditional text template (or several conditional text templates). If the condition for that text template isn't satisfied, a blank page is included in the assembled document in the spot the template would have been.

This is generally an odd, but workable solution, until you get multiple conditional text templates in a document assembly package. It creates confusion among end users as to why something blank came out with their documents and makes it look like something went wrong.

This only happens with text templates. Does not happen with PDF templates. If a PDF template's condition isn't satisfied, it just does nothing. That's the expected behavior and should be what text templates do as well.

Text GI
Assembly test for DAT (July 2020).zip

Conditions are [Have children TF] = true for second text template to assemble and [Client first name TE] = JessBob for PDF Template to assemble.

Test answer files: answer (63).zip

The text template header does not print the first 2-3 lines of text in the final document. Putting blank spaces does allow header text to show, which seems to indicate it's a CSS or done-ssr rendering issue with the template header.

unable to install or manage npm with nvm after runnning latest windows system update.

getting errors such as coreybutler/nvm-windows#580

Downloading node.js version 12.22.1 (64-bit)...
Complete
Downloading npm version 6.14.12... Download failed. Rolling Back.
Rollback failed. remove C:\Users\a2j\AppData\Roaming\nvm\temp\npm-v6.14.12.zip: The process cannot access the file because it is being used by another process.
Could not download npm for node v12.22.1.
Please visit https://github.com/npm/npm/releases/tag/v6.14.12 to download npm.
It should be extracted to C:\Users\a2j\AppData\Roaming\nvm\v12.22.1

reinstalling nv, and disabling windows defender does not fix and in at least one case has made the problem worse resulting in this error occasionally occuirring

ERROR open \settings.txt: [...]

coreybutler/nvm-windows#22.

nvm-windows latest release is from 2018 and appears to be under less active development. Investigating alternatives such as nvs (https://github.com/jasongin/nvs) and linkedin's volta (https://github.com/volta-cli/volta#installing-volta)

HummusJS is now an abandoned project since Nov 2019. Need to find an alternative soon.

https://github.com/galkahana/HummusJS#hummusjs
https://pdfhummus.com/post/188923357126/ending-support-for-pdfwriterhummus

[email protected]: Please upgrade to @mapbox/node-pre-gyp: the non-scoped node-pre-gyp package is deprecated and only the @mapbox scoped package will recieve updates in the future

text templates are not rendering and yielding the following error on one azure box

3|CALI A2JDAT  | can-view-scope::attr is deprecated, please use peek, get or set
3|CALI A2JDAT  | Failed promise: TypeError: Cannot read property 'TEXT_NODE' of undefined
3|CALI A2JDAT  |     at Object.live.text (file:C:/inetpub/wwwroot/a2jdat-20201118/a2jdat/node_modules/can-view-live/lib/text.js:19:27)
3|CALI A2JDAT  |     at Text.branchRenderer (file:C:/inetpub/wwwroot/a2jdat-20201118/a2jdat/node_modules/can-stache/src/mustache_core.js:406:11)
3|CALI A2JDAT  |     at Comment.callback (file:C:/inetpub/wwwroot/a2jdat-20201118/a2jdat/node_modules/can-view-target/can-view-target.js:223:18)
3|CALI A2JDAT  |     at hydrateCallbacks (file:C:/inetpub/wwwroot/a2jdat-20201118/a2jdat/node_modules/can-view-target/can-view-target.js:262:39)
3|CALI A2JDAT  |     at Object.hydrate (file:C:/inetpub/wwwroot/a2jdat-20201118/a2jdat/node_modules/can-view-target/can-view-target.js:284:4)
3|CALI A2JDAT  |     at HelperOptions.convertedRenderer (file:C:/inetpub/wwwroot/a2jdat-20201118/a2jdat/node_modules/can-stache/src/utils.js:80:17)
3|CALI A2JDAT  |     at HelperOptions.fn (file:C:/inetpub/wwwroot/a2jdat-20201118/a2jdat/node_modules/can-observation-recorder/can-observation-recorder.js:87:18)
3|CALI A2JDAT  |     at A2JVariableVM.ifHelper (file:C:/inetpub/wwwroot/a2jdat-20201118/a2jdat/node_modules/can-stache/helpers/core.js:215:26)
3|CALI A2JDAT  |     at {{if(canShowAnswer)}} (file:C:/inetpub/wwwroot/a2jdat-20201118/a2jdat/node_modules/can-stache/expressions/call.js:110:16)
3|CALI A2JDAT  |     at Observation.onBound (file:C:/inetpub/wwwroot/a2jdat-20201118/a2jdat/node_modules/can-observation/can-observation.js:87:27)

Investigating

Staging had zoom setting at 1.0 which was causing text templates to produce fonts that were too small for their set size. Mike and I tested it and he found the 1.6 setting most closely matches expected font sizes in PDFs.

need to update the zoom setting to 1.6 in all environments and restart

LSNJ reports the attached guide with the attached answerfile renders question marks in text boxes in pdf template.

vcco-ar-test-assemble-1.pdf
VCCO-AR (1).zip
guest192.168.230.131.anx.zip

CKEditor defaults to double space on hitting the enter key, and will single space only if you hit shift+enter. There's likely a toolbar button for changing this option that we are not showing, and should add it.

For consistency and for better support should migrate to volta-cli everywhere and drop nvm.
https://github.com/volta-cli/volta

Update the development branch folder structure to more closely match the current main branch for easier deploys.

As a user of assistive technology (AT), I do not want the preview document to include validation errors, so that my AT can accurately interpret and parse content.

When searching for variables to insert into a template, the variables are listed in no apparent order.

Tested in staging v 6.0 2020-07-29

PDF templates can set the margins for Addendum pages, but have had request to be able to set margins in text templates as well. Should look into global setting vs page by page setting, likely having both options with warnings if updating a global setting does not match an already set local page.

This issue is for planning and scoping the DAT Accessibility project (TIG 20042). Feel free to add to this issue with brainstorming ideas. When ideas have formed into distinct tasks, we'll break them off into separate issues.

"As a user I should be able to..."

• verify my answers before my final printable document is generated

"As an author I should be able to..."

• learn about making my underlying PDF accessible from the A2J Authoring Guide
• watch a video that explains how to make my underlying PDF accessible on the A2J Author YouTube Channel
• learn about making my underlying text template accessible from the A2J Authoring Guide
• watch a video that explains how to make my text template accessible on the A2J Author YouTube Channel

Incorporate error messages from the joi library to display a more verbose error message.

The validate function in validate.js returns an error object for a failed request.
This object has a details array which can be used to display relevant messages to make debugging easier.

Having no magic defaults would make administration considerably more easy.

These are the keys that should throw an error. Minimum should throw an error on missing or empty keys. Ideal would also throw error on key that does not make sense, e.g. path does not actually contain expected item.

GUIDES_DIR
GUIDES_URL
VIEWER_PATH
WKHTMLTOPDF_PATH
WKHTMLTOPDF_DPI
WKHTMLTOPDF_ZOOM

related to https://github.com/CCALI/CAJA/issues/1962

As a user, I want the page title to be more descriptive, so I can more easily understand what the preview page is when it’s opened and I perceive it in my list of browser tabs/windows.

Currently, “A2J Test Assemble” is the page’s title. If possible, this title should be changed to be more meaningful.

A2J DAT needs the readme.md updated to show how to install and use the new DAT. Also add the new GNU AGPLv3 license.

Created a simple test interview for addendumlabel testing on hydra attached below.

Loaded it into hydra and ran a test with a repeat loop that should contain four items: Car, House, Toy, and Ball as seen here in the screenshot I took before I hit the assemble command.

The output of the document showed car car and house. Toy and Ball are missing. The template is set up to append to an addendum, just the overflow. I purposely did not give it an addendum label.

Test for AddendumLabel Template Issue 4 26 2021.zip

The output PDF with the errors is attached as well.
test-for-addendum-label-template-issue-4-26-2021 (1).pdf

Legacy text templates are not created with a state object by default, which is checked here for conditional renders

throwing an error. Templates without a state object should also return true and render by default.

When you have a text variable grouped with Ignore text overflow checked, but don't fill in an addendumLabel it will fail the assembly saying it's required in an error thrown in the DAT / Node console. If you add a label it will render/work, even though it should not care about a label.

Likely needs to be fixed in the patcher, or in code that calls the patchMultilineText() function call:

Breakpoint for checking the patches and if they have an addendumLabel provided and are of type multi-line-text is here:

Sometimes other instances of grouped text variables have an addendumLabel set in the patch (matching the variable name) but don't show or render anything in the Variable modal. This may be a separate issue/edge case.

Attached sample of failing Variable setup:

attached guide's has pdf and text templates. When assembling combined document, not assembling text templates when with or without answers. Text templates alone do assemble properly. No errors are generated in console or in pm2 logs.

Sample A2J Guided Interview (for Sample Exercise).zip

When there are template errors from validation, it shows a list of errors that may or may not be directly related to the problem. In the below error case, the actual error was that the child "overflow" fails because [child "addendumLabel" fails because ["addendumLabel" is not allowed to be empty]]. Should see if one error is polluting or causing another, or if it's just reporting 'all the errors' because of a bad config or bad error condition defintion.

UnhandledPromiseRejectionWarning: ValidationError: child "patches" fails because ["patches" at position 35 fails because [child "type" fails because ["type" must be one of [text]], child "overflow" fails because [child "addendumLabel" fails because ["addendumLabel" is not allowed to be empty]], child "type" fails because ["type" must be one of [table-text]], child "type" fails because ["type" must be one of [checkmark]]]]

This warning as well, which will most likely go away as CanJS packages are updated, but we should find which specific one is logging this warning:

can-view-scope::attr is deprecated, please use peek, get or set

from CAJA, @tobiasnteireho comments
this is showing up in the latest dat on windows 10 but not in linux.
this is being observed in linux on prod and may be linked to occasional failures to produce non-blank documents.

This is related to the Viewer issue to allow Text template previews CCALI/a2jviewer#170

• add a /preview route and pair with a function to handle generating the html preview
• refactor any current functionality in the /assemble route/function to allow sharing of reused code for this new feature
• make sure the preview does not change or break the assemble code

Generally a text template has a 1 inch-ish looking margin around the entire border of the document.

When a custom footer is added, the margin goes down to nothing/zero inch

When a custom header is added, the margin appears to come back

Please check if there is a margin setting that isn't triggered when the footer is added or is changed when a footer is added. @AnubhavDhingra

Here's a basic interview with a template that has a header and a footer to demonstrate the issue.
My Interview (8-25-2021).zip

***Note to myself: contact Gail Pals when issue is resolved.

node-pre-gyp and hummus are deprecated and do not work properly on versions of nod beyond 12.20.1

Set font to Times New Roman size 12 in Template Options, add Rich Text element, add IF element with Rich Text inside it.

Assemble DAT and get document with different font styles and sizes. Looks like IF element is rendering Times New Roman 12, but Rich Text alone is not.

Test GI: DAT font test.zip

Using ckeditor to insert a variable in the text template header does not render any value associated with that variable during assembly/test assembly.