boyle / 2018-measure-stress Goto Github PK

Users should be able to create a new patient profile within the application. This can likely be done with a form. Fields to be collected will be specified in a JSON document and rendered as input boxes, radio buttons, etc.

Li C, Cui Z, Zheng W, Xu C, Ji R and Yang J 2018
Action-attending graphic neural network
IEEE Transactions on Image Processing 27(7) 3657–3670.
https://doi.org/10.1109/TIP.2018.2815744

Reproduce something like this in tensorflow.

follow up to issues #4 and #5; for the "head" compute node which is not yet being actively used.

• enable backup to /cubed at carleton
• enable rsnapshot of /data directory

Both these functionalities are coded into default.sh, though the backup has to be enabled on the Carleton end as well.

The tablet app needs a way to store configuration off the tablet. The API needs to support this functionality before it can be implemented for the tablet... so store a JSON config file for each clinician?

How does this fit into the data storage structure?

• create user directories on new user registration
• add tests for user directory creation (success/fail)
• use cookie to identify current user when API end-point GET/PUT api/v1/u
• support PUT for user data (preliminary #27)

There are >50 different "apps" to choose from; it should be possible to log which "app" is selected for a given activity.

Note: There is a possibility to use some sort of autocomplete with fuzzy matching so that they do not have to scroll between 50 activities.

Jest fails to run test suite on my mac (macOS Sierra).

 FAIL  ducks/user.test.js
Test suite failed to run

    Couldn't find preset "module:metro-react-native-babel-preset" relative to directory "/Users/fcharih/Projects/2018-measure-stress/app"

      at node_modules/babel-core/lib/transformation/file/options/option-manager.js:293:19
          at Array.map (<anonymous>)
      at OptionManager.resolvePresets (node_modules/babel-core/lib/transformation/file/options/option-manager.js:275:20)
      at OptionManager.mergePresets (node_modules/babel-core/lib/transformation/file/options/option-manager.js:264:10)
      at OptionManager.mergeOptions (node_modules/babel-core/lib/transformation/file/options/option-manager.js:249:14)
      at OptionManager.init (node_modules/babel-core/lib/transformation/file/options/option-manager.js:368:12)
      at File.initOptions (node_modules/babel-core/lib/transformation/file/index.js:212:65)
      at new File (node_modules/babel-core/lib/transformation/file/index.js:135:24)
      at Pipeline.transform (node_modules/babel-core/lib/transformation/pipeline.js:46:16)

The program rsnapshot keeps copies from a couple hours ago... prevent serious errors with delete/modify and we need it back.

What data do we want to share between client and server. In this iteration: http GET only.

• user/clinician configuration (/v1/u/<config.json|patients.json>)
  • uses username from login
  • symptom short names
  • associated patient IDs (generate from patient data)
• patient data (/v1/p/<patient#>/<session#>/<datafiles.(dat,json)>)
  • vicon data
  • heart rate
  • force plates
  • signs & symptoms (app data)
  • currently only inserted using file /upload endpoint
• versions (/v1/ver/<app|web>)
  • report on website revision
  • report on latest app revision (tablet)
  • later... app update pull? (what is our tablet update mechanism!)

Authentication on the web server needs to be enabled around uploads and the API interface. Its only a matter of time before this gets abused by a random passer-by/script-kiddy.

Various security vulnerabilities flagged by github.

Hook up app to Redux store for global state management. The Redux state tree would include, among others:

• User information
• User preferences (to be rehydrated upon retrieval from server)
• Current session metadata
• Current session state (is an activity ongoing, where in the session are we, etc.)
• UI state

New Session + Scan QR takes you to the Review modal, not New Session creation.

New Session + Skip (QR) correctly takes you to the new session creation modal.

The new user authentication key is part of the code. Prior to moving the repository to being public, I need to put that key into the config and change it. The actual config file will be held outside the repository, with dummy values provided in the repo and for testing.

Setup a confirmation, so that the admin actually authorizes the new user. This prevents a script-kiddie from brute-forcing the authorization key and being able to create a new user which has full access to our data.

Don't lock tablet when recording data.

Sitting with Roger, we are thinking it is important to be able to sort out what went wrong if someone enters the wrong patient number of session number. We don't delete data, so its still there, in principle, but untangling the mess after the fact could be difficult.

We need a log of user activities, for logged in users.

Tried changing return code to 201 "created" on registration success (and redirect to login page), but then the redirect doesn't happen in the browser.

Make Francois happy by returning a nice code on success for creating new users.

Problem: Local authentication should be supported. Clinicians should be able to authenticate even if the network is down.

Certbot renews the SSL certificate as a cron job. It currently fails.

The certbot sets .well-known/*.http http files to be served from /var/www/saans.ca/htdoc/.well-known, or something similar. Flask hides this directory, and needs an explicit pass-through in the web app code.

The SSL certificate cannot be renewed until this issue is fixed and the current certificate expires Jan 16, 2019.

http://api.saans.ca/ != https://api.saans.ca/app.fcgi/api/v1

The link given by slack when you don't "copy link" is:
https://slack-redir.net/link?url=http%3A%2F%2Fapi.saans.ca%2Fv1&v=3

My suspicion: the lighttpd rewrite rules are cutting out early when it goes http --> https.

This link works properly (http --> https)
https://slack-redir.net/link?url=https%3A%2F%2Fapi.saans.ca%2Fv1&v=3

This also breaks it, without slack in the path:
http://api.saans.ca --> https://api.saans.ca/app.fcgi/api ... but
https://api.saans.ca works just fine.

Persistent storage volumes need to be hooked up and used for all the core data. If a server goes down, we should not lose our data.

• data
• user credentials

setup travis-CI to run on flask code

Within a session, the user will be able to move sliders to quantify physiological observations. Courtney has given me a few types of observations for which sliders should be added.

Ideally, it should be possible to reorder the sliders.

It should be possible to select one of the following common events at any point during the session:

• Putting on HR monitor
• Putting on harness
• Markering subject
• Entering Treatment Room
• Starting treatment session
• Current doing VR Treatment
• Break in between treatment (standing or sitting)
• End of Treatment Session
• Leaving treatment area
• Removal of markers
• Removal of harness
• Removal of HR monitor
• Education with Clinician?
• Guided relaxation

Questionnaire is available here.

See https://github.com/realpython/flask-jwt-auth

We can also authenticate via JWT for the app.

All the activities should be on the same scrollable plot. Rest periods should also be included; i.e. the app should alternate between activity and rest.

We would like to be able to create endpoints for PUT requests so that files can be saved (patient records, annotator config files, etc.)

Roger has pointed out that we need a way to recover from a lost password.

This needs an email mechanism and the ability to store a "key."
We can use the same mechanism for new user creation (#29).

Users will need a configured email address, and that email address will need to be validated.

The app should be able to be robust to loss of internet connectivity. The most obvious way to deal with that is to implement local storage:

1. Data is saved on the device during the session;
2. At end of session: an attempt is made to upload the session data to the server;

• If upload is unsuccessful; the file is stored in a directory of failed uploads.
• User is asked to answer a security question whose answer will let them claim ownership of the file once connection is re-established. (i.e. what was your first pet's name, paternal grandmother's given name, etc.)

3. When connection is re-established, file ownership can be claimed and file is uploaded.
4. File integrity verification (with hashing?)

It appears that the Let's Encrypt SSL Certificate is not trusted when requests are made outside of the browser. I experience issues with both curl and inside the app when I make requests at the login endpoint.

Output of a cURL request:

$ curl 'https://saans.ca/auth/login' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' \
-H 'Accept-Language: en-CA,en-US;q=0.7,en;q=0.3'\
-H 'Content-Type: application/x-www-form-urlencoded' \
--data 'username=fcharih&password=<mypassword>'

curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

Problem:
Inevitably, the event timestamps will have some offset with respect to the physiological measurements. These two types of data (measurements and events) will need to be aligned.

I am assuming that there will be two devices:

1. The tablet
2. A desktop computer to which the sensors are connected

Potential solutions:
Simplest Have the person holding the tablet and the person at the computer press a button simultaneously at the beginning of the session. The timestamps can be used to compute offsets.

New user registration button on login page does not take user to registration page. Instead it does the "submit form" action. This is because it is a "button" item instead of an "a" link.

See stackoverflow: How to create an HTML button that acts like a link?.

Increase file size limit. Currently 500 MB.

What should happen when the network is unavailable?

We may see very large file uploads. It would be nice for that to provide feedback on the upload.

Looks like one easy to install approach with a nice implementation would be
https://github.com/flowjs/flow.js

which has an easy-to-adapt-to-flask python implementation on the server-side
https://pypi.org/project/flowjs/

Flask provides a nice web testing framework. I should write some tests for the API and web site services.

When a new version of the app is deployed, the app should update automatically on the tablet.

See https://codeburst.io/react-native-updates-with-vs-app-center-codepush-3d56ef07f1c4 f for an example of how this can be done.

Naturally this should not happen in the middle of an annotation session. This could be done when the user logs in.

Github is hassling me automatically about security problems with some of the app dependencies app/yarn.lock. I'll update the dependencies for the app. Hopefully nothing breaks. It should be easy to roll-back if anything is hosed.

The comments box now lets me enter the comment, keyboard pops up, and I can save my comment. I scrolled back and double tapped the comment in the plot to edit the contents (session still running). Then went back "reset" to the current time to create another comment. It keeps loading the original (first) comment, rather than creating a new comment. Or at least, it does not create a new icon on the graph. Its hard to see if its just going back to the first comment or creating new comments starting from the original text and saving them correctly. Issue #38 (keyboard not activating) is fixed and happy.

setup travis-CI to run on app code

SSQ seems to happen at somewhat arbitrary times within the session. For example, sometimes it happens before markering, sometimes after. Particularly at the end it depends on how ill the patient is. De-marking might happen first or might want SSQ as they walk off the platforms. We could treat the SSQ as a formalized time stamped comment and run it whenever someone wants it. We lose the enforcement. Maybe this needs some sort of flag above the session plot as it runs to indicate an SSQ is appropriate... for example outside of each activity reset the "needs SSQ" flag.

Anyways, ideas for discussion.

If the web server is unavailable, we still need to collect data. Issue#2 deals with the login, if the user has authenticated with the current device previously. This addresses the vast majority of start up connectivity problems.

Finally, we get down to a much reduced failure case, which we can address in some later sprint.

If the device is new, or this is a new user, we still need to collect data and deal with authentication later.

In this case, we should have a "guest" login. Data will be stored locally under the guest user id (0).
Any data with user id (0) will not pushed from the local cache to the server.

Logged in users will be able to see and claim the guest data, at which point it will be re-tagged in the local file cache with that user id, and therefore be clear to transmit to the server at the next opportunity.

The core data needs to be backed up at Carleton. This data needs a storage location and a reachable address for rsync and friends. Persistent storage volume needs to be backed up.