joefutrelle / ifcb-dashboard Goto Github PK

Some repackaging has been done and now one needs to install python-celery-common

The accession and acquisition workers fail to start when the Vagrant VM boots, because they set their PYTHONPATH to /vagrant, but that occurs during the Ubuntu boot process before /vagrant is mounted.

A workaround is to include this extra provisioning line at the end of Vagrantfile to restart the supervisor workers after /vagrant is mounted:

config.vm.provision :shell, :inline => "sudo supervisorctl restart all", run: "always"

The celery command installed by default uses Python 3 if it is present.

The "big hammer" solution of uninstalling Python 3 works, but there must be a better way.

This is mostly useful for debugging and shouldn't be enabled in a production environment; it should be commented out with a reasonable explanation.

http://stackoverflow.com/questions/35519389/vagrant-cannot-find-box

Looks like the solution is to downgrade Vagrant

the workflow service configuration includes this stanza:

WSGIDaemonProcess workflow threads=6

Typical operations against the workflow service cause IntegrityError exceptions, which are not handled properly, and result in ORM sessions getting into unrecoverable states.

This is especially a problem for accession.

A workaround is simply to set threads to 1, and that should be done immediately. Long-term there needs to be a solution where concurrency works. I believe the bug was introduced when switching from SQLite to PostgreSQL, which have differing concurrency behaviors.

Editing /etc/fstab is not for the faint of heart, so I should probably switch to using credential files, and put them in /vagrant so they can be edited from the host OS (if that's feasible).

/var/log/apache2/error.log complains about missing packages, stating "Target WSGI script /vagrant/dashboard.wsgi cannot be loaded as Python Module.

Workaround
pip install emails
pip install email_validator
service apache2 restart

https://www.vagrantup.com/docs/synced-folders/basic_usage.html

Python is completely broken if pyOpenSSL is installed:

sudo python -m easy_install --upgrade pyOpenSSL

I commented this line but there are still errors with missing dependencies (email_validator and emails already reported in a previous issue). Even if you try to install them after the machine was created gives some errors.

Any help appreciated.

put images here for dashboard user guide

A supervisor bug prevents it from starting in 16.04; the recommendation given in this post is to install the newer version from PyPi, but so far that does not seem to fix the problem.

http://serverfault.com/questions/430999/cant-reload-supervisor-with-supervisorctl-reload-command

This error appears every time I try loading the dashboard.

[Wed May 20 06:18:15.509757 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] mod_wsgi (pid=66): Exception occurred processing WSGI script '/vagrant/dashboard.wsgi'.
[Wed May 20 06:18:15.509802 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] Traceback (most recent call last):
[Wed May 20 06:18:15.509823 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1836, in call
[Wed May 20 06:18:15.510232 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] return self.wsgi_app(environ, start_response)
[Wed May 20 06:18:15.510248 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1820, in wsgi_app
[Wed May 20 06:18:15.510268 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] response = self.make_response(self.handle_exception(e))
[Wed May 20 06:18:15.510278 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1403, in handle_exception
[Wed May 20 06:18:15.510291 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] reraise(exc_type, exc_value, tb)
[Wed May 20 06:18:15.510298 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app
[Wed May 20 06:18:15.510309 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] response = self.full_dispatch_request()
[Wed May 20 06:18:15.510316 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1477, in full_dispatch_request
[Wed May 20 06:18:15.510326 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] rv = self.handle_user_exception(e)
[Wed May 20 06:18:15.510332 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1381, in handle_user_exception
[Wed May 20 06:18:15.510342 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] reraise(exc_type, exc_value, tb)
[Wed May 20 06:18:15.510348 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1475, in full_dispatch_request
[Wed May 20 06:18:15.510358 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] rv = self.dispatch_request()
[Wed May 20 06:18:15.510364 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1461, in dispatch_request
[Wed May 20 06:18:15.510388 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] return self.view_functionsrule.endpoint
[Wed May 20 06:18:15.510399 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] File "/vagrant/oii/ifcb2/dashboard/app.py", line 537, in nearest
[Wed May 20 06:18:15.510734 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] resp = canonicalize_bin(bin)
[Wed May 20 06:18:15.510754 2020] [wsgi:error] [pid 66:tid 139854026680064] [remote 172.17.0.1:37964] UnboundLocalError: local variable 'bin' referenced before assignment

If I click on Metrics or other links next to View, I receive an Internal Server Error message with a similar error message in error.logs

local variable 'b' referenced before assignment

Need this in PowerShell

$env:Path = $env:Path + ";C:\Program Files\Git\usr\bin"

It appears that requests has been dropped as a dependency by some package that was being installed in 14.04, so it needs to be added to the provisioning script for 16.04

In Xenial, service status is no longer reported the same way, so do_wait.sh is completely broken.

New status output looks like this:

* postgresql.service - PostgreSQL RDBMS
   Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Tue 2017-03-28 16:40:58 UTC; 4min 15s ago
  Process: 749 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
 Main PID: 749 (code=exited, status=0/SUCCESS)

Mar 28 16:40:23 vagrant systemd[1]: Starting PostgreSQL RDBMS...
Mar 28 16:40:23 vagrant systemd[1]: Started PostgreSQL RDBMS.
Mar 28 16:40:58 vagrant systemd[1]: Stopped PostgreSQL RDBMS.
Mar 28 16:43:11 vagrant systemd[1]: Stopped PostgreSQL RDBMS.

looks easier to parse, should be easy fix.

It's a large package and I'm the only one who uses it, so I can install it myself once the machine is provisioned.

Using sudo cat to modify /etc/fstab results in permission denied. I fixed this by using this instead, but it might be overkill:

sudo bash -c 'cat < /etc/fstab/ <<EOF
//{path-to-mount} /mnt/ifcb cifs username={username},password={password} 0 0
EOF'

The provisioning script currently is hardcoded to use 9.3 so it breaks under Ubuntu 16.04

It no longer works to simply restart supervisor; it needs to be explicitly told to reload the config files, as described in this SO

http://serverfault.com/questions/211525/supervisor-not-loading-new-configuration-files

In the before_first_request hook of the dashboard app, the ORM is initialized, and then on the first request a session is used to query against the ORM which throws up errors that the tables in the ORM schema don't exist.

The session is probably stale and needs to be refreshed after the metadata.create_all call, because if I just restart Apache everything works on restart, which means there's nothing wrong with the way the session is set up except maybe the order of execution.

The behavior is that interactive backend tools will not run, giving a permissions error. This may be platform-specific as it has been observed on a Mac. It likely results from git's behavior during cloning.

A potential workaround is to set permissions on those scripts during provisioning.

This was accidentally omitted, so generated links incorrectly point back at demi.

The trusty32 box apparently has a different directory layout (no /vagrant directory) so the provisioning script fails if you just change the box to trusty32. That could be addressed in several ways including making a separate Vagrantfile, or making the provisioning section "smart" and have it detect which directory layout is in use.