csiro-crop-informatics / fieldprime_server

REST-API for the data collection android application fieldprime_app

FieldPrime

This is the FieldPrime server REST-API for the FieldPrime scoring app.

It is a python/flask web application with a mysql backend.

Building and Deploying:

The FieldPrime server can be deployed manually by copied the required files and setting up apache/mysql. It can also be built with docker/docker-compose simply by:

docker-compose build docker-compose up

Configuration:

Default configutaion options (fpa/fp_common/default_config.py) can be over-ridden in fpa/fp_common/config.py

SSL certificates should be placed in sslCerts for surver to be able to use https

fieldprime_server's Issues

Missing directories in fpa

The fpa/Dockerfile at line #6 tries to copy fp_app_api into the docker image:

COPY fp_app_api /app/fp_app_api

The same problem occurs on line #22 with the sslStuff directory:

COPY sslStuff /app/sslStuff

The problem is that the fp_app_api and sslStuff directories don't seem to exist in the repository. Are these built elsewhere?

Explicit help in setup

Hi all

I'm having trouble knowing how to what variables need to be changed in order to get FieldPrime up and running.

The README.md explains that

Default configutaion options (fpa/fp_common/default_config.py) can be over-ridden in fpa/fp_common/config.py

But it looks like there are a couple of other options that need to be changed:

$ grep -r REMOVED .
./fpa/fp_common/default_config.py:SECRET_KEY = '** REMOVED **'   # NB FieldPrime also uses this explicitly
./fpa/fp_web_admin/fpWebAdmin.py:app.secret_key = '** REMOVED **'
./fp_mysql/sql/init.sql:  '** REMOVED **', 

In ./fp_mysql/sql/init.sql, it looks to be setting up a passhash for the default user 'mk'. The script ./fp_mysql/scripts/make_passhash.py looks to generate hashes using the passlib.apps library, so I tried:

python ./fp_mysql/scripts/make_passhash.py foobar
$6$rounds=710330$TsTPfN1CJMTBJa2B$X62x7s8g2Ud.mmqevBCj1J1.68i/xtqJypGFvfXjQILsE3EDFxKary8.ElIKTCSBpccOzQJu01oVKuDCKe6pX1

which I then used to replace ** REMOVED ** in fp_mysql/sql/init.sql to give:

source /fieldprime/fpsys.sql

grant all on *.* to fpwserver@'%' identified by 'bar';
flush privileges;

set @DB_NAME='fp_main';
set @PROJ_NAME='main';
set @CONTACT_NAME='Michael Kirk';
set @CONTACT_EMAIL='[email protected]';
create database if not exists fp_main;
use fp_main;
source /fieldprime/fprime.create.tables.sql;
insert fpsys.project (name, dbname) values (@PROJ_NAME, database());
insert project (id, up_id, name, contactName, contactEmail)
  values ((select id from fpsys.project where name=@PROJ_NAME),
  null, @PROJ_NAME, @CONTACT_NAME, @CONTACT_EMAIL);
insert system (name, value) values ('contactName', @CONTACT_NAME), ('contactEmail', @CONTACT_EMAIL);

use fpsys
insert user (login,name,passhash,login_type,permissions) values ('mk','m k',
  '$6$rounds=710330$TsTPfN1CJMTBJa2B$X62x7s8g2Ud.mmqevBCj1J1.68i/xtqJypGFvfXjQILsE3EDFxKary8.ElIKTCSBpccOzQJu01oVKuDCKe6pX1', 
  3, 1);
insert userProject values (1,1,1);

Then running

docker-compose build && docker-compose up

Gets fieldprime up and running, but logging in on the main page with credentials mk:foobar gives a 500 Internal Server Error stemming from pwd_context.verify(password, phash) in fp_common/fpsys.py:

fp_server      | [Thu Mar 01 04:59:20.051222 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010] mod_wsgi (pid=22): Exception occurred processing WSGI script '/tmp/mod_wsgi-localhost:80:1001/htdocs/'.
fp_server      | [Thu Mar 01 04:59:20.051360 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010] Traceback (most recent call last):
fp_server      | [Thu Mar 01 04:59:20.051418 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]   File "/usr/local/python/lib/python2.7/site-packages/mod_wsgi/server/__init__.py", line 1490, in handle_request
fp_server      | [Thu Mar 01 04:59:20.054770 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]     return self.application(environ, start_response)
fp_server      | [Thu Mar 01 04:59:20.054820 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]   File "/usr/local/python/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__
fp_server      | [Thu Mar 01 04:59:20.055848 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]     return self.wsgi_app(environ, start_response)
fp_server      | [Thu Mar 01 04:59:20.055888 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]   File "/usr/local/python/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
fp_server      | [Thu Mar 01 04:59:20.056254 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]     response = self.make_response(self.handle_exception(e))
fp_server      | [Thu Mar 01 04:59:20.056287 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]   File "/usr/local/python/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception
fp_server      | [Thu Mar 01 04:59:20.056527 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]     reraise(exc_type, exc_value, tb)
fp_server      | [Thu Mar 01 04:59:20.056559 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]   File "/usr/local/python/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
fp_server      | [Thu Mar 01 04:59:20.056798 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]     response = self.full_dispatch_request()
fp_server      | [Thu Mar 01 04:59:20.056830 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]   File "/usr/local/python/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
fp_server      | [Thu Mar 01 04:59:20.057065 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]     rv = self.handle_user_exception(e)
fp_server      | [Thu Mar 01 04:59:20.057097 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]   File "/usr/local/python/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
fp_server      | [Thu Mar 01 04:59:20.057383 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]     reraise(exc_type, exc_value, tb)
fp_server      | [Thu Mar 01 04:59:20.057421 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]   File "/usr/local/python/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
fp_server      | [Thu Mar 01 04:59:20.057815 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]     rv = self.dispatch_request()
fp_server      | [Thu Mar 01 04:59:20.057864 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]   File "/usr/local/python/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
fp_server      | [Thu Mar 01 04:59:20.058196 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]     return self.view_functions[rule.endpoint](**req.view_args)
fp_server      | [Thu Mar 01 04:59:20.058237 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]   File "/app/fp_web_admin/fpWebAdmin.py", line 1898, in urlMain
fp_server      | [Thu Mar 01 04:59:20.059204 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]     authOK = fpsys.userPasswordCheck(username, password)
fp_server      | [Thu Mar 01 04:59:20.059248 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]   File "/app/fp_common/fpsys.py", line 465, in userPasswordCheck
fp_server      | [Thu Mar 01 04:59:20.059767 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]     return pwd_context.verify(password, phash)
fp_server      | [Thu Mar 01 04:59:20.059806 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]   File "/usr/local/python/lib/python2.7/site-packages/passlib/context.py", line 2553, in verify
fp_server      | [Thu Mar 01 04:59:20.060967 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]     record = self._get_or_identify_record(hash, scheme, category)
fp_server      | [Thu Mar 01 04:59:20.061018 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]   File "/usr/local/python/lib/python2.7/site-packages/passlib/context.py", line 2260, in _get_or_identify_record
fp_server      | [Thu Mar 01 04:59:20.061343 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]     return self._identify_record(hash, category)
fp_server      | [Thu Mar 01 04:59:20.061390 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]   File "/usr/local/python/lib/python2.7/site-packages/passlib/context.py", line 1457, in identify_record
fp_server      | [Thu Mar 01 04:59:20.061446 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010]     raise ValueError("hash could not be identified")
fp_server      | [Thu Mar 01 04:59:20.061827 2018] [wsgi:error] [pid 22:tid 140210303379200] [remote 134.7.58.56:56010] ValueError: hash could not be identified

Am I copying in the wrong string into the init.sql file? What format should the passhash entry have? Can you give an example?

Thanks!