nextstrain / flora

DB management, APIs, web portals etc.

This repository is archived and contains the content used to build the documentation and splash page found in nextstrain.org. This content can now be found here.

License and copyright

Copyright 2014-2018 Trevor Bedford and Richard Neher.

Source code to Nextstrain is made available under the terms of the GNU Affero General Public License (AGPL). Nextstrain is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

flora's Issues

IO directory names

@jameshadfield and @barneypotter24 ---

I'd like to aim for as much consistency as possible between modules. Currently we have:

• sacra/input/
• sacra/output
• flora/downloaded_data/
• augur/builds/zika/prepared/
• augur/builds/zika/processed/
• augur/builds/zika/auspice/
• auspice/data

I think it makes sense to have augur behave differently, but my preference would be to replace the above with:

• sacra/data/
• flora/data/
• augur/builds/zika/prepared/
• augur/builds/zika/processed/
• augur/builds/zika/auspice/
• auspice/data/

It was always easier for me to think about one directory of data files, where both GenomeFastaResults.fasta is dropped into and zika.fasta is parsed into.

request for code review

@trvrb @barneypotter24

Flora is now working, in that it can create tables on a locally running RethinkDB database, upload JSON documents, and download them as (a) an entire dump of the database (JSON), (b) a FASTA file or (c) a JSON with matched (1-1) sequences and metadata.

https://github.com/nextstrain/flora/ describes how to run, and there's a testing JSON posted on slack.

At this point i'm interested in the general interface, pitfalls to avoid, and directions to go in. Thanks!

Flora requires databases and tables to exist

Currently, running python scripts/run.py --db lassa upload --filename ../sacra/output/lassa.json when there is no existing lassa database results in the following error:

Traceback (most recent call last):
  File "scripts/run.py", line 63, in <module>
    args.func(**vars(args))
  File "/Users/trvrb/Documents/src/flora/scripts/upload.py", line 31, in upload
    modify_db(rdb, table_name, rows, tables_primary_keys[table_name], preview, **kwargs)
  File "/Users/trvrb/Documents/src/flora/scripts/upload.py", line 43, in modify_db
    primary_keys_db = set(rdb.table(table_name).get_all(*primary_keys_sacra).get_field(pkey).run())
  File "/usr/local/lib/python2.7/site-packages/rethinkdb/ast.py", line 123, in run
    return c._start(self, **global_optargs)
  File "/usr/local/lib/python2.7/site-packages/rethinkdb/net.py", line 625, in _start
    return self._instance.run_query(q, global_optargs.get('noreply', False))
  File "/usr/local/lib/python2.7/site-packages/rethinkdb/net.py", line 471, in run_query
    raise res.make_error(query)
rethinkdb.errors.ReqlOpFailedError: Database `lassa` does not exist in:

Just like directories, if a Rethink database doesn't exist. Flora should create it.

Also, even if lassa database exists, flora crashes if the lassa database doesn't already have strains, samples, etc... tables.

Add ? to missing FASTA fields

Currently flora downloads FASTA headers as:

>AY628204|Z148||L|na||

Missing attributes shouldn't be ||. Fauna uses ? for these missing values. Flora download should look like:

>AY628204|Z148|?|L|na|?|

I like ? because this is what BEAST uses for missing attributes, but I would be very happy with null or NA as well. But it should be something.