seattleflu / id3c Goto Github PK

Happy to help if possible.

When running sqitch revert on a recent copy of the seattleflu db from production, I get the following error:

Change "warehouse/target/organism @2019-07-12" required by currently deployed change: seattleflu/id3c-customizations:warehouse/target/data

Truncating warehouse.organism produces this error:

seattleflu=# truncate warehouse.organism;
ERROR:  cannot truncate a table referenced in a foreign key constraint
DETAIL:  Table "target" references "organism".
HINT:  Truncate table "target" at the same time, or use TRUNCATE ... CASCADE.

Running sqitch revert warehouse/kit works, though.

In the current consensus genome etl process we have the function upsert_genomic_sequences that will update a genomic sequence when there's a conflict on genomic_sequence.identifier. However, this identifier is not unique between sequence read sets, which will lead to consensus genomes without genomic sequences linked to them.

Hypothetical situation:

1. sample1, organism1, sequence-read-set1 creates consensus-genome1
   - creates genomic_sequence_1 with identifier sample1-organism1-segment

2. sample1, organism1, sequence-read-set2 creates consensus-genome2
   - the genomic sequence will have the same identifier sample1-organism1-segment so
   will perform update on genomic_sequence_1, which is still linked to consensus-genome1

Writing this all out made me realize that the easiest fix might not be to change the upsert process but to make the genomic sequence identifiers more unique.

Thoughts @tsibley @kairstenfay ?

Hi!

I have some free time this week and would like to attempt upgrading postgresql.

Is this a good idea and time to do this?

Also, should I go for postgresql 12?

I encountered this issue when attempting to run redcap-det generate locally.

I have the following REDCap tokens declared in my environment:

REDCAP_API_URL='https://redcap.iths.org/api/'
REDCAP_SFS_KIOSK_ENROLLMENTS_TOKEN='...'
REDCAP_SFS_HOUSEHOLD_OBSERVATIONAL_TOKEN='...'
REDCAP_SWAB_N_SEND_TOKEN='...'
REDCAP_SINGLE_SWAB_AND_SEND_TOKEN='...'
REDCAP_HOUSEHOLD_INTERVENTION_STUDY_TOKEN='...'

The root of my problem is that I was providing the wrong token name: specifically, I was passing REDCAP_SWAB_AND_SEND_TOKEN instead of REDCAP_SINGLE_SWAB_AND_SEND_TOKEN.

i experienced two different results based on two attempts I made to run the command.

First I ran:

$ REDCAP_API_TOKEN=$REDCAP_SWAB_AND_SEND_TOKEN PGDATABASE=seattleflu pipenv run id3c redcap-det generate --project-id 17561

and received an error of:

requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://redcap.iths.org/api/

Second, I ran:

$ pipenv run id3c redcap-det generate --token-name REDCAP_SWAB_AND_SEND_TOKEN --project-id 17561 > redcap-records.ndjson

and received an error of :

KeyError: 'REDCAP_SWAB_AND_SEND_TOKEN'

I think we could improve the error handling with better messaging here, similar to what @tsibley has already done at https://github.com/seattleflu/id3c/blob/redcap-etl/lib/id3c/cli/redcap.py#L42.