rogertangos / openpds Goto Github PK

It's not that we don't love Mongo, but that it's adding too much complexity to the code. Please checkout a new branch and get this running with just sqlite and postgres.

Decide on an testing framework and start testing data insertion. Since this is going directly into the DB (and not using ORM), you'll need to write some SQL script to remove the data after it's been inserted.

SQLite, Mongo, and Postgres are all tightly connected in a big oms_pds/pds/internal.py file right now. This "SQLiteInternalDataStore", "PostgresInternalDataStore", "InternalDataStore", "DualInternalDataStore" stuff is maddening.

Please make separate files for each type of DB, import it into internal.py, and let a single "InternalDataStore" class use those methods.

This is a mess. We should just cut out support for Mongo at this point, and focus on the postgres db, since this is what datahub will be using.

sqlite should stay (for the time being), just for testing purposes, but there's no need to do any significant work here.

There's good material here. It'd be much better in wiki form.

This is a placeholder, so that you know that I know that the labs' code is incredibly disorganized right now. In an ideal world, each labs' code would be orthogonal, and there would be a few lab helper classes that they could call. (Location helpers, for instance.)

For now, we're putting off these improvements. Once openPDS is tightly connected with DataHub, many of the group queries will be run on the database, and much of the lab code won't be useful anymore. Then, we'll refactor.

os_connector and funf_connector are the only two existing. We need a cookbook for making new connectors.

I really should have done this long ago.