mobleylab / guthriesolv Goto Github PK

There are a few major issues with CSV files:

1. Commas may pop up inside data, which means cells need to be quoted
2. There are many different flavors of quoting schemes. This is very difficult to convey
3. Commas themselves create a huge amount of noise when reading a CSV document

TSV uses tabs instead of commas and has none of these problems as tabs should not be in the data itself.

I think it should be possible to directly import and export the table through pandas.

Should set up tests that attempt to process the main csv file and (possibly) ensure that all the SMILES can be processed and convert to the specified names

Right now it's pretty hard to figure out what's actually in the database - it would be great to have an explanation of what each column is in the README as well as the kinds of data that might be in each (especially the categorical fields, like for the measurement types)

Typically, people expect DOIs, PubMed identifiers (PMIDs), or arXiv identifiers for publications. The bibliography in this repository is a bit old school, which means that the provenance information associated with each row of the database isn't really machine-readable nor actionable. It would be a great contribution if someone could untangle all of the references in https://github.com/MobleyLab/GuthrieSolv/blob/master/guthrie_references_and_status.csv and replace them with DOIs or PMIDs.

@davidlmobley I was thinking of how I would go about cleaning up/exploring the data in this repo now that there's a little infrastructure, but it occurred to me that this would be an excellent opportunity to mentor an excellent undergraduate or green graduate student on how practical data science goes (beyond using the iris dataset 😆) and a bit of machine learning. Let me know if someone immediately comes to mind, otherwise I will continue taking a crack at it myself