traitecoevo / data_versioning Goto Github PK

• repo name: wcornwell/taxonlookup, traitecoevo/baad.data
• single file name: plant_lookup.csv, baad_data.zip
• hook to load into R: read_csv, baad's bespoke functions
• additional version hook? (not clear, used in baad)

Basically defining the types of problems and the proposed solutions

Subsequently discuss medium versus large data

this arises from @richfitz comments in 384b57b

Does one exist already?

Throughout we use dataset and database somewhat interchangeably. Can I standardise?
My preference is to label any single product being delivered via datastorr a dataset. And reserve use of database for things like genbank.

Thoughts anyone?

for the sake of marketing and communications, i think we should pick a consistent name for the concept we are discussing.

lightweight versioned data (or, LVD in the ms) could work but of course there are many alternatives

Assuming we write a paper, where to submit:

• Scientific Data has an Article section: The ‘Article’ format can be used to present original reports on systems or techniques that clearly advance data sharing and reuse to support reproducible research. This includes research on sharing, managing and processing scientific research data. Articles describing data repositories, standards and ontologies are welcome when they include compelling demonstrations of data exchange, enrichment or knowledge generation made possible by the system or standard.
• PLoS Computtational Biology Research articles must be declared as belonging to one of the following categories: General, Methods or Software. Software articles form a specific sub-category. ...Research articles specifically designated as Methods papers should describe outstanding methods of exceptional importance that have been shown, or have the promise to provide new biological insights. The method must already be widely adopted, or have the promise of wide adoption by a broad community of users. Enhancements to existing published methods will only be considered if those enhancements bring exceptional new capabilities. Methods articles and Software articles require presubmission inquiries.
• Methods in Ecology & Evolution: As an application note. We have a good track record here.

Do we comment on licenses? Currently using MIT (taxonlookup), BSD_2 (baad.data)

data archiving for papers is more-or-less solved. see 384b57b

Maybe we need to pull out the CKAN, Fli, Dat and put it at the end, so that we separate what is possible now, from where things are going in the future?

• Abstract (WC)
• Keywords
• Import Figures #10 (WC)
• Try new figure #11 (DF)
• Discussion (limited). General probelm. An example of solution. Key features (WC, DF)
• MP comments, needs clearer exposition of how this would work. Currently have lists of benefits, but no overview (MP)
• Concise description of what datastorr does (MP)
• Title for paper #13
• Better name for concept #12
• More citations of prior work
• One more review of tables (DF)

what should go here (to replace 'REFS')? https://github.com/traitecoevo/data_versioning/blob/master/ms.tex#L123

options:

• Why We Need Versioned Data and an Easy Workflow for Setting It Up

• Versioned data: why it's needed and how it can be achieved easily, cheaply, and now

There's a lot of prior work in this space, and @cboettig tells me that people have tried this approach and ended up in a mess. Do we solve this problem? Or are we working with data that will always be simple enough to not get in a quagmire? How do people tell when they need to move to something more heavyweight?

Cut?

I really think that Daniel should go first (I thought I added comments on this in my edits but obviously didn't). Daniel and Will have clearly done the most work and are still actively engaged in careers where authorship order matters!

I am thinking something that shows where the different technologies exist

This one is too complex, but that type of thing:

As pointed out by @cboettig, think about what happens with forked data. Do forks get numbers? What happens on a merge? (technically, can gh forks handle releases?)

Seems like a good time to assign an icon to datastorr. Or even a hex sticker.

In #11 I put in a placeholder.

From memory storr is named after the old man of storr, so icon could be an outline of a mountain range?

apparently there's an r package for creating Hex stickers, suitably named hexSticker

At @richfitz -- what's your thoughts on this?

This fig should do two things, explain the numbers:

And also show series of releases, e.g.

Do we want to include examples that are not yet public, like fungaltraits and austraits?

@richfitz: what other examples of people using datastorr you know of?

I'm thinking that this whole paragraph could be cut without much loss as it seems largely repetitive with earlier text.

overall, i think we should keep the discussion/future directions section as short as possible since most of our good points are made throughout the other sections