iscn / socdrahr2 Goto Github PK

• evaluate dataset
• complete template checklist
• write up summary report narrative
• good commenting
• external review by someone else

• Evaluate dataset
• merge study data
• clean up profile data
• clean up layer data
• make graphics
• Write up summary report narrative
• Good commenting
• External review by someone else

see 07dc166

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

• evaluate dataset
• fix cryptic citation in site notes
• write up summary report narrative
• good commenting
• external review by someone else

The ISCN database takes a long time to load. Is there anyway to speed this up?

bce72b2

Also looking for ways to reduce the file size.

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

Add documentation to the variable table from ISCN3. Add a description and any common units to the master variable table. Including:

• al_dith
• al_other
• al_ox
• base_sum
• bs
• bs_sum
• bulk_density_other
• bulk_density_sample
• bulk_density_total
• bulk_density_whole
• ca_al
• ca_ext
• cec_sum
• data_collection
• dataset_type
• ecec
• ecoregion
• fe_dith
• fe_other
• fe_ox
• h_ext
• horizon
• horizon_designation
• horizon_designation_other
• k_ext
• landuse
• latitude
• layer_ISCN_soc_count
• layer_bottom
• layer_carbon_count
• layer_count
• layer_soc_count
• locator_alias
• locator_parent_alias
• longitude
• mg_ext
• mn_dith
• mn_other
• mn_ox
• na_ext
• p_bray
• p_meh
• p_other
• p_ox
• profile_ISCN_soc_count
• profile_bottom
• profile_count
• profile_soc
• profile_soc_carbon_flag
• profile_soc_count
• profile_soc_depth
• profile_soc_spatial_flag
• profile_top
• profile_zero_ref
• site_ISCN_soc_count
• site_count
• site_soc_count
• zn_ext

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

The header data for the Pangaea download could use a robust parser. Below is an example of a hard coded parser for the C-PEAT project but it sure does feel like a regular data tree with a general solution somewhere.

https://github.com/ISCN/soilDataR/blob/c869479a3d01f1cce0209ab99e5daf84ef8c6d47/R/readCPEAT.R#L146

• Evaluate dataset
• merge study data
• clean up profile data
• clean up layer data
• make graphics
• Write up summary report narrative
• Good commenting
• External review by someone else

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

When trying to run the code through are on my Windows operating system, I run into many errors with the trying to run functions that have to open excel files and anywhere where the code calls for lines with "/n". I believe this is because Windows uses "/n/r" to indicate both new lines and return.

• Evaluate dataset
• Fix maps
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else
• Determine where temperature/precipitation data is (not found in ISCN)

• Evaluate dataset
• merge study data
• clean up profile data
• clean up layer data
• make graphics
• Write up summary report narrative
• Good commenting
• External review by someone else!

• evaluate dataset
• deal with two citations
• fix cryptic vegation
• write up summary report narrative
• good commenting
• external review by someone else

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

435 topsoil samples (0–10 cm); network includes 87 globally distributed plots and five soil replicates per plot

Delgado-baquerizo, Manuel; Ochoa-Hueso, Raúl (2020): Dataset of "Soil element coupling is driven by ecological context and atomic mass". figshare. Dataset. https://doi.org/10.6084/m9.figshare.13138811.v1 with paper citation: https://onlinelibrary.wiley.com/doi/full/10.1111

• Evaluate dataset
• merge with profile level contributions
• figure out citation/referencing/study level attributions
• merge study data
• clean up profile data
• clean up layer data
• make graphics
• Write up summary report narrative
• Good commenting
• External review by someone else

About 800 samples with P measured using Hadley method (physiochemical fractionation). Scrapped both the Chinese and English literature. We would need to expand the fractionation descriptions.

Hou, E., Tan, X., Heenan, M. et al. A global dataset of plant available and unavailable phosphorus in natural soils derived by Hedley method. Sci Data 5, 180166 (2018). https://doi.org/10.1038/sdata.2018.166

• Evaluate dataset
• merge study data
• clean up profile data
• clean up layer data
• make graphics
• Write up summary report narrative
• Good commenting
• External review by someone else

I just uploaded the HeckmanSwansonBiscuitBurn.rmd in commit 73284e1 can you check it out when you get the chance @ktoddbrown

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

• Evaluate dataset
• merge study data
• clean up profile data
• clean up layer data
• make graphics
• Write up summary report narrative
• Good commenting
• External review by someone else

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

Currently the package structure is messy.

Currently identified tasks:

• dependent libraries need to be imported not called via 'library' in the R code.

• better and revised function documentation

Commented one of the tables on the report, starting at line 66. see commit 1ade55f @ktoddbrown

• Evaluate Template
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

Treatments need to be pulled out to their own data frame.

Given the below example

with this key:

the process from template function breaks. You can hack it by renaming the profiles uniquely and treating the treatment as if it was a measurement, see the soils-long-tail-recovery/hackathon/example.*

I substituted out the select function that we were calling before and added updated comments. see commit 71ec46f @ktoddbrown

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

We need to take the three table key developed here: https://github.com/ESIPFed/soil_data_model_survey/tree/master/data and rewrite the data harmonization script. In theory this should be pretty straight forward.

Working on the threeTblKey branch we need to do the following

• move the readKeyedData to readOneTableKeyData and make a note that this is depreciated.
• transfer existing data keys to new format
• check runtime for data processes, can we make this a general solution and move away from one-off large data processing with the new tidyverse?

Investigate https://rstudio.github.io/distill/website.html to create a webpage front for this repo.

We deleted the duplicate dataset_layer and dataset_profile. See commit edb3b06 @ktoddbrown

• Finalize ISCN 3_1 function to reflect template processing in markdown document
• Merge markdown documents into one master markdown
  • replace processing script with call to ISCN 3_1 function
  • generalize maps (include spreadsheet detailing ISCN 3.5 individualized map code revisions)
• Adding a QAQC function to ISCN 3_1 to catch one-off data cleaning in datasets
• Add a QAQC script to look at bounds in columns to apply to multiple ISCN versions
• Metadata annotation for archiving on EDI https://environmentaldatainitiative.org/
• Track down and organize data citations and reingest requests
• Update Github webpage for package

• Evaluate dataset
• merge study data
• clean up profile data
• clean up layer data
• make graphics
• Write up summary report narrative
• Good commenting
• External review by someone else

• Evaluate dataset
• Complete template checklist
• Write up summary report narrative
• Good commenting
• External review by someone else

• Evaluate dataset
• merge study data
• clean up profile data
• clean up layer data
• make graphics
• Write up summary report narrative
• Good commenting
• External review by someone else

see d6d4e6a

Link below of spreadsheet with summary of changes made between ISCN3 to ISCN3.5, as well as status, contributor, file name, and other notes.
https://docs.google.com/spreadsheets/d/1x9ZC6zcAhcCtkrzO2jd-LtBsPMFVT_0LPqnB-v1w-ZM/edit?usp=sharing

Do we have this in ISCN?
http://icp-forests.net/page/level-ii

Apparently they have soil chem data from almost 800 forests? Though admittedly, I have no clue what soil chem actually includes, but can always find out.