Comments (6)
Originally, REddyProc
has been designed for a defined workflow.
It is not recommended to run gapfilling or partitioning more than once with the same class, instead, instantiate a new object by sEddyProc$new
from the same data. Currently, the code
adds new columns to the results data.frame. This may cause conflicts when columns with the same name are existing. At some instances, the code checks for existing columns, deletes those, and issues the mentioned warning to notify the user that this is probably not what the user wanted.
A redesign would require quite some work and needs to be thoroughly tested.
Hence, for repeated processing, please, instantiate a new object before, or use the functions that are not part of sEddyProc
class. In the second case you need to take care yourself for storing the outputs. In your case you would use function partitionNEEGL
.
Regarding the additional output columns:
The many GPP_suffix and Reco_suffix columns, and the many columns of parameters and intermediate results are already confusing to some users. Hence, we decided to restrict the suffix to the final results.
If you need quantiles of the intermediate output, you need to take care for the output yourself for now between runs with different suffixes.
If you make a strong case for a common use case, that requires the distribution of some specific intermediate results, we could negotiate to add suffixes for some of additional output columns.
from reddyproc.
Thank you for explanations! For me the most important are the parameters of the light response curve as they can be used for further analysis. Though I do not need the quantiles of the parameters, I want to make sure that the one I want to use are not overwritten. I see that it could be difficult to implement. I add my workaround:
# Save colnames to a list after sMRFluxPartition method
colnames <- list()
colnames$FP_MR05 <- colnames(EddyProc.C$sExportResults())
# Lasslop et al. (2010)
suffixes <- c('uStar', 'U05', 'U50', 'U95', 'UNone')
FP_temp <- 'Tsoil'
saveRDS(EddyProc.C, file = "EddyProc.C.rds")
FP_GL10_out <- EddyProc.C$sExportResults()[, 0]
for (i in suffixes) {
rm(EddyProc.C)
EddyProc.C <- readRDS("EddyProc.C.rds")
EddyProc.C$sGLFluxPartition(
TempVar.s = paste(FP_temp, "_f", sep = ""),
QFTempVar.s = paste(FP_temp, "_fqc", sep = ""),
Suffix.s = i)
if (i == 'uStar') saveRDS(EddyProc.C, file = "EddyProc.C_GL10_uStar.rds")
out <- EddyProc.C$sExportResults()
cols <- colnames(out)
cols_out <- cols[!cols %in% colnames$FP_MR05]
out <- out[cols_out]
corr_filter <- !cols_out %in% grep(paste(suffixes, collapse = "|"),
cols_out, value = TRUE)
names(out)[corr_filter] <- paste(cols_out[corr_filter], i, sep = '_')
FP_GL10_out <- cbind(FP_GL10_out, out)
}
rm(EddyProc.C)
EddyProc.C <- readRDS("EddyProc.C_GL10_uStar.rds")
colnames$FP_GL10 <- colnames(FP_GL10_out)
The output that I want is then kept in FP_GL10_out
. I also did not notice before that the active binding in EddyProc.C
leads to "cloning" when the object is resaved under different name. The results of computations done on the new object will be also transferred to the original one. Thus I better save the object as a file and reload it after each iteration. I hope it could also help to others.
from reddyproc.
Hello @lsigut.
Have a look at pull request #35.
I added argument uStarScenKeep
to sEddyProc_sApplyUStarScen
and the bootstrapped partitioning methods.
By default results for the first uStar threshold scenario (usually the non-bootstrapped) are kept.
Please, comment on if this resolves your needs and on the interface (argument naming ...).
from reddyproc.
Hello @progtw,
thanks for having a look into it! I was just expecting that the solution would be adequate to nighttime-based partitioning where all scenario-specific variables are renamed using the suffix system. I was expecting that this would be the only needed step - to evaluate which columns are scenario specific, attach suffix and let those that are the same for all scenarios to rewrite without warning. All such outputs would be then possible to get when extracting results. But of course I do not know the internal design of sGLFluxPartition
. I am not sure if it is not too complicated to use it now. The test_that
example is not well readable for me. Some example in vignette could help!
from reddyproc.
The complications for this proposal for the design are one issue, but the main I repeat:
The many GPP_suffix and Reco_suffix columns, and the many columns of parameters and intermediate results are already confusing to some users. Hence, we decided to restrict the suffix to the final results.
In hindsight, it might have been better also for the night-time partitioning to report only the gross fluxes with suffixes and the others outputs only for one scenario, but I do not want to change the interface now.
In your workaround you demonstrate how to take care of this using the Suffix
argument for other ouptuts in your user-loop:
for (suffix in scenarios) {
EProc$sGLFluxPartition(..., Suffix = suffix)
myStorageOfCols[,suffix] <- EProc$sExportResults()[[interestingColName]]
}
But for your purpose, as I understood, you need only the output of a single specific uStar threshold scenario and this is now supported with the suggested PR directly.
In the PR (pull request) #35 I extended the example in the vignette uStarCases.Rmd
by daytime partitioning and added some explanation.
from reddyproc.
Great, thanks for the explanations, also for the vignette!
from reddyproc.
Related Issues (20)
- provide function to get current uStar suffixes
- replace deprecatd select_
- avoid warning on replacing columns during repeated flux partitioning across uStar scenarios
- export to data.frame in Fluxnet15 format
- omit LRC uncertainty bootstrap on uStar scenarios except one
- estimate VPD of long gaps from minimum temperature
- assign default units when loading fluxnet15 formatted files
- do no not break in LRC fitter on error when inverting singular Hessian
- Spare fit including VPD effect, if there is no large VPD in daily dataset
- allow suppressing progress output in day-time partitioning
- detail the violation when checking for numeric columns
- about footprint modeling HOT 2
- "Time stamp is not equidistant" when data are indeed half-hourly HOT 1
- change default of argument uStarScenKeep
- sEddyProc_sPlotNEEVersusUStarForSeason throws warning
- Remove strong dependency on mlegp
- Error in fConvertTimeToPosix -> missing value where TRUE/FALSE needed HOT 1
- Provide comma-delimited ASCII output HOT 1
- provide possibility for user-defined gapfilling function used after uStar filtering HOT 1
- How to generate GPP without dependence on the Ustar? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from reddyproc.