Can someone assist, am having problems to install climdex.pcic.ncdf on R-3.2

Warning in untar2(tarfile, files, list, exdir, restore_times) :
skipping pax global extended headers

• installing to library ‘/home/cchisanga/R/x86_64-pc-linux-gnu-library/3.2’
• installing source package ‘climdex.pcic.ncdf’ ...
  ERROR: a 'NAMESPACE' file is required
• removing ‘/home/cchisanga/R/x86_64-pc-linux-gnu-library/3.2/climdex.pcic.ncdf’

Dear all,
my name is Fabian Wachsmann and I am working in a CMIP6 project (named DICAD) at the German Climate Computing Center. I am developing the CDO tool and try to enhance it to be usable for CMIP6 data.

One objective is to build in the climate extremes indices calculation methods recommended by the ETCCDI and used within this Rclimdex package. As a first step, I compared the climate indices calculated with the Rclimdex package with the ones calculated with CDOs.

While doing that, some questions emerged and I hoped one of you could help me with these ones:

1. How are different calendars processed in the Rclimdex? I have a CMIP5 model output time series as an input file which uses a Gregorian calendar. However, the threshold output of the Rclimdex only includes 365 timesteps although there are multiple years in that time series with 366 days. Is the 29th of Feb. skipped?

2. When calculating cdd and sdii, the Rclimdex output include nan's. Do you know where they come from? Since the maximum value in the field of the first time step of cdd is 364 instead of 365 days (which actually should be in there) I suspect that this could cause from a initialization within Rclimdex.

3. The quantile calculation method explained in Zhang et al. (2005) corresponds to the R quantile function with type 6 (see https://stat.ethz.ch/R-manual/R-devel/library/stats/html/quantile.html ). However, in the documentation (https://cran.r-project.org/web/packages/climdex.pcic/climdex.pcic.pdf ) it says for climdex.quantile that "This is a reimplementation of R’s type=8". What quantile estimation is used in Rclimdex eventually?

4. What is the recent status of a metadata standard for the climate indices? I think, with respect to the Rclimdex output, there have to be corrections. For example, variables of the Rclimdex output are named *ETCCDI which cannot be compliant to a convention.

Sorry for putting them all together. Please let me know if there is a better place to ask these questions and to whom I can address them.

Best regards and thank you very much for an answer,
Fabi

I am trying to use Climpact to calculate ETCCDI indices on netcdf files in ubuntu. I have followed the procedure given in user guide but unable to install. The error showing ERROR: dependencies ‘ncdf4’, ‘ncdf4.helpers’ are not available for package ‘climdex.pcic.ncdf’ @heroldn @jmccomb

Please help me as this is urgent,
i have been battling with computation of indices using climdex.pcic.ncdf

I get the error below
Rscript climpact2.ncdf.wrapper1.r
Loading required package: PCICt
Loading required package: methods
Loading required package: lmomco
Loading required package: parallel

Package SPEI (1.6) loaded [try SPEINews()].

Error in ncdf4.helpers::nc.get.dim.for.axis(f.example, v.example, "T")$name :
$ operator is invalid for atomic vectors
Calls: create.indices.from.files -> create.ncdf.output.files
Execution halted

Would it be possible to release this package on CRAN?

The reason I'm asking is that we're trying to use this package with ESMValTool, so we would like to install it as a dependency. However, to ensure the quality and reliability of ESMValTool we can only allow packages that can be installed from official repositories as dependencies.

Help!

I am getting this error in proj4::project which is called by create.indices.from.files. Any idea what the reason might be? My netCDF files use a Lambert Conformal Conic projection.

Error in proj4::project(list(x = x.subset.vals, y = y.subset.vals), projection,  : 
  major axis or radius = 0 or not given

This is the output from ncdump for one of the netCDF files for the variable that defines the projection (lambert_conformal_conic). I added the last two attributes to the original file (reference_ellipsoid_name and earth_radius) myself to see if that helped, but I still get the above error. Can anyone help? Thanks!

    short lambert_conformal_conic ;
            lambert_conformal_conic:grid_mapping_name = "lambert_conformal_conic" ;
            lambert_conformal_conic:longitude_of_central_meridian = -100. ;
            lambert_conformal_conic:latitude_of_projection_origin = 42.5 ;
            lambert_conformal_conic:false_easting = 0. ;
            lambert_conformal_conic:false_northing = 0. ;
            lambert_conformal_conic:standard_parallel = 25., 60. ;
            lambert_conformal_conic:semi_major_axis = 6378137. ;
            lambert_conformal_conic:inverse_flattening = 298.257223563 ;
            lambert_conformal_conic:reference_ellipsoid_name = "WGS 84" ;
            lambert_conformal_conic:earth_radius = 6378137. ;

Please, anybody can help me
I have an issue using "climdex.pcic.ncdf"
The error message is:
package ‘climdex.pcic.ncdf’ is not available (for R version 3.1.2)

Hi @corviday ,
I was wondering whether you could help me with the following issue:
Whenever I try to extract a spcified index (e.g. rx5day) from a file (infile)

infile <- "pr_day_UKESM1-0-LL_ssp585_r1i1p1f2_gn_20150101-20181230.nc"
threshfile <- "UKESM1-0-LL_historical_r1i1p1f2_gn.thresh_1981-2010.nc"

create.indices.from.files(infile, 'output_path', output.filename.template=infile, author.data, climdex.vars.subset = "rx5day", climdex.time.resolution = c("annual"), axis.to.split.on = "Y", fclimdex.compatible = TRUE, base.range = c(1981,2010), parallel = 6, verbose = TRUE, thresholds.files = threshfile, max.vals.millions = 20, cluster.type = "SOCK")

the process fails with the following error message:

Error in check.quantile.validity(quantiles, present.var.list, days.in.base) :
Quantiles must be present for all variables provided.

However, why are quantiles needed in the first place, since Rx5day does not rely on quantiles? Second, all quantiles as computed by create.thresholds.from.file are part of the threshold file threshfile : tx10thresh tx90thresh r95thresh r99thresh. Thus it should actually have all the necessary input right?

However, if I extract the indices the very same way, but from the input file infile <- "pr_day_UKESM1-0-LL_historical_r1i1p1f2_gn_19710101-20101230.nc" the original thresholds were calculated with, then it works with specifying only one index.

Thank you very much!

I wish to compute the Consecutive wet and dry days with custom threshold of 2.5 mm. The default threshold is 1 mm. Is it possible ?

Many thanks for your great work.

When I use this tool for rainfall data from observation, I surprisingly find that it requires a unit of kg m-2 d-1, and it can't take mm. I checked the code but can't find an option to change this requirement. Is it possible to make this controlled by a switch? After all, people may work on model outputs or observational datasets.

Hello,

what are the units of precipitation and temperature to be used as input for computing the climpact indices?

I am using the climdex.pcic.ncdf R package with climate model data. My precipitation data have the unit kg*m-2*s-1 and temperature K. Is this correct?

Thanks

Dear all,

I run "create.indices.from.files" in a single cpu and it works fine. When I try to execute the "create.indices.from.files" assigning multiple threads to the calculation, I get the following error regarding ncdf4:

Error in R_nc4_create: Permission denied (creation mode was 4096)
Error in ncdf4::nc_create(paste(out.dir, cdx.dat$filename[x], sep = "/"), :
Error in nc_create!

Any ideas?

Thanks in advance!

Josep Maria

Hello Everybody,
I am trying to compute my data in the climdex.pcic.ncdf package, But I have problems because I don't have the information in the proper format.

In this sense, how I prepare my information to work with the climdex.pcic.ncdf package?

Thank you!

Hello @bronaugh @jameshiebert @corviday,

quick question:

does the R package climdex.pcic.ncdf convert automatically precipitation from kg m-2 s-1 to kg m-2 d-1?

Thanks

Hello everyone,
I have a problem with the calculation indices with new thresholds using thresholds contained in a specific file with climdex.pcic.ncdf package.
I can calculate the thresholds but I Can not calculate the indices with new thresholds
This is the error:

Error in FUN(X[[i]], ...) :
Temperature in-base quantiles must contain 10th and 90th percentiles.

Please someone can help me?
Many thanks in advance.

I wrote the next script.

require(climdex.pcic.ncdf)
###calculate thresholds --- I can calculate the thresholds
input.files <- c("tasmax_day_rcp85-area_r1i1p1_1-CanESM2_20160101-20351231.nc","tasmin_day_rcp85-area_r1i1p1_1-CanESM2_20160101-20351231.nc","pr_day_rcp85-area_r1i1p1_1-CanESM2_20160101-20351231.nc")
author.data <- list(institution="University")
create.thresholds.from.file(input.files, "thresholds.nc", author.data, base.range=c(2016,2025), parallel=FALSE)

###calculate indices with new thresholds – I can not calculate the indices with new ###thresholds

input.files <- c("tasmax_day_rcp85-area_r1i1p1_1-CanESM2_20160101-20351231.nc","tasmin_day_rcp85-area_r1i1p1_1-CanESM2_20160101-20351231.nc","pr_day_rcp85-area_r1i1p1_1-CanESM2_20160101-20351231.nc")
author.data <- list(institution="University"")
create.indices.from.files(input.files, "Results/", input.files[1], author.data, base.range=c(2016, 2025), parallel=8, thresholds.files="thresholds.nc", max.vals.millions = 10, cluster.type = "SOCK")

I am trying to compute the Rx5day values for a netcdf file and I keep getting this error:
Error in min(ts) : invalid 'type' (closure) of argument

The code I've been using is:
input.files <- c("WAH_PRCP_198601_201411_daily_60mem_360monthly_fldmean_mm.nc")
author.data <- list(institution="Looney Bin", institution_id="LBC")
create.indices.from.files(input.files, "output.dir/",
input.files[1], author.data,
climdex.vars.subset = ("rx5day"),
climdex.time.resolution = c("monthly"),
base.range=c(1986, 2014),
variable.name.map = c(tmax = "tasmax", tmin = "tasmin", prec = "PRCP", tavg = "tas"),
parallel=FALSE)

Any help you can offer to help me extract the Rx5day values will be greatly appreciated,

Thank you.

Hi James

the NAMESPACE file is missing for the install_github process
exportPattern( "." )

Apparently, the 'NAMESPACE' file of the package is missing.

Hello,

I have a problem with the computation of indices from a netcdf file using thresholds contained in a specific file. I first encountered this problem on my own set of data but even with the test file given in the "sample_data" directory, I'm not able to compute the different indices.
I followed the next procedure :

STEP 1 :

I create the thresholds file from the netcdf file given in the "sample_data" directory ("./sample_data/climpact2.sampledata.gridded.1991-2010.nc") with the "climpact2.ncdf.thresholds.wrapper.r" file.
Here what's in this file for me :

=========================== : BEGIN OF FILE "climpact2.ncdf.thresholds.wrapper.r"
`

------------------------------------------------

This wrapper script calls the 'create.thresholds.from.file' function from the modified climdex.pcic.ncdf package

to create thresholds, using data and parameters provided by the user.

------------------------------------------------

library(climdex.pcic.ncdf)

list of one to three input files. e.g. c("a.nc","b.nc","c.nc")

#input.files = c("./NCDF_DATA/CLIMPACT2_PRESENT.nc")
input.files = c("./sample_data/climpact2.sampledata.gridded.1991-2010.nc")

list of variable names according to above file(s)

vars=c(tmax="tmax", tmin="tmin", prec="precip")

output file name

#output.file = "./output_thresholds/thresholds_2000-2015_from_PRESENT.nc"
output.file = "./output_thresholds/thresholds.nc"

author data

author.data=list(institution="IRD", institution_id="MU")

reference period

base.range=c(1995,2005)

number of cores to use (or FALSE)

parallel = FALSE

print messages?

verbose=TRUE

######################################

Do not modify without a good reason.

fclimdex.compatible=FALSE

create.thresholds.from.file(input.files,output.file,author.data,variable.name.map=vars,base.range=base.range,parallel=parallel,verbose=verbose,fclimdex.compatible=fclimdex.compatible)
`
=========================== : END OF FILE "climpact2.ncdf.thresholds.wrapper.r"

Once this R script is ran, I obtain the thresholds file stored in the "output_thresholds" directory : "./output_thresholds/thresholds_TEST.nc". This file of 98M contained the different thresholds variables computed by the former R script. Here an the result that I get after apllying the command "ncdump -h output_thresholds/thresholds.nc | grep double " (I pipe the whole result through the grep command to avoid all the file details. I can give you the complete result if needed. The result showed here allows us to see which thresholds variables are present in the thresholds file...) :

"
double time(time) ;
double time_bnds(time, bnds) ;
double lon(lon) ;
double lat(lat) ;
double tx05thresh(time, lat, lon) ;
double tx10thresh(time, lat, lon) ;
double tx50thresh(time, lat, lon) ;
double tx90thresh(time, lat, lon) ;
double tx95thresh(time, lat, lon) ;
double tn05thresh(time, lat, lon) ;
double tn10thresh(time, lat, lon) ;
double tn50thresh(time, lat, lon) ;
double tn90thresh(time, lat, lon) ;
double tn95thresh(time, lat, lon) ;
double tx90thresh_15days(time, lat, lon) ;
double tn90thresh_15days(time, lat, lon) ;
double tavg90thresh_15days(time, lat, lon) ;
double tavg05thresh(lat, lon) ;
double tavg95thresh(lat, lon) ;
double r95thresh(lat, lon) ;
double r99thresh(lat, lon) ;
double base_time(base_time) ;
double txraw(base_time, lat, lon) ;
double tnraw(base_time, lat, lon) ;
double precraw(base_time, lat, lon) ;
"

STEP 2 :

I now want to use this theshold file to compute indices from the same original netcdf file. I use the R script file "climpact2.ncdf.wrapper.r" for that purpose. Here the content of this file :

=========================== : BEGIN OF FILE "climpact2.ncdf.wrapper.r"
`
====# -------------------impact2.ncdf.wrapper.r----------------------------

This wrapper script calls the 'create.indices.from.files' function from the modified climdex.pcic.ncdf package

to calculate ETCCDI, ET-SCI and other indices, using data and parameters provided by the user.

Note even when using a threshold file, the base.range parameter must still be specified accurately.

------------------------------------------------

library(climdex.pcic.ncdf)

list of one to three input files. e.g. c("a.nc","b.nc","c.nc")

infiles="./sample_data/climpact2.sampledata.gridded.1991-2010.nc"

list of variable names according to above file(s)

vars=c(prec="precip",tmax="tmax", tmin="tmin")

output directory. Will be created if it does not exist.

outdir="./output/"

Output filename format. Must use CMIP5 filename convention. i.e. "var_timeresolution_model_scenario_run_starttime-endtime.nc"

file.template="var_test_test_test_test_1995-2005.nc"

author data

author.data=list(institution="My University", institution_id="MU")

reference period

base.range=c(1995,2005)

number of cores to use, or FALSE for single core.

cores=FALSE

list of indices to calculate, or NULL to calculate all.

indices=NULL #NULL #c("hw","tnn")

input threshold file to use, or NULL for none.

#thresholds.files=NULL
thresholds.files="./output_thresholds/thresholds.nc"

#######################################################

Esoterics below, do not modify without a good reason.

definition used for Excess Heat Factor (EHF). "PA13" for Perkins and Alexander (2013), this is the default. "NF13" for Nairn and Fawcett (2013).

EHF_DEF = "PA13"

axis to split data on. For chunking up of grid, leave this.

axis.name="Y"

Number of data values to process at once. If you receive "Error: rows.per.slice >= 1 is not TRUE", try increasing this to 20. You might have a large grid.

maxvals=10

output compatible with FCLIMDEX. Leave this.

fclimdex.compatible=FALSE

Call the package.

create.indices.from.files(infiles,outdir,file.template,author.data,variable.name.map=vars,base.range=base.range,parallel=cores,axis.to.split.on=axis.name,climdex.vars.subset=indices,thresholds.files=thresholds.files,fclimdex.compatible=fclimdex.compatible,
cluster.type="SOCK",ehfdef=EHF_DEF,max.vals.millions=maxvals,
thresholds.name.map=c(tx05thresh="tx05thresh",tx10thresh="tx10thresh", tx50thresh="tx50thresh", tx90thresh="tx90thresh",tx95thresh="tx95thresh",
tn05thresh="tn05thresh",tn10thresh="tn10thresh",tn50thresh="tn50thresh",tn90thresh="tn90thresh",tn95thresh="tn95thresh",
tx90thresh_15days="tx90thresh_15days",tn90thresh_15days="tn90thresh_15days",tavg90thresh_15days="tavg90thresh_15days",
tavg05thresh="tavg05thresh",tavg95thresh="tavg95thresh",
txraw="txraw",tnraw="tnraw",precraw="precraw",
r95thresh="r95thresh", r99thresh="r99thresh"))
`
======================= : END OF FILE "climpact2.ncdf.wrapper.r"

As you can see my thresholds file point on the good file, and my reference period is the same than the one specified in the "climpact2.ncdf.thresholds.wrapper.r" file : 1995-2005.

When I try to run this R script, I get an error. Here the error messages that I get :
"
Le chargement a nécessité le package : PCICt
Le chargement a nécessité le package : methods
Loading required package: lmomco
Loading required package: parallel
Loading required package: ggplot2

Package SPEI (1.7) loaded [try SPEINews()].

Error in check.quantile.validity(quantiles, present.var.list, days.in.base) :
Temperature in-base quantiles must contain 10th and 90th percentiles.
Calls: create.indices.from.files ... compute.climdex.indices -> -> check.quantile.validity
"

Please someone can help me?
Thanks a lot.

Hi There,

Everytime, I try to install the packages, it says like "not available for R version 3.3.2). As per the website, while checking the library, it gives "error in library, no such package found".

This is great package, I really look forward to using it. Hence I humbly request you for your valuable support. If possible, manual also would be great help.