Subsetting on DelayedArray objects with list elements fails about delayedarray

[hpages]

Good. I'm closing this. Feel free to re-open if you run into further problems when trying to subset a DelayedArray object of type "list".

[hpages]

The problem should be addressed in DelayedArray 0.7.29 (ecddbd7).

Don't forget to use drop=FALSE if you want the result of the above subsetting to be returned as a DelayedArray object (in which case subsetting is delayed).

Note that DelayedArray objects of type "list" have not received a lot of testing. Right now they work almost by chance. There are certainly things that won't work for these objects like trying to coerce them to HDF5Array:

library(HDF5Array)
m <- matrix(list(1:5, NULL, "a"), nrow=10, ncol=9)
M <- DelayedArray(m)
as(M, "HDF5Array")
# Error in h5createDataset(filepath, name, dim, storage.mode = type, size = size,  : 
#   datatype list not yet implemented. Try 'double', 'integer', or 'character'.

One thing to keep in mind is that even though an operation like log(M) seems to work:

M2 <- log(M)

it is actually invalid (like log(m) is). But because log(M) is delayed, the error is also delayed i.e. we only get it when realization is triggered (implicitly by show() or explicitly with something like as.matrix(M2)):

M2
# Error in .Primitive("log")(list()) : 
#   non-numeric argument to mathematical function

H.

[hpages]

I actually just changed this (70fd89a). I didn't realize until now this would be so easy to do (without impacting performance). H.

[Liubuntu]

Hi @hpages ,

Seems that the DelayedArray constructor over an array of type list could not work now...

 > aa <- array(list(), dim=10)
!> DelayedArray(aa)
 Error in type(x) :
   type() only supports array-like objects. See ?type in the DelayedArray
   package.
!> packageVersion("DelayedArray")
 [1] ‘0.7.30’

[hpages]

mmmh... I can't reproduce this. Could it be that you have some packages loaded that conflict with DelayedArray? Can you try this again in a fresh R session and show your sessionInto()? Here is what I get:

> library(DelayedArray)
> DelayedArray(array(list(), dim=10))
<10> DelayedArray object of type "list":
 [1]  [2]  [3]    .  [9] [10] 
NULL NULL NULL    . NULL NULL 
> sessionInfo()
R version 3.5.1 Patched (2018-08-01 r75051)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.5 LTS

Matrix products: default
BLAS: /home/hpages/R/R-3.5.r75051/lib/libRblas.so
LAPACK: /home/hpages/R/R-3.5.r75051/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] parallel  stats4    stats     graphics  grDevices utils     datasets 
[8] methods   base     

other attached packages:
[1] DelayedArray_0.7.30 BiocParallel_1.15.8 IRanges_2.15.16    
[4] S4Vectors_0.19.19   BiocGenerics_0.27.1 matrixStats_0.54.0 

loaded via a namespace (and not attached):
[1] compiler_3.5.1

Thanks!

[Liubuntu]

Yes, it works in a fresh new R session. I have a local repo for DelayedArray, where I have exported the DelayedUnaryOp and DelayedOp manually because of a bug in devtools (r-lib/devtools#1845), so that every time I use document() or load_all(), the class of DelayedMatrix will not lose any extended classes.

 > getClass("VCFMatrix")
 Class "VCFMatrix" [package "VCFArray"]

 Slots:

 Name:  seed
 Class:  ANY

 Extends:
 Class "DelayedMatrix", directly
 Class "VCFArray", directly
 Class "DelayedArray", by class "DelayedMatrix", distance 2
 Class "DelayedUnaryOp", by class "DelayedMatrix", distance 2
 Class "DelayedOp", by class "DelayedMatrix", distance 2
 Class "Array", by class "DelayedMatrix", distance 2
 Class "DataTable", by class "DelayedMatrix", distance 2
 Class "DataTable_OR_NULL", by class "DelayedMatrix", distance 3

Otherwise, with a second load_all() or document() it will look like this:

 > getClass("VCFMatrix")
 Class "VCFMatrix" [package "VCFArray"]

 Slots:

 Name:  seed
 Class:  ANY

 Extends:
 Class "DelayedMatrix", directly
 Class "VCFArray", directly
 Class "DelayedArray", by class "DelayedMatrix", distance 2
 Class "Array", by class "DelayedMatrix", distance 2
 Class "DataTable", by class "DelayedMatrix", distance 2
 Class "DataTable_OR_NULL", by class "DelayedMatrix", distance 3

My error from last comment might comes from here because I have only updated the DelayedArray 0.7.30 and didn't manually install the local changes...

Qian