bnprks / bpcells Goto Github PK

View Code? Open in Web Editor NEW

132.0 132.0 11.0 30.13 MB

Scaling Single Cell Analysis to Millions of Cells

Home Page: https://bnprks.github.io/BPCells

License: Other

R 10.38% C++ 37.52% CMake 0.15% C 51.77% Shell 0.13% Dockerfile 0.05% CSS 0.01%

bpcells's People

Contributors

Stargazers

Watchers

Forkers

tncowart brgew onurcanbektas hangzhang ycli1995 aaronwolf1995 theballerinathatcodes wjgreenleaf linuxpham yunuuuu topcrusader

bpcells's Issues

write_matrix_dir error

Hi,

I got the error when I apply the write_matrix_dir function. I have tried to re-install the BPCells, but still got this problem. Any suggestion? Thanks!

> write_matrix_dir(dat, "tmp")
Warning: Matrix compression performs poorly with non-integers.
• Consider calling convert_matrix_type if a compressed integer matrix is intended.
This message is displayed once every 8 hours.

 *** caught segfault ***
address 0x2b, cause 'memory not mapped'

Traceback:
 1: write_function(it, dir, buffer_size, overwrite, mat@transpose)
 2: write_matrix_dir(dat, "tmp")

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection: 3
Error in eval(formal.args[[as.character(substitute(arg))]], envir = sys.frame(sysP)) :
  Value of SET_STRING_ELT() must be a 'CHARSXP' not a 'builtin'

can BPCells work with dotplot? I have the following error:
dotplot<-DotPlot(object = obj, features = canonical_markers, group.by='seurat_clusters',cols = "Spectral") + theme(axis.text.x = element_text(angle = 45, size = 8), legend.text = element_text(size = 8), legend.title = element_text(size = 8))

Error in .rowNamesDF<-(x, value = value) :
duplicate 'row.names' are not allowed
In addition: Warning message:
non-unique values when setting 'row.names':

Bug when trying to index IterableMatrix

Hi @bnprks, thanks in advance for continuing to maintain such a useful package!

I've been running into an issue with a bit of an edge case recently when trying to index an IterableMatrix with a list of 0 features. I do this, for example, when trying to get the percentage of cells that express mitochondrial features. Doing this with a sparse matrix would return a matrix with no rows, and although the IterableMatrix output seems to suggest that is what is happening, doing some downstream functions on the object shows this is not the case.

For an example:
I create a test IterableMatrix w 20 rows and 20 cells

> test
20 x 20 IterableMatrix object with class MatrixSubset

Row names: MIR1302-2HG, FAM138A ... RNF223
Col names: cell1, cell2 ... cell20

Data type: float
Storage order: column major

Queued Operations:
1. Load compressed matrix from directory ~/test
2. Select rows: 1, 2 ... 37 and cols: 1, 2 ... 20

> as(test, "dgCMatrix")
20 x 20 sparse Matrix of class "dgCMatrix"
  [[ suppressing 20 column names 'cell1', 'cell2', 'cell3' ... ]]
                                                                                                                           
MIR1302-2HG .        .        . .        .        .        . .        . . . . . .        .         . . . .        .        
FAM138A     .        .        . .        .        .        . .        . . . . . .        .         . . . .        .        
OR4F5       .        .        . .        .        .        . .        . . . . . .        .         . . . .        .        
OR4F29      .        .        . .        .        .        . .        . . . . . .        .         . . . .        .        
OR4F16      .        .        . .        .        .        . .        . . . . . .        .         . . . .        .        
LINC01409   .        .        . .        .        .        . .        . . . . . .        .         . . . .        .        
FAM87B      .        .        . .        .        .        . .        . . . . . .        .         . . . .        .        
LINC01128   .        .        . .        .        .        . .        . . . . . 1.021765 .         . . . .        .        
LINC00115   .        .        . .        .        .        . .        . . . . . .        .         . . . .        .        
FAM41C      .        .        . .        .        0.784961 . .        . . . . . .        .         . . . .        .        
LINC02593   .        .        . .        .        .        . .        . . . . . .        .         . . . .        .        
SAMD11      .        .        . .        .        .        . .        . . . . . .        .         . . . .        .        
NOC2L       1.512336 .        . 1.792176 1.421819 .        . 1.216483 . . . . . .        0.9432759 . . . .        0.8852077
KLHL17      .        .        . .        .        .        . .        . . . . . .        .         . . . .        .        
PLEKHN1     .        .        . .        .        .        . .        . . . . . .        .         . . . .        .        
PERM1       .        .        . .        .        .        . .        . . . . . .        .         . . . .        .        
HES4        .        .        . .        .        .        . .        . . . . . .        .         . . . .        .        
ISG15       .        1.404183 . 2.398350 3.264126 .        . .        . . . . . 1.021765 0.9432759 . . . 2.555951 .        
AGRN        .        .        . .        1.421819 .        . .        . . . . . .        .         . . . .        .        
RNF223      .        .        . .        .        .        . .        . . . . . .        .         . . . .        .

Subsetting the matrix with 0 features seems to correctly give a 0 x 20 matrix. However, if I get the colSums of the matrix its greater than 0 and converting this subsetted matrix to a sparse matrix leads to a strange result.

> features <- grep("^MT-", rownames(test), value = T)
> features
character(0)
> test[features,]
0 x 20 IterableMatrix object with class MatrixSubset

Row names: unknown names
Col names: cell1, cell2 ... cell20

Data type: float
Storage order: column major

Queued Operations:
1. Load compressed matrix from directory ~/test
2. Select rows: all and cols: 1, 2 ... 20

> colSums(test[features,])
   cell1    cell2    cell3    cell4    cell5    cell6    cell7    cell8    cell9   cell10   cell11   cell12   cell13   cell14   cell15 
2203.956 2367.250 1867.033 1742.972 2418.026 2665.404 1226.463 1975.834 2367.414 2239.525 2029.724 1633.182 1250.033 2427.025 2412.731 
  cell16   cell17   cell18   cell19   cell20 
2366.497 2084.104 1745.754 1774.932 2716.428 

> as(test[features,], "dgCMatrix")
37412 x 20 sparse Matrix of class "dgCMatrix"
  [[ suppressing 20 column names 'cell1', 'cell2', 'cell3' ... ]]
  [[ suppressing 20 column names 'cell1', 'cell2', 'cell3' ... ]]
                                                          
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . 1.021765 . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . 0.784961 . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .
<NA> . . . . . .        . . . . . . . .        . . . . . .

 ..............................
 ........suppressing 37362 rows in show(); maybe adjust 'options(max.print= *, width = *)'
 ..............................
  [[ suppressing 20 column names 'cell1', 'cell2', 'cell3' ... ]]
                                                   
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . 1.021765 . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .
<NA> . . . . . . . . . . . . . .        . . . . . .

Indexing with 1 or greater features works fine, this just occurs with 0. Thanks for any advice or solutions you may have!

Best,
Gesi

Installation issue on Mac OS

Hi,
I got an error to install BPCells.
My environment is macOS 12.6, intel 64-bit, R 4.2.3 on RStudio-2023.03.0-386.

I installed in advance hdf5 with;
% conda install -c anaconda hdf5

Then restarted RStudio and;
>remotes::install_github("bnprks/BPCells")

.
.

installing source package ‘BPCells’ ...
** using staged installation
Testing hdf5 by compiling example program...

Retrying with pkg-config flags...
Error running 'pkg-config hdf5 --cflags --libs'

Searching for hdf5 in a conda env...
no conda environment found in '$CONDA_PREFIX'

Unable to locate libhdf5. Please install manually or edit compiler flags.
ERROR: configuration failed for package ‘BPCells’

removing ‘/Library/Frameworks/R.framework/Versions/4.2/Resources/library/BPCells’
Warning message:
In i.p(...) :
installation of package ‘/var/folders/63/ngc0xtk17p5cbbs6762b48d80000gn/T//RtmpAUI64J/filec47557cbe71/BPCells_0.1.0.tar.gz’ had non-zero exit status

The error messages seems saying R cannot find hdf5.
I wonder what exactly I should do to overcome this error.
Any help I will be deeply grateful.

load data

Hi,

Thank you for developing this tool.
I am trying to use open_matrix_anndata_hdf5 to load anndata from a paper. I got below error:

Error in eval(expr, envir, enclos): Error in opening AnnData matrix: "X" is a dataset rather than a group. This likely indicates a dense matrix which is not yet supported by BPCells.

Does this mean this anndata is not compatible with BPCells? Is there a way to solve it?
My second question is can BPCells load in MEX-format gene-barcode counts from 10x output?

Thank you!
Best,
Qian

write_matrix_dir crashes

Hi,

thanks for the lovely package. I would like to use it for quite a few cells. Everything works nicely until i reach about 5 Million cells. Above this threshold the function write_matrix_dir() and my entire R session cashes. I also do not get a warning or error.
I work in R/4.3.0 and use gcc/10.2.

Do you have an idea what the problem could be? Would increasing the buffer parameter in function could help to circumvent this?

Many thanks and kind regards

Installation issue

Hi there,

I am running into the following error during installation:

remotes::install_github("bnprks/BPCells", quiet = TRUE)

Warning message:
In i.p(...) :
  installation of package '/var/folders/04/jw8n0g8j29d6gdvcqwjjlp740000gq/T//Rtmpqppt53/fileff639b123f9/BPCells_0.1.0.tar.gz' had non-zero exit status

Any suggestions on how I can fix this?

Are there dependencies that I could be missing?

write_matrix_dir only writes one layer

Hello, thanks for making such a useful package!

I am having an issue with writing a merged v5 Seurat object to disk...

Using the following code:

write_matrix_dir(mat = seur1[["RNA"]]$counts, dir = countsdir)

gives me the error:

In LayerData.Assay5(object = x, layer = i) :
multiple layers are identified by countssampleID01 counts.sampleID02 counts.sampleID03 counts.sampleID04 counts.sampleID05 
only the first layer is used

And then only the first layer is written to disk, but I want all of the layers written... Do you know how I should do this?

Can't install BPCells

Hi, I'm trying to install BPCells but there are some errors like
"/usr/home/qgg/holi/anaconda3/envs/r4-base/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/home/qgg/holi/anaconda3/envs/r4-base/lib/libhdf5.so: undefined reference to `memcpy@GLIBC_2.14'"
and
"Unable to locate libhdf5. Please install manually or edit compiler flags".
However, I have already install the hdf5 by "conda install -c anaconda hdf5". Could you please help me to deal with it? Thank you.

Can't install BPCells on MacOS

The Error:

remotes::install_github("bnprks/BPCells")
Downloading GitHub repo bnprks/BPCells@HEAD
── R CMD build ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
✔  checking for file ‘/private/var/folders/vs/2n7hpqjd7s70rsg0ys8kzgc40000gn/T/RtmpXZPBBi/remotes4497101206fb/bnprks-BPCells-c647fb1/DESCRIPTION’ ...
─  preparing ‘BPCells’:
✔  checking DESCRIPTION meta-information ...
─  cleaning src
─  running ‘cleanup’
─  checking for LF line-endings in source and make files and shell scripts (423ms)
─  checking for empty or unneeded directories
─  building ‘BPCells_0.1.0.tar.gz’
   
* installing *source* package ‘BPCells’ ...
** using staged installation
Testing hdf5 by compiling example program...


Retrying with pkg-config flags...
Error running 'pkg-config hdf5 --cflags --libs'


Searching for hdf5 in a conda env...
no conda environment found in '$CONDA_PREFIX'


Unable to locate libhdf5. Please install manually or edit compiler flags.
ERROR: configuration failed for package ‘BPCells’
* removing ‘/Library/Frameworks/R.framework/Versions/4.2/Resources/library/BPCells’
Warning message:
In i.p(...) :
  installation of package ‘/var/folders/vs/2n7hpqjd7s70rsg0ys8kzgc40000gn/T//RtmpXZPBBi/file449714bda8bf/BPCells_0.1.0.tar.gz’ had non-zero exit status

First I installed Macports and HDF5 with
sudo port install hdf5
having the base MacOS environment activated (as default in the terminal)
The error persisted

Then I installed HDF5 using homebrew having the base MacOS environment activated (as default in the terminal) following this post. The error persisted.

Then I installed HDF5 with conda having the base env activated
% conda install -c anaconda hdf5
But the error persisted

I've tried, in R

Sys.setenv(HDF5_DIR = "/opt/homebrew/Cellar/hdf5/1.14.0")
Sys.setenv(HDF5_LIBRARIES = "/opt/homebrew/Cellar/hdf5/1.14.0/lib")
remotes::install_github("bnprks/BPCells")

To no avail

What else could I try?
Thank you in advance for your time

Data load error

I just starting using BPCells and was following along the tutorial. But in the RNA matrix conversion section, the command with open_matrix_10x_hdf5 gets stuck (doesn't finish for hours):
mat_raw <- open_matrix_10x_hdf5("pbmc_3k_10x.h5", feature_type="Gene Expression") %>% write_matrix_dir("pbmc_3k_rna_raw")

It shows no error, progress, nor it stops running. Is there a bug in open_matrix_10x_hdf5, or I'm doing something wrong?

Compilation issues on macOS

BPCells fails to compile on macOS; the error happens in multiple C++ files at

class _LIBCPP_TYPE_VIS path

error: 'path' is unavailable: introduced in macOS 10.15
    std::filesystem::path dir

anndata compression in benchmark

Hey all, just wanted to raise a quick issue about the benchmarks:

BPCells uses bitpacking compression to provide 4-6x smaller storage for counts matrices compared to an uncompressed binary format like AnnData.

AnnData isn't inherently uncompressed. You can specify the kind of compression used in the hdf5 format with the compression key word argument (docs). You could use more modern (and much faster) compressors like lzf and zstd as well, but there may be compatibility with different hdf5 libraries.

With the zarr format, you have access to a much wider range of compressors, with blosc used by default.

RSpectra::eigs()/eigs_sym() compatibility with BPCells

Hello,

I am trying to use BPCells to work with a large connectivity matrix, and ideally I would need to apply the eigs_sym() function from RSpectra to extract the top k eigenvalues/vectors from that matrix. Is there a way to make the BPCells matrix compatible with this additional RSpectra function?

Thank you for your help :)

Pietro

"data layers are not joined. Please run JoinLayers"

I took a seq dataset into Seurat to find markers.
I run this code:FindMarkers(pbmc, ident.1 = 2, min.pct = 0.25)
Result in:Error in FindMarkers.StdAssay(object = data.use, slot = slot, cells.1 = cells$cells.1, :
data layers are not joined. Please run JoinLayers

How to solve this problem?

Installation under CentOS Linux for R 4.2.2 results in undefined symbol error: H5P_CLS_GROUP_ACCESS_ID_g

Hi, I am trying to install BPCells into R 4.2.2 in an HPC environment, and my attempt ends with an undefined symbol error:

Downloading GitHub repo bnprks/BPCells@HEAD
Skipping 1 packages ahead of CRAN: colorspace
── R CMD build ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────
✔  checking for file ‘/tmp/RtmpRDAmB1/remotesa77663e4571/bnprks-BPCells-ff447ae/DESCRIPTION’ (677ms)
─  preparing ‘BPCells’: (339ms)
✔  checking DESCRIPTION meta-information ...
─  cleaning src
─  running ‘cleanup’
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘BPCells_0.1.0.tar.gz’
   
Installing package into ‘/scg/apps/software/r/4.2.2/lib’
(as ‘lib’ is unspecified)
* installing *source* package ‘BPCells’ ...
** using staged installation
Testing hdf5 by compiling example program...
Found working hdf5
HDF5_CFLAGS=''
HDF5_LIBS='-lhdf5'

Testing architecture flag support support...'-march=native' succeeded
ARCH_FLAG='-march=native'
** libs
g++ -std=gnu++14 -I"/scg/apps/software/r/4.2.2/lib64/R/include" -DNDEBUG  -I'/scg/apps/software/r/4.2.2/lib/Rcpp/include' -I'/scg/apps/software/r/4.2.2/lib/RcppEigen/include' -I/usr/local/include  -march=native -std=c++17 -DRCPP_EIGEN -DEIGEN_PERMANENTLY_DISABLE_STUPID_WARNINGS -Wno-ignored-attributes -Wno-unknown-pragmas  -fpic  -mtune=generic -march=x86-64  -c bitpacking_io.cpp -o bitpacking_io.o

<COMPILER LINES REMOVED HERE>

*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for 'BPCells' in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/scg/apps/software/r/4.2.2/lib/00LOCK-BPCells/00new/BPCells/libs/BPCells.so':
  /scg/apps/software/r/4.2.2/lib/00LOCK-BPCells/00new/BPCells/libs/BPCells.so: undefined symbol: H5P_CLS_GROUP_ACCESS_ID_g
Error: loading failed
Execution halted
ERROR: loading failed
* removing '/scg/apps/software/r/4.2.2/lib/BPCells'
* restoring previous '/scg/apps/software/r/4.2.2/lib/BPCells'```

R seems to find HDF5 alright and it's a recent version: 1.14.0.  I'm assuming this isn't an HDF5 version problem, because that symbol has been around a long time and the compilation would likely have missed other symbols, too.  I'm also assuming that it isn't a gcc version problem (9.3.0), because it would probably have more problems with more symbols than just one.

Do you have any ideas here?  Happy to give you more information on my environment as you need it.

Thank you for your help.
Keith

Issue with configure script

Originally posted by @brgew in #21 (comment)

Hi Ben,

Uh-oh. Installing BPCells using devtools::load_all() in R 4.2.0 on my
docker machine running debian 'bookworm/sid' fails in the 'configure'
script with the message

Re-compiling BPCells
─ installing source package ‘BPCells’ ...
** using staged installation
Testing hdf5 by compiling example program...

Retrying with pkg-config flags...
Found working hdf5
HDF5_CFLAGS='-I/usr/include/hdf5/serial'
HDF5_LIBS='-L/usr/lib/x86_64-linux-gnu/hdf5/serial -lhdf5'

Testing architecture flag support support...'-march=native' succeeded
ARCH_FLAG='-march=native'
Warning in system(cmd) : error in running command
ERROR: configuration failed for package ‘BPCells’
─ removing ‘/tmp/RtmpvoyPv0/devtools_install_21cf1bd3efc8/BPCells’
Error in (function (command = NULL, args = character(), error_on_status = TRUE, …:
! System command 'R' failed

(I am using the docker image with Github actions so I am trying to
understand why my regression tests fail now.)

I don't understand what's happening -- perhaps you have some insight.

The problem goes away when I either add a printf statement at the end of
the script or remove the recently added lines

if [ -n $ENABLE_INSTALL_COUNTING ]; then
curl --silent "
https://plausible.benparks.net/flask-plausible/bpcells-configure-success" >
/dev/null 2> /dev/null
fi

or run ./configure on the command line.

Oh, I added the double quotes around the URL, which did not help.

The configure script runs using the bash version

Bash versionGNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu)

in both the devtools::install_github() case and the manual ./configure case.

I appreciate your consideration and assistance.

Thank you.

Ever grateful,
Brent

Issue installing BPCell on Windows RStudio

When I am trying to install BPCell, It throws following error

_-- R CMD build --------------------------------------------------------------------------------------------------------------------------------
v checking for file 'C:\Users\sindiris\Documents\R\R-4.1.3\library\BPCells/DESCRIPTION' (359ms)

preparing 'BPCells': (669ms)
v checking DESCRIPTION meta-information ...
cleaning src
checking for LF line-endings in source and make files and shell scripts (369ms)
checking for empty or unneeded directories
building 'BPCells_0.1.0.tar.gz'
Warning: file 'BPCells/cleanup' did not have execute permissions: corrected
Warning: file 'BPCells/configure' did not have execute permissions: corrected

Running "C:/Users/sindiris/Documents/R/R-4.1.3/bin/x64/Rcmd.exe" INSTALL
"C:\Users\sindiris\AppData\Local\Temp\1\Rtmp61cBHj/BPCells_0.1.0.tar.gz" --install-tests

installing to library 'C:/Users/sindiris/Documents/R/R-4.1.3/library'
installing source package 'BPCells' ...
** using staged installation
Testing hdf5 by compiling example program...
tools/h5write.c:19:10: fatal error: hdf5.h: No such file or directory
#include "hdf5.h"
^~~~~~~~
compilation terminated.

Retrying without -lsz flag...
tools/h5write.c:19:10: fatal error: hdf5.h: No such file or directory
#include "hdf5.h"
^~~~~~~~
compilation terminated.

Unable to locate libhdf5. Please install manually or edit compiler flags.
ERROR: configuration failed for package 'BPCells'

removing 'C:/Users/sindiris/Documents/R/R-4.1.3/library/BPCells'
restoring previous 'C:/Users/sindiris/Documents/R/R-4.1.3/library/BPCells'
Error in (function (command = NULL, args = character(), error_on_status = TRUE, ...:
! System command 'Rcmd.exe' failed

Exit status: 1
stdout & stderr:

Type .Last.error to see the more details._

Then I changed hfwrite.c file by including the complete path of the HDF5 library like below
#include "C:/Program Files/HDF_Group/HDF5/1.14.1/include/hdf5.h"
Then I am getting the following issue

-- R CMD build --------------------------------------------------------------------------------------------------------------------------------
v checking for file 'C:\Users\sindiris\Documents\R\R-4.1.3\library\BPCells/DESCRIPTION' (345ms)

preparing 'BPCells': (658ms)
v checking DESCRIPTION meta-information ...
cleaning src
checking for LF line-endings in source and make files and shell scripts (380ms)
checking for empty or unneeded directories
building 'BPCells_0.1.0.tar.gz'
Warning: file 'BPCells/cleanup' did not have execute permissions: corrected
Warning: file 'BPCells/configure' did not have execute permissions: corrected

Running "C:/Users/sindiris/Documents/R/R-4.1.3/bin/x64/Rcmd.exe" INSTALL
"C:\Users\sindiris\AppData\Local\Temp\1\Rtmp61cBHj/BPCells_0.1.0.tar.gz" --install-tests

installing to library 'C:/Users/sindiris/Documents/R/R-4.1.3/library'
installing source package 'BPCells' ...
** using staged installation
Testing hdf5 by compiling example program...
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lhdf5
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lsz
collect2.exe: error: ld returned 1 exit status

Retrying without -lsz flag...
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lhdf5
collect2.exe: error: ld returned 1 exit status

Unable to locate libhdf5. Please install manually or edit compiler flags.
ERROR: configuration failed for package 'BPCells'

removing 'C:/Users/sindiris/Documents/R/R-4.1.3/library/BPCells'
restoring previous 'C:/Users/sindiris/Documents/R/R-4.1.3/library/BPCells'
Error in (function (command = NULL, args = character(), error_on_status = TRUE, ...:
! System command 'Rcmd.exe' failed

Exit status: 1
stdout & stderr:

It looks like BPCell is not able to locate "C:\Program Files\HDF_Group\HDF5\1.14.1\lib" path . Can you please help me resolve this ? Where can I declare (by using a global variable or sthng) the above path so that its visible to BPCell. ?

Following is my session info

sessionInfo() R version 4.1.3 (2022-03-10) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 19044) Matrix products: default locale: [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C LC_TIME=English_United States.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] Rcpp_1.0.11 pillar_1.9.0 compiler_4.1.3 later_1.3.1 urlchecker_1.0.1 prettyunits_1.1.1 profvis_0.3.7 [8] remotes_2.4.2.1 tools_4.1.3 digest_0.6.33 pkgbuild_1.4.2 pkgload_1.3.2.1 memoise_2.0.1 lifecycle_1.0.3 [15] tibble_3.2.1 pkgconfig_2.0.3 rlang_1.1.1 shiny_1.7.4.1 cli_3.6.1 rstudioapi_0.15.0 fastmap_1.1.1 [22] stringr_1.5.0 desc_1.4.2 fs_1.6.3 htmlwidgets_1.6.2 vctrs_0.6.3 devtools_2.4.5 rprojroot_2.0.3 [29] glue_1.6.2 R6_2.5.1 processx_3.8.1 fansi_1.0.4 sessioninfo_1.2.2 callr_3.7.3 purrr_1.0.2 [36] magrittr_2.0.3 ps_1.7.5 promises_1.2.1 ellipsis_0.3.2 htmltools_0.5.5 usethis_2.2.2 mime_0.12 [43] xtable_1.8-4 httpuv_1.6.11 utf8_1.2.3 stringi_1.7.12 miniUI_0.1.1.1 cachem_1.0.8 crayon_1.5.2
--

|

Reference mapping

I have 10x multimome and 10xsciATACseq.
I would like to reference map using Seurat V5, is there a vignette to use bpcells instead of Signac to do so?

Installation issue (Amazon Linux 2, hdf5 from source)

[Issue split out from a comment on another issue]

Same here, i'm running into the same issue as @sanderslhc I installed hdf5 from source and can successfully run h5cc -showconfig and it's in path! During the install process it runs pkg-config hdf5 --cflags --libs to validate that hdf5 exists and it's not finding it, so it's suggesting we find a non-existing file: hdf5.pc and add it to the PKG_CONFIG_PATH. @bnprks isn't there a better way to validate hdf5 exists like using h5cc?
Also i'm using Amazon Linux 2 on an EC2 it's closer to centos and rhel than to ubuntu.

Originally posted by @steklemichael in #15 (comment)

Need some help to extract data from a shared object.

I am a beginner to use BPCells. Thank you very much for developing and maintaining this awesome package! My collaborator shared a large collection of multiome data as a Seurat object (Assay5), and I would like to extract the RNA count matrix from the object. The count matrix obj@assays$RNA@layers$counts comes with the following description.

36588 x 10 IterableMatrix object with class RenameDims

Row names: A1BG, A1BG-AS1 ... ZZEF1
Col names: A_AAACAGCCAAACCTAT-1, A_AAACGTACACGAACAG-1 ... A_AACTCACAGGATTGAG-1

Data type: uint32_t
Storage order: column major

Queued Operations:
1. Load compressed matrix from directory "/the path that hosts the data"
2. Select rows: 1, 2 ... 36588 and cols: 826155, 826180 ... 826291
3. Reset dimnames

I tried two commands temp %>% write_matrix_memory(compress = F) and as.matrix(obj@assays$RNA@layers$counts) that lead to the same error Missing directory: /the path that hosts the data.

I understand the on-disk feature of BPCells and this error is clearly related to the first queued operation. But obj@assays$RNA@layers$counts is about 153MB in memory which makes me think the matrix is already available in the memory. If so, is there a way to modify the embedded operations and extract the count matrix?
A follow up question: Is it possible to share a BPCells object with others without sharing the original on-disk files?

BPCells operations fail

There is a bug in BPCells after several operations in the IterableMatrix matrix.
After row subset, row centering, row standardization, the matrix cannot be converted sparse matrix (crash R session) but matrix multiplication still can work. With additional min_scalar step, the matrix multiplication doesn't work.
Here is an reproducible example:

library(Seurat)
library(BPCells)
library(dplyr)
library(SeuratData)
data('pbmc3k')
pbmc3k <- UpdateSeuratObject(pbmc3k)
pbmc3k <- FindVariableFeatures(pbmc3k)%>%ScaleData()%>%RunPCA()
mat <- write_matrix_memory(pbmc3k[['RNA']]@data) 
mat <- mat[VariableFeatures(pbmc3k),]


feature_stat <- matrix_stats(matrix = mat, row_stats = c('variance'))
feature.mean <- feature_stat$row_stats['mean',]
feature.sd <- sqrt(feature_stat$row_stats['variance',])
mat.scale <- (mat - feature.mean)/feature.sd

# work
proj_1 <- t(mat.scale) %*% pbmc3k[['pca']]@feature.loadings[VariableFeatures(pbmc3k),]

# not work
mat.sparse_1 <- as(object = mat.scale, Class = 'dgCMatrix')


mat.scale <- min_scalar(mat = mat.scale, val = 10)

# not work
mat.sparse_2 <- as(object = mat.scale, Class = 'dgCMatrix')

# not work
proj_2 <-t(mat.scale) %*% pbmc3k[['pca']]@feature.loadings[VariableFeatures(pbmc3k),]

Markers different in BPCells than in Seurat

@bnprks Thank you so much for this awesome package. Using BPCells is worlds faster and worlds more memory efficient, which allows analyzing huge datasets. The integration by @Gesmira and others into Seurat is also great.

I struggle with finding the top markers. Unfortunately, it's not possible until now to use the BPCells Matrix in Seurat satijalab/seurat#7516.

However, I saw that you also offered the marker_features function which works directly on the BPCells Matrix and can be easily integrated with the Seurat workflow. It's worlds faster, however the results differ.
P values maybe different because of Bonferroni, but log2FC should be the same (or is there a mistake in my code?) and the order of the genes should be the same

I created a reproducible example with parts of my own published dataset.

https://osmzhlab.uni-muenster.de/nfs/bpcells/bpcells_debug.html

Maybe @Gesmira can also help? I think this would be a very easy way for Seurat to integrate a very fast FindMarkers alternative that works on BPCell Matrix.

Thanks!

Installation error

I tried to install BPCells on a different machine and it failed this time with following error message

factorization/GLMFactorization.h:153:74:   required from here
/data/home/wm8285/R/x86_64-pc-linux-gnu-library/4.2/RcppEigen/include/Eigen/src/Core/CoreEvaluators.h:960:8: warning: ignoring attributes on template argument ‘Eigen::internal::packet_traits<double>::type’ {aka ‘__vector(2) double’} [-Wignored-attributes]
make: *** [factorization/GLMFactorization.o] Error 1
ERROR: compilation failed for package ‘BPCells’
* removing ‘/data/home/wm8285/R/x86_64-pc-linux-gnu-library/4.2/BPCells’
Warning message:
In i.p(...) :
  installation of package ‘/tmp/RtmplGK56U/file4947546c0202/BPCells_0.1.0.tar.gz’ had non-zero exit status

Could you help resolve this error? Thanks!

Unable to open file

Hi Ben,

Thank you for making such an amazing tool accessible to the community.

I'm using BPCells open_matrix_10x_hdf5() function to read an .h5 file as part of the Seurat v5 workflow, and having an error (please see below).

The .h5 file is located in someone else's directory (/outs/filtered_feature_bc_matrix.h5) and I'm getting access as part of a group. Here are the settings for the directory and file itself: drwxr-s---, -rw-r-----.

Interestingly, I can read the same .h5 file from the same directory with Read10x_h5() function in Seurat with no issues.

Also, when I copy the .h5 into my own directory, then it works without any errors with the open_matrix_10x_hdf5() function.

Seems like there's a certain permission setting for the directory and/or file for BPCells to process it, and somehow that is not an issue for the Seurat function.

Do you have any suggestions to fix this problem without having to copy all the data into my own directory (it's a pretty large dataset)?

Thank you!
Tayfun

Error in dims_matrix_10x_hdf5_cpp(path, buffer_size) : HDF5-API Errors:
    error #000: H5F.c in H5Fopen(): line 602: unable to open file
        class: HDF5
        major: File accessibilty
        minor: Unable to open file

    error #001: H5Fint.c in H5F_open(): line 990: unable to open file: time = Fri May  5 14:17:36 2023
, name = '/outs/filtered_feature_        class: HDF5
        major: File accessibilty
        minor: Unable to open file

    error #002: H5FD.c in H5FD_open(): line 991: open failed
        class: HDF5
        major: Virtual File Layer
        minor: Unable to initialize object

    error #003: H5FDsec2.c in H5FD_sec2_open(): line 337: unable to open file: name = '/outs/filtered_feature_bc_matrix.h5', errno = 13,        class: HDF5
        major: File accessibilty

cannot install BPcells on ARM mac- cannot find hdf5

Hi
I am having trouble installing BPcells on an ARM mac
remotes::install_github("bnprks/BPCells", quiet = F)
Downloading GitHub repo bnprks/BPCells@HEAD
Running R CMD build...

checking for file ‘/private/var/folders/1d/nnj_zbrn2_l3zl_yqqhs67t00000gn/T/RtmppNzrDM/remotes4ed525d57d8d/bnprks-BPCells-71d2de7/DESCRIPTION’ ... OK
preparing ‘BPCells’:
checking DESCRIPTION meta-information ... OK
cleaning src
running ‘cleanup’
checking for LF line-endings in source and make files and shell scripts
checking for empty or unneeded directories
building ‘BPCells_0.1.0.tar.gz’
Installing package into ‘/Users/cparekh/seurat_beta/renv/library/R-4.2/aarch64-apple-darwin20’
(as ‘lib’ is unspecified)
installing source package ‘BPCells’ ...
** using staged installation
Testing hdf5 by compiling example program...

Retrying with pkg-config flags...
Error running 'pkg-config hdf5 --cflags --libs'

Searching for hdf5 in a conda env...
no conda environment found in '$CONDA_PREFIX'

Unable to locate libhdf5. Please install manually or edit compiler flags.
ERROR: configuration failed for package ‘BPCells’

removing ‘/Users/cparekh/seurat_beta/renv/library/R-4.2/aarch64-apple-darwin20/BPCells’
Warning message:
In i.p(...) :
installation of package ‘/var/folders/1d/nnj_zbrn2_l3zl_yqqhs67t00000gn/T//RtmppNzrDM/file4ed57ef47581/BPCells_0.1.0.tar.gz’ had non-zero exit status

I installed hdf5 via homebrew but I still get the same error during BPcells installation. hdf5 is installed in "/opt/homebrew/Cellar" since that is the default package installation location for homebrew. I added /opt/homebrew/Cellar to the path in R and it shows up in the path:

Sys.getenv("PATH")
[1] "/usr/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin:/Users/cparekh/Applications/quarto/bin:/Library/TeX/texbin:/usr/texbin:/Applications/RStudio.app/Contents/Resources/app/quarto/bin:/Applications/RStudio.app/Contents/Resources/app/bin/postback:/opt/homebrew/bin:/opt/homebrew/sbin:/opt/homebrew/Cellar:/opt/homebrew/lib"

However, BPcells still cannot find hdf5 during installation and then fails to install.

Thank you very much

BPCells matrix to sparse matrix

I've been using this package for some weeks now and it's amazing how it allows to process big datasets in R. After doing some work, I need to export a trimmed verison of the dataset in the usual sparse matrix format, for compatibility reasons, but I am failing to find a way to do so, would you be so kind to point me into the right direction?

Support on using in Windows?

Seurat V5 is coming, but my installation of BPCells in Windows never suceeded. Do you have any plan for support using BPCells in Windows?

Converting ArchR project to BPCells

Hi @bnprks,

I am excited to try BPCells on my current ATACseq project but I am having issues migrating my data from ArchR.

My current project contains 500.000 cells across 100 samples, and while extracting the fragments of a single sample and converting them to BPCells fragments works as expected, I am running into trouble when I try to scale it to the full project exceeding the R vector limitations combining the samples.

I can see from the Seurat v5 tutorial on BPCells that they are creating individual matrices and merging them when creating the Seurat object, but i am uncertain if this would work with ATACseq.

What would be the best approach to converting my ArchR project to BPCells?

Best regards
Jonas

export rbind2 and cbind2 methods?

Hi Ben,

I hope that you haven't answered this question already.

I am testing some matrix combining code and I see errors telling me that BPCells::rbind2 and BPCells::cbind2 are not exported. I modified the BPCells/R/matrix.R code to export the methods and our code works now. Am I messing up yet again? (I hope not.)

If it seems reasonable to add these exports, I can create a pull request with the modifications.

I appreciate your patience with me.

Ever grateful,
Brent

Peak matrix creation with missing chromosomes

Creating a peak matrix where the peak set includes chromosomes not present in the fragments currently yields an unintended error. This should probably just be a warning but otherwise continue.

Unsuccessful installation on Windows

I'm trying to install BPCells to follow this Seurat tutorial(https://satijalab.org/seurat/articles/seurat5_bpcells_interaction_vignette.html), and I'm receiving this console readout:

Installing package into ‘C:/Users/BYKOVAM/AppData/Local/R/win-library/4.2’
(as ‘lib’ is unspecified)

installing source package 'BPCells' ...
** using staged installation
Testing hdf5 by compiling example program...
C:\rtools42/x86_64-w64-mingw32.static.posix/bin/ld.exe: cannot find -lsz
collect2.exe: error: ld returned 1 exit status

Unable to locate libhdf5. Please install manually or edit compiler flags.
ERROR: configuration failed for package 'BPCells'

removing 'C:/Users/BYKOVAM/AppData/Local/R/win-library/4.2/BPCells'
Warning message:
In i.p(...) :
installation of package ‘C:/Users/BYKOVAM/AppData/Local/Temp/RtmpERPVuP/file6d45421204a8e/BPCells_0.1.0.tar.gz’ had non-zero exit status

I'm installing this in Rstudio version 2022.07.2 Build 576 on a Windows 10 laptop

Cannot install BPCells on Mac

Hi @bnprks,
Thank you for developing BPCells.

I still cannot install BPCells on Mac, even following your suggestions at #3 (comment).

I have installed clang, hdf5, using Conda. I also generated an R Makevars file as below:

However, when installing BPCells, I encounter the following issue:

I am using macOS 12.4 (M1 chip). R is 4.2.2.

Do you have any idea what is going on here? Many thanks in advance.

Submit to CRAN or CONDA

Hi,

Was just curious if there was a plan to upload to CRAN with the release of Seuratv5? I know these are independent packages, but since Seurat v5 depends on BPCells, would be nice to install via CRAN or CONDA.

Thanks 🙂

write_matrix_dir does not write rownames and colnames of a matrix transformed from dgCMatrix

g = readRDS("raw.RDS") # previous Assay3 in dgCMatrix
g[["RNA"]] = as(g[["RNA"]], Class = "Assay5")
g[["RNA"]]$counts = as(g[["RNA"]]$counts, "IterableMatrix", strict = F)
write_matrix_dir(mat = g[["RNA"]]$counts, dir = "BPCell/counts", overwrite = T)
g = CreateSeuratObject(open_matrix_dir("BPCell/counts"))
g[["RNA"]]

the output of g[["RNA"]] is a matrix with rownames of Feature1 Feature2 Feature3 ... and colnames of Cells_1 Cell_2 ...

BPCells to h5AD

Hi,

Thanks for creating this great tool! I'd like to use data I currently have stored in a BPCells matrix with a library only found in Python and that takes in h5ad Anndata files - does BPCells have functionality to write matrices to h5ad?

Thanks!

`write_matrix_hdf5` crashes when wrapped in other function

Hi, @bnprks ,

When I'm trying to wrap write_matrix_hdf5() into my package, it sometimes crashes the R session.

I design a function writeH5BPCE() to write a IterableMatrix, along with the metadata or something else related to the project, into an HDF5 file. Everything seems fine when I try only single omics.

library(BPCells.Experiment)

## Load the example RNA + ATAC dataset in BPCells.Experiment
scme <- load_example_scme()
scme

Default experiment:  RNA 
reducedDimNames(0):
colPairNames(0):
A SingleCellMultiExperiment object of 2 listed
 experiments with user-defined names and respective classes.
 Containing an ExperimentList class object of length 2:
 [1] RNA: BPCExperiment with 217 rows and 90 columns
 [2] ATAC: ChromBPCExperiment with 1308 rows and 90 columns
Functionality:
 experiments() - obtain the ExperimentList instance
 colData() - the primary/phenotype DataFrame
 sampleMap() - the sample coordination DataFrame
 `$`, `[`, `[[` - extract colData columns, subset, or experiment
 *Format() - convert into a long or wide DataFrame
 assays() - convert ExperimentList to a SimpleList of matrices
 exportClass() - save data to flat files

Write the RNA experiment into H5, and it worked. In this scope, the function to write a 10xMatrixH5 into an H5 group mainly just wraps write_matrix_hdf5() with some verbose messages.

bpce <- writeH5BPCE(scme[[1]], "bpce.h5", overwrite = TRUE)

Writing assay 'counts'
Writing a IterableMatrix:
 Class: 10xMatrixH5
 File: /develop/ycli1995/BPCells.testwrapper/bpce.h5
 Group: /assays/counts
Writing rowRanges
Writing rowData
Writing colData
Writing metadata
Writing altExps
Writing reducedDims
Writing colPairs
Writing rowPairs
Add H5 attribute indicating S4 class: BPCExperiment
Updating altExps
Updating assays
Reading assay 'counts'

Similarly, writing the ATAC experiment into H5 also worked.

cbpce <- writeH5BPCE(scme[[2]], "cbpce.h5", overwrite = TRUE)
cbpce
Writing assay 'counts'
Writing a IterableMatrix:
 Class: RenameDims
 File: /develop/ycli1995/BPCells.testwrapper/cbpce.h5
 Group: /assays/counts
Writing rowRanges
Writing rowData
Writing colData
Writing metadata
Writing altExps
Writing reducedDims
Writing colPairs
Writing rowPairs
Writing fragments
Writing a IterableFragments:
 Class: CellSelectName
 File: /develop/ycli1995/BPCells.testwrapper/cbpce.h5
 Group: /fragments
Writing geneAnnot
Writing seqinfo
Add H5 attribute indicating S4 class: ChromBPCExperiment
Updating altExps
Updating assays
Reading assay 'counts'
Updating fragments
Reading fragments

However, for the multi-omics data it failed.

scme2 <- writeH5SCME(scme, "scme.h5", overwrite = TRUE)

# Writing experiment: RNA
Writing assay 'counts'
Writing a IterableMatrix:
 Class: 10xMatrixH5
 File: /develop/ycli1995/BPCells.testwrapper/scme.h5
 Group: /experiments/RNA/assays/counts

Basically, writeH5SCME() just extracts the RNA and ATAC experiments, and calls writeH5BPCE() as above in a for loop. It should have worked. However, writeH5SCME() stucked while writing the 10xMatrixH5 into a H5 group, according to the progress messages.
No output followed anymore, neither error nor warning. The R session just somehow crashes, and ctrl+c cannot interupt the program. In the meantime, if I check htop, the R session was just hanging out there and doing nothing.

If I manually extract those two experiments on the console, and call writeH5BPCE() in a for loop, exactly like what writeH5SCME() does, it worked without any interuption!

file <- "scme2.h5"
name <- "/"
gzip_level <- 0L
verbose <- TRUE
object <- h5Prep(scme)
exp.names <- names(x = object$experiments)
for (i in exp.names) {
  if (verbose) {
    message("# Writing experiment: ", i)
  }
  writeH5BPCE(
    object = object$experiments[[i]],
    file = file,
    name = file.path(name, "experiments", i),
    gzip_level = gzip_level,
    verbose = verbose
  )
  BPCells.Experiment:::.set_h5attr_bpce(
    object = exp,
    file = file,
    name = file.path(name, "experiments", i),
    verbose = verbose
  )
  if (verbose) {
    message("Done\n")
  }
}

# Writing experiment: RNA
Writing assay 'counts'
Writing a IterableMatrix:
 Class: 10xMatrixH5
 File: /develop/ycli1995/BPCells.testwrapper/scme2.h5
 Group: /experiments/RNA/assays/counts
Writing rowRanges
Writing rowData
Writing colData
Writing metadata
Writing altExps
Writing reducedDims
Writing colPairs
Writing rowPairs
Add H5 attribute indicating S4 class: BPCExperiment
Updating altExps
Updating assays
Reading assay 'counts'
Done

# Writing experiment: ATAC
Writing assay 'counts'
Writing a IterableMatrix:
 Class: RenameDims
 File: /develop/ycli1995/BPCells.testwrapper/scme2.h5
 Group: /experiments/ATAC/assays/counts
Writing rowRanges
Writing rowData
Writing colData
Writing metadata
Writing altExps
Writing reducedDims
Writing colPairs
Writing rowPairs
Writing fragments
Writing a IterableFragments:
 Class: CellSelectName
 File: /develop/ycli1995/BPCells.testwrapper/scme2.h5
 Group: /experiments/ATAC/fragments
Writing geneAnnot
Writing seqinfo
Add H5 attribute indicating S4 class: ChromBPCExperiment
Updating altExps
Updating assays
Reading assay 'counts'
Updating fragments
Reading fragments
Done

Considering all of above, I suspect there must be something wrong when I call write_matrix_hdf5() in the function calling stack. However, I cannot find any clue why the writing didn't work inside writeH5BPCE(), but worked outside it. This has been trapping me for a few days.
Is there any advice from you? I push a repository in case that you may need to reproduce the results: https://github.com/ycli1995/BPCells.testwrapper

open_matrix function for .mtx files with associated cell and gene info

Hi,
The open_matrix_10x_hdf5 and open_matrix_anndata_hdf5 functions are really handy for dealing with large datasets. Do you have or expect to make similar functionality for reading in files that are in the format of matrix.mtx, featurs.csv, and barcodes.csv? Or have tips for doing so? For example, the data format of the 1 million cell Parse dataset. Thanks!

Rename row names of 10x HDF5 feature matrix cannot be stored on disk

I altered the row names of a 10x HDF5 feature matrix and used write_matrix_dir to save as BPCells directory. However, the newly loaded matrix's row names remain unchanged.
Example:

> brain.data <- open_matrix_10x_hdf5(path = "/brahms/haoy/seurat5/object/mouse_brain_1M/1M_neurons_filtered_gene_bc_matrices_h5.h5")
> head(rownames(brain.data))
[1] "ENSMUSG00000051951" "ENSMUSG00000089699" "ENSMUSG00000102343" "ENSMUSG00000025900" "ENSMUSG00000109048" "ENSMUSG00000025902"
> rownames(brain.data) <- paste0('a', 1:nrow(brain.data))
> head(rownames(brain.data))
[1] "a1" "a2" "a3" "a4" "a5" "a6"
> write_matrix_dir(mat = brain.data, dir = "~/test/bp_mat_rename", overwrite = T)
> mat2 <- open_matrix_dir("~/test/bp_mat_rename")
> head(rownames(mat2))
[1] "ENSMUSG00000051951" "ENSMUSG00000089699" "ENSMUSG00000102343" "ENSMUSG00000025900" "ENSMUSG00000109048" "ENSMUSG00000025902"

BPcells compilation

Hi there,
I am trying to install BPCells on a high performance compute cluster
R version 4.2.2 (2022-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.6 LTS

I am running into the error:
Unable to compile program with C++17 std::filesystem. Please install a newer compiler version and set CC and CXX in ~/.R/Makevars

I have tried setting my Makevars to
CC = /usr/bin/gcc CXX = /usr/bin/g++ -std=c++17
It seems like it should be set up to work:
pkgbuild::check_build_tools() Your system is ready to build packages!

The system seems to have c++17 available when I try
g++ -std=c++17
Could you advise me anyway to get this installed?

Issue with installation

Hi, I have installation issues, seems to be compilation issue, waiting for CRAN release!
devtools::install_github('bnprks/BPCells')
Downloading GitHub repo bnprks/BPCells@HEAD
── R CMD build ────────────────────────────────────────────────────────────────────────────
checking for file ‘/N/remotes88a5520d901/bnprks-BPCells✔ checking for file ‘/N/R/tmp/RtmpPMLyoa/remotes88a5520d901/bnprks-BPCells-9d1a4ca/DESCRIPTION’ (1s)
─ preparing ‘BPCells’: (4.4s)
✔ checking DESCRIPTION meta-information ...
─ cleaning src
─ running ‘cleanup’
─ checking for LF line-endings in source and make files and shell scripts (2s)
─ checking for empty or unneeded directories (936ms)
─ building ‘BPCells_0.1.0.tar.gz’
Warning in sprintf(gettext(fmt, domain = domain), ...) :
one argument not used by format 'invalid uid value replaced by that for user 'nobody''
Warning: invalid uid value replaced by that for user 'nobody'
Warning in sprintf(gettext(fmt, domain = domain), ...) :
one argument not used by format 'invalid gid value replaced by that for user 'nobody''
Warning: invalid gid value replaced by that for user 'nobody'

Installing package into ‘/R/x86_64-pc-linux-gnu-library/4.1’
(as ‘lib’ is unspecified)

installing source package ‘BPCells’ ...
** using staged installation
Testing hdf5 by compiling example program...
Found working hdf5
HDF5_CFLAGS=''
HDF5_LIBS='-lhdf5'

Testing architecture flag support support...'-march=native' succeeded
ARCH_FLAG='-march=native'
** libs
icpc -std=gnu++14 -I"/N/soft/rhel7/r/4.1.1/lib64/R/include" -DNDEBUG -I'/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include' -I'/R/x86_64-pc-linux-gnu-library/4.1/RcppEigen/include' -I/N/soft/rhel7/pcre2/10.34/include -march=native -std=c++17 -DRCPP_EIGEN -DEIGEN_PERMANENTLY_DISABLE_STUPID_WARNINGS -Wno-ignored-attributes -Wno-unknown-pragmas -fpic -g -O2 -c bitpacking_io.cpp -o bitpacking_io.o
icpc: command line warning #10148: option '-Wno-ignored-attributes' not supported
icpc -std=gnu++14 -I"/N/soft/rhel7/r/4.1.1/lib64/R/include" -DNDEBUG -I'/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include' -I'/R/x86_64-pc-linux-gnu-library/4.1/RcppEigen/include' -I/N/soft/rhel7/pcre2/10.34/include -march=native -std=c++17 -DRCPP_EIGEN -DEIGEN_PERMANENTLY_DISABLE_STUPID_WARNINGS -Wno-ignored-attributes -Wno-unknown-pragmas -fpic -g -O2 -c clustering.cpp -o clustering.o
icpc: command line warning #10148: option '-Wno-ignored-attributes' not supported
icpc -std=gnu++14 -I"/N/soft/rhel7/r/4.1.1/lib64/R/include" -DNDEBUG -I'/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include' -I'/R/x86_64-pc-linux-gnu-library/4.1/RcppEigen/include' -I/N/soft/rhel7/pcre2/10.34/include -march=native -std=c++17 -DRCPP_EIGEN -DEIGEN_PERMANENTLY_DISABLE_STUPID_WARNINGS -Wno-ignored-attributes -Wno-unknown-pragmas -fpic -g -O2 -c fragment_io.cpp -o fragment_io.o
icpc: command line warning #10148: option '-Wno-ignored-attributes' not supported
icpc -std=gnu++14 -I"/N/soft/rhel7/r/4.1.1/lib64/R/include" -DNDEBUG -I'/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include' -I'/R/x86_64-pc-linux-gnu-library/4.1/RcppEigen/include' -I/N/soft/rhel7/pcre2/10.34/include -march=native -std=c++17 -DRCPP_EIGEN -DEIGEN_PERMANENTLY_DISABLE_STUPID_WARNINGS -Wno-ignored-attributes -Wno-unknown-pragmas -fpic -g -O2 -c fragment_utils.cpp -o fragment_utils.o
icpc: command line warning #10148: option '-Wno-ignored-attributes' not supported
icpc -std=gnu++14 -I"/N/soft/rhel7/r/4.1.1/lib64/R/include" -DNDEBUG -I'/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include' -I'/R/x86_64-pc-linux-gnu-library/4.1/RcppEigen/include' -I/N/soft/rhel7/pcre2/10.34/include -march=native -std=c++17 -DRCPP_EIGEN -DEIGEN_PERMANENTLY_DISABLE_STUPID_WARNINGS -Wno-ignored-attributes -Wno-unknown-pragmas -fpic -g -O2 -c matrix_io.cpp -o matrix_io.o
icpc: command line warning #10148: option '-Wno-ignored-attributes' not supported
/geode2/soft/hps/rhel7/gcc/9.1.0/bin/../include/c++/9.1.0/tuple(553): error: pack "_UElements" does not have the same number of elements as "_Elements"
_and<is_nothrow_assignable<_Elements&, _UElements>...>::value;
^
detected during:
instantiation of "bool std::tuple<_Elements...>::__nothrow_assignable<_UElements...>() [with _Elements=<uint32_t={unsigned int}, uint32_t={unsigned int}, uint32_t={unsigned int}>, _UElements=<>]" at line 272 of "matrixIterators/StoredMatrixSorter.h"
instantiation of "void BPCells::StoredMatrixTransposeWriter::write(BPCells::MatrixLoader &, std::atomic *) [with T=uint32_t={unsigned int}]" at line 118 of "matrix_io.cpp"
implicit generation of "BPCells::StoredMatrixTransposeWriter::~StoredMatrixTransposeWriter() noexcept [with T=uint32_t={unsigned int}]" at line 118 of "matrix_io.cpp"
instantiation of class "BPCells::StoredMatrixTransposeWriter [with T=uint32_t={unsigned int}]" at line 118 of "matrix_io.cpp"
implicit generation of "BPCells::StoredMatrixTransposeWriter::StoredMatrixTransposeWriter(BPCells::WriterBuilder &, const char *, uint64_t={unsigned long}, uint64_t={unsigned long}, bool) [with T=uint32_t={unsigned int}]" at line 118 of "matrix_io.cpp"
instantiation of class "BPCells::StoredMatrixTransposeWriter [with T=uint32_t={unsigned int}]" at line 118 of "matrix_io.cpp"
instantiation of "SEXP write_matrix_transpose_dir(SEXP, std::string, std::string, uint64_t={unsigned long}, uint64_t={unsigned long}, bool) [with T=uint32_t={unsigned int}]" at line 139 of "matrix_io.cpp"

compilation aborted for matrix_io.cpp (code 2)
make: *** [matrix_io.o] Error 2
ERROR: compilation failed for package ‘BPCells’

How to choose rownames to use for cellranger .h5 file

Hello again,

When loading an .h5 file with Seurat function Read10X_h5, we can choose which annotation object to use for rownames, and, for example, use gene symbol instead of ENSG. Instead, when reading an .h5 file with BPCells function open_matrix_10x_hdf5, the matrix is loaded with ENSG as rownames.
I am not familiar with .h5 files, and I can't find a way to read the desired gene symbol as rownames when loading the matrix with BPCells package

`svds` behaves differently when `storage_order` is "row"

Hi, @bnprks. I followed the BPCells tutorials but computed SVD using svds() instead of irlba(). For example, in the RNA part:

svd <- svds(mat_norm, k=50)
pca <- multiply_cols(svd$v, svd$d)
cat(sprintf("PCA dimensions: %s\n", toString(dim(pca))))

I got the same results as irlba:

PCA dimensions: 2768, 50

However, when it came to ATAC's svd_atac, I got this:

svd_atac <- svds(mat_lsi_norm, 10)
pca_atac <- multiply_cols(svd_atac$v, svd_atac$d)
cat(sprintf("PCA dimensions: %s\n", toString(dim(pca_atac))))

PCA dimensions: 211807, 10

In other word, it seemed that the svd_atac$v and svd_atac$u were actually swapped, compared to RNA's svd$v and svd$u. When I check the input matrix, I found storage_order(mat_lsi_norm) was "row" while storage_order(mat_norm) was "col" as most scRNA-seq matrix in R would be.

Do you think BPCells::svds() should give u and v consistently regardless of storage_order, or should we just leave the users to check storage_order after calling it?

Dimension names lost in dgCMatrix %*% IterableMatrix multiply.

Hi,

I suspect that the matrix multiply for a dgCMatrix %*% IterableMatrix does not include the names for at least one of the dimensions. Does that make sense? Here is what I see

The R code looks like

  mapping <- my.dMcast(groupings2, form)
  colnames(mapping) <- substring(colnames(mapping), 2)
  result <- Matrix::t(mapping) %*% x
  # result has no colnames
  browser()

and the mapping, x, and result objects look like

Browse[1]> str(mapping)
Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
  ..@ i       : int [1:42035] 6 8 19 20 23 24 27 28 30 41 ...
  ..@ p       : int [1:32] 0 13963 14356 14710 14989 15234 15476 15716 15892 15998 ...
  ..@ Dim     : int [1:2] 42035 31
  ..@ Dimnames:List of 2
  .. ..$ : NULL
  .. ..$ : chr [1:31] "1" "10" "11" "12" ...
  ..@ x       : num [1:42035] 1 1 1 1 1 1 1 1 1 1 ...
  ..@ factors : list()
  ..$ assign   : int [1:31] 1 1 1 1 1 1 1 1 1 1 ...
  ..$ contrasts:List of 1
  .. ..$ A:Formal class 'ddiMatrix' [package "Matrix"] with 4 slots
  .. .. .. ..@ diag    : chr "U"
  .. .. .. ..@ Dim     : int [1:2] 31 31
  .. .. .. ..@ Dimnames:List of 2
  .. .. .. .. ..$ : chr [1:31] "1" "10" "11" "12" ...
  .. .. .. .. ..$ : chr [1:31] "1" "10" "11" "12" ...
  .. .. .. ..@ x       : num(0)

Browse[1]> str(x)
Formal class 'ConvertMatrixType' [package "BPCells"] with 5 slots
  ..@ matrix   :Formal class 'TransformBinarize' [package "BPCells"] with 7 slots
  .. .. ..@ matrix       :Formal class 'MatrixSubset' [package "BPCells"] with 7 slots
  .. .. .. .. ..@ matrix       :Formal class 'MatrixDir' [package "BPCells"] with 7 slots
  .. .. .. .. .. .. ..@ dir        : chr "/home/brent/git/monocle3/monocle.bpcells.20230901.cfd034512a5b.tmp"
  .. .. .. .. .. .. ..@ compressed : logi FALSE
  .. .. .. .. .. .. ..@ buffer_size: int 8192
  .. .. .. .. .. .. ..@ type       : chr "double"
  .. .. .. .. .. .. ..@ dim        : int [1:2] 42035 20271
  .. .. .. .. .. .. ..@ transpose  : logi TRUE
  .. .. .. .. .. .. ..@ dimnames   :List of 2
  .. .. .. .. .. .. .. ..$ : chr [1:42035] "cele-001-001.CATGACTCAA" "cele-001-001.AAGACGGCCA" "cele-001-001.GCCAACGCCA" "cele-001-001.ATAGGAGTAC" ...
  .. .. .. .. .. .. .. ..$ : chr [1:20271] "WBGene00000001" "WBGene00000002" "WBGene00000003" "WBGene00000004" ...
  .. .. .. .. ..@ row_selection: int(0)
  .. .. .. .. ..@ col_selection: Named int [1:42035] 1 2 3 4 5 6 7 8 9 10 ...
  .. .. .. .. .. ..- attr(*, "names")= chr [1:42035] "cele-001-001.CATGACTCAA" "cele-001-001.AAGACGGCCA" "cele-001-001.GCCAACGCCA" "cele-001-001.ATAGGAGTAC" ...
  .. .. .. .. ..@ zero_dims    : logi [1:2] FALSE FALSE
  .. .. .. .. ..@ dim          : int [1:2] 42035 20271
  .. .. .. .. ..@ transpose    : logi TRUE
  .. .. .. .. ..@ dimnames     :List of 2
  .. .. .. .. .. ..$ : chr [1:42035] "cele-001-001.CATGACTCAA" "cele-001-001.AAGACGGCCA" "cele-001-001.GCCAACGCCA" "cele-001-001.ATAGGAGTAC" ...
  .. .. .. .. .. ..$ : chr [1:20271] "WBGene00000001" "WBGene00000002" "WBGene00000003" "WBGene00000004" ...
  .. .. ..@ row_params   : num[0 , 0 ]
  .. .. ..@ col_params   : num[0 , 0 ]
  .. .. ..@ global_params: num [1:2] 0 1
  .. .. ..@ dim          : int [1:2] 42035 20271
  .. .. ..@ transpose    : logi TRUE
  .. .. ..@ dimnames     :List of 2
  .. .. .. ..$ : NULL
  .. .. .. ..$ : NULL
  ..@ type     : chr "uint32_t"
  ..@ dim      : int [1:2] 42035 20271
  ..@ transpose: logi TRUE
  ..@ dimnames :List of 2
  .. ..$ : chr [1:42035] "cele-001-001.CATGACTCAA" "cele-001-001.AAGACGGCCA" "cele-001-001.GCCAACGCCA" "cele-001-001.ATAGGAGTAC" ...
  .. ..$ : chr [1:20271] "WBGene00000001" "WBGene00000002" "WBGene00000003" "WBGene00000004" ...

Browse[1]> str(result)
Formal class 'MatrixMultiply' [package "BPCells"] with 5 slots
  ..@ left     :Formal class 'TransformBinarize' [package "BPCells"] with 7 slots
  .. .. ..@ matrix       :Formal class 'MatrixSubset' [package "BPCells"] with 7 slots
  .. .. .. .. ..@ matrix       :Formal class 'MatrixDir' [package "BPCells"] with 7 slots
  .. .. .. .. .. .. ..@ dir        : chr "/home/brent/git/monocle3/monocle.bpcells.20230901.cfd034512a5b.tmp"
  .. .. .. .. .. .. ..@ compressed : logi FALSE
  .. .. .. .. .. .. ..@ buffer_size: int 8192
  .. .. .. .. .. .. ..@ type       : chr "double"
  .. .. .. .. .. .. ..@ dim        : int [1:2] 42035 20271
  .. .. .. .. .. .. ..@ transpose  : logi TRUE
  .. .. .. .. .. .. ..@ dimnames   :List of 2
  .. .. .. .. .. .. .. ..$ : chr [1:42035] "cele-001-001.CATGACTCAA" "cele-001-001.AAGACGGCCA" "cele-001-001.GCCAACGCCA" "cele-001-001.ATAGGAGTAC" ...
  .. .. .. .. .. .. .. ..$ : chr [1:20271] "WBGene00000001" "WBGene00000002" "WBGene00000003" "WBGene00000004" ...
  .. .. .. .. ..@ row_selection: int(0)
  .. .. .. .. ..@ col_selection: Named int [1:42035] 1 2 3 4 5 6 7 8 9 10 ...
  .. .. .. .. .. ..- attr(*, "names")= chr [1:42035] "cele-001-001.CATGACTCAA" "cele-001-001.AAGACGGCCA" "cele-001-001.GCCAACGCCA" "cele-001-001.ATAGGAGTAC" ...
  .. .. .. .. ..@ zero_dims    : logi [1:2] FALSE FALSE
  .. .. .. .. ..@ dim          : int [1:2] 42035 20271
  .. .. .. .. ..@ transpose    : logi TRUE
  .. .. .. .. ..@ dimnames     :List of 2
  .. .. .. .. .. ..$ : chr [1:42035] "cele-001-001.CATGACTCAA" "cele-001-001.AAGACGGCCA" "cele-001-001.GCCAACGCCA" "cele-001-001.ATAGGAGTAC" ...
  .. .. .. .. .. ..$ : chr [1:20271] "WBGene00000001" "WBGene00000002" "WBGene00000003" "WBGene00000004" ...
  .. .. ..@ row_params   : num[0 , 0 ]
  .. .. ..@ col_params   : num[0 , 0 ]
  .. .. ..@ global_params: num [1:2] 0 1
  .. .. ..@ dim          : int [1:2] 42035 20271
  .. .. ..@ transpose    : logi TRUE
  .. .. ..@ dimnames     :List of 2
  .. .. .. ..$ : NULL
  .. .. .. ..$ : NULL
  ..@ right    :Formal class 'Iterable_dgCMatrix_wrapper' [package "BPCells"] with 4 slots
  .. .. ..@ mat      :Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
  .. .. .. .. ..@ i       : int [1:42035] 6 8 19 20 23 24 27 28 30 41 ...
  .. .. .. .. ..@ p       : int [1:32] 0 13963 14356 14710 14989 15234 15476 15716 15892 15998 ...
  .. .. .. .. ..@ Dim     : int [1:2] 42035 31
  .. .. .. .. ..@ Dimnames:List of 2
  .. .. .. .. .. ..$ : NULL
  .. .. .. .. .. ..$ : chr [1:31] "1" "10" "11" "12" ...
  .. .. .. .. ..@ x       : num [1:42035] 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. .. ..@ factors : list()
  .. .. ..@ dim      : int [1:2] 31 42035
  .. .. ..@ transpose: logi TRUE
  .. .. ..@ dimnames :List of 2
  .. .. .. ..$ : chr [1:31] "1" "10" "11" "12" ...
  .. .. .. ..$ : NULL
  ..@ dim      : int [1:2] 31 20271
  ..@ transpose: logi TRUE
  ..@ dimnames :List of 2
  .. ..$ : chr [1:31] "1" "10" "11" "12" ...
  .. ..$ : NULL


Browse[2]> str(as(result, 'dgCMatrix'))
Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
  ..@ i       : int [1:213536] 0 1 2 4 5 6 8 9 11 21 ...
  ..@ p       : int [1:20272] 0 17 31 48 68 73 93 100 105 108 ...
  ..@ Dim     : int [1:2] 31 20271
  ..@ Dimnames:List of 2
  .. ..$ : chr [1:31] "1" "10" "11" "12" ...
  .. ..$ : NULL
  ..@ x       : num [1:213536] 157 7 3 3 1 1 14 2 581 1 ...
  ..@ factors : list()

I appreciate your consideration and guidance.

Thank you.

Ever grateful,
Brent

support on qs file?

BPCells is really an exciting tool for large datasets!
For large datasets, since saveRDS and readRDS are too slow, will BPCells have support on qs file reading and writing in the future?Especially for the behaviour of queued operations of reading on-disk ount matrix.

Installation on Mac Clang Issue?

Hi,

Trying to install the package and running into issue that at first seemed similar to #3. Following those instructions I ran clang install via conda and added lines to my RMakevars.

# clang: start
CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include
CC=/Users/marsh_mbp/opt/anaconda3/envs/basic/bin/clang
CXX=/Users/marsh_mbp/opt/anaconda3/envs/basic/bin/clang-cpp

SHLIB_CXXLDFLAGS+=-Wl,-rpath,${R_HOME}/lib ${R_HOME}/lib/libc++abi.1.dylib
SHLIB_CXX14LDFLAGS+=-Wl,-rpath,${R_HOME}/lib ${R_HOME}/lib/libc++abi.1.dylib
# clang: end

Now I no longer get the error about not having right compiler but I still get non-zero exit status and failed install. Full install warnings and errors and session info below.

Thanks!!
Sam

Build Messages

Running `R CMD build`...
* checking for file ‘/private/var/folders/92/rwb8659d3jl658jf5nmmvw580000gn/T/RtmpTTpNt9/remotes82b74e12e8b9/bnprks-BPCells-2af1b98/DESCRIPTION’ ... OK
* preparing ‘BPCells’:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* running ‘cleanup’
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* building ‘BPCells_0.1.0.tar.gz’
* installing *source* package ‘BPCells’ ...
** using staged installation
Testing hdf5 by compiling example program...
Found working hdf5
HDF5_CFLAGS=''
HDF5_LIBS='-lhdf5'

Testing architecture flag support support...'-march=native' succeeded
ARCH_FLAG='-march=native'
** libs
using C++ compiler: ‘Apple clang version 13.0.0 (clang-1300.0.27.3)’
using SDK: ‘’
clang++ -arch x86_64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/Rcpp/include' -I'/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include' -I/opt/R/x86_64/include   -march=native -std=c++17 -DRCPP_EIGEN -DEIGEN_PERMANENTLY_DISABLE_STUPID_WARNINGS -Wno-ignored-attributes -Wno-unknown-pragmas  -fPIC  -falign-functions=64 -Wall -g -O2  -c bitpacking_io.cpp -o bitpacking_io.o
clang++ -arch x86_64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/Rcpp/include' -I'/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include' -I/opt/R/x86_64/include   -march=native -std=c++17 -DRCPP_EIGEN -DEIGEN_PERMANENTLY_DISABLE_STUPID_WARNINGS -Wno-ignored-attributes -Wno-unknown-pragmas  -fPIC  -falign-functions=64 -Wall -g -O2  -c fragment_io.cpp -o fragment_io.o
clang++ -arch x86_64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/Rcpp/include' -I'/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include' -I/opt/R/x86_64/include   -march=native -std=c++17 -DRCPP_EIGEN -DEIGEN_PERMANENTLY_DISABLE_STUPID_WARNINGS -Wno-ignored-attributes -Wno-unknown-pragmas  -fPIC  -falign-functions=64 -Wall -g -O2  -c fragment_utils.cpp -o fragment_utils.o
In file included from fragment_utils.cpp:6:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigen.h:25:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigenForward.h:30:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/unsupported/Eigen/../../Eigen/Core:367:
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/Meta.h:320:25: warning: 'result_of<Eigen::internal::scalar_product_op<double> (const double &, const double &)>' is deprecated [-Wdeprecated-declarations]
  typedef typename std::result_of<T>::type type1;
                        ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:33:20: note: in instantiation of template class 'Eigen::internal::result_of<Eigen::internal::scalar_product_op<double> (const double &, const double &)>' requested here
  typedef typename result_of<
                   ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/XprHelper.h:497:56: note: in instantiation of template class 'Eigen::internal::traits<Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Matrix<double, 1, -1, 1>>, const Eigen::Block<const Eigen::Map<Eigen::Matrix<double, -1, -1, 0>, 0>, 1, -1, false>>>' requested here
template<typename Derived, typename XprKind = typename traits<Derived>::XprKind, typename StorageKind = typename traits<Derived>::StorageKind>
                                                       ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:149:22: note: in instantiation of default argument for 'generic_xpr_base<Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Matrix<double, 1, -1, 1>>, const Eigen::Block<const Eigen::Map<Eigen::Matrix<double, -1, -1, 0>, 0>, 1, -1, false>>>' required here
  : public internal::generic_xpr_base<CwiseBinaryOp<BinaryOp, Lhs, Rhs> >::type
                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:78:10: note: in instantiation of template class 'Eigen::CwiseBinaryOpImpl<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Matrix<double, 1, -1, 1>>, const Eigen::Block<const Eigen::Map<Eigen::Matrix<double, -1, -1, 0>, 0>, 1, -1, false>, Eigen::Dense>' requested here
  public CwiseBinaryOpImpl<
         ^
./matrixIterators/MatrixOps.h:32:63: note: in instantiation of template class 'Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Matrix<double, 1, -1, 1>>, const Eigen::Block<const Eigen::Map<Eigen::Matrix<double, -1, -1, 0>, 0>, 1, -1, false>>' requested here
                res.col(row_data[i]) += ((double)val_data[i]) * B.row(col);
                                                              ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/type_traits:4009:34: note: 'result_of<Eigen::internal::scalar_product_op<double> (const double &, const double &)>' has been explicitly marked deprecated here
template <class _Callable> class _LIBCPP_DEPRECATED_IN_CXX17 result_of;
                                 ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1066:39: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17'
#  define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED
                                      ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1043:48: note: expanded from macro '_LIBCPP_DEPRECATED'
#    define _LIBCPP_DEPRECATED __attribute__ ((deprecated))
                                               ^
In file included from fragment_utils.cpp:6:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigen.h:25:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigenForward.h:30:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/unsupported/Eigen/../../Eigen/Core:367:
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/Meta.h:320:25: warning: 'result_of<Eigen::internal::scalar_quotient_op<double> (const double &, const double &)>' is deprecated [-Wdeprecated-declarations]
  typedef typename std::result_of<T>::type type1;
                        ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:33:20: note: in instantiation of template class 'Eigen::internal::result_of<Eigen::internal::scalar_quotient_op<double> (const double &, const double &)>' requested here
  typedef typename result_of<
                   ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/XprHelper.h:497:56: note: in instantiation of template class 'Eigen::internal::traits<Eigen::CwiseBinaryOp<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>>>' requested here
template<typename Derived, typename XprKind = typename traits<Derived>::XprKind, typename StorageKind = typename traits<Derived>::StorageKind>
                                                       ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:149:22: note: in instantiation of default argument for 'generic_xpr_base<Eigen::CwiseBinaryOp<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>>>' required here
  : public internal::generic_xpr_base<CwiseBinaryOp<BinaryOp, Lhs, Rhs> >::type
                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:78:10: note: in instantiation of template class 'Eigen::CwiseBinaryOpImpl<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, Eigen::Dense>' requested here
  public CwiseBinaryOpImpl<
         ^
./matrixIterators/MatrixOps.h:246:43: note: in instantiation of template class 'Eigen::CwiseBinaryOp<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>>' requested here
        res.col_stats.row(1) = mean * nnz / n_rows;
                                          ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/type_traits:4009:34: note: 'result_of<Eigen::internal::scalar_quotient_op<double> (const double &, const double &)>' has been explicitly marked deprecated here
template <class _Callable> class _LIBCPP_DEPRECATED_IN_CXX17 result_of;
                                 ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1066:39: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17'
#  define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED
                                      ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1043:48: note: expanded from macro '_LIBCPP_DEPRECATED'
#    define _LIBCPP_DEPRECATED __attribute__ ((deprecated))
                                               ^
In file included from fragment_utils.cpp:6:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigen.h:25:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigenForward.h:30:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/unsupported/Eigen/../../Eigen/Core:367:
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/Meta.h:320:25: warning: 'result_of<Eigen::internal::scalar_square_op<double> (const double &)>' is deprecated [-Wdeprecated-declarations]
  typedef typename std::result_of<T>::type type1;
                        ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseUnaryOp.h:21:20: note: in instantiation of template class 'Eigen::internal::result_of<Eigen::internal::scalar_square_op<double> (const double &)>' requested here
  typedef typename result_of<
                   ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/XprHelper.h:497:56: note: in instantiation of template class 'Eigen::internal::traits<Eigen::CwiseUnaryOp<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>' requested here
template<typename Derived, typename XprKind = typename traits<Derived>::XprKind, typename StorageKind = typename traits<Derived>::StorageKind>
                                                       ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseUnaryOp.h:95:22: note: in instantiation of default argument for 'generic_xpr_base<Eigen::CwiseUnaryOp<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>' required here
  : public internal::generic_xpr_base<CwiseUnaryOp<UnaryOp, XprType> >::type
                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseUnaryOp.h:55:29: note: in instantiation of template class 'Eigen::CwiseUnaryOpImpl<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, Eigen::Dense>' requested here
class CwiseUnaryOp : public CwiseUnaryOpImpl<UnaryOp, XprType, typename internal::traits<XprType>::StorageKind>, internal::no_assignment_operator
                            ^
./matrixIterators/MatrixOps.h:253:24: note: in instantiation of template class 'Eigen::CwiseUnaryOp<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>' requested here
            (m2 + mean.square() * nnz * ((double)n_rows - nnz) / n_rows) / (n_rows - 1);
                       ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/type_traits:4009:34: note: 'result_of<Eigen::internal::scalar_square_op<double> (const double &)>' has been explicitly marked deprecated here
template <class _Callable> class _LIBCPP_DEPRECATED_IN_CXX17 result_of;
                                 ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1066:39: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17'
#  define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED
                                      ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1043:48: note: expanded from macro '_LIBCPP_DEPRECATED'
#    define _LIBCPP_DEPRECATED __attribute__ ((deprecated))
                                               ^
In file included from fragment_utils.cpp:6:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigen.h:25:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigenForward.h:30:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/unsupported/Eigen/../../Eigen/Core:367:
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/Meta.h:320:25: warning: 'result_of<Eigen::internal::scalar_difference_op<double> (const double &, const double &)>' is deprecated [-Wdeprecated-declarations]
  typedef typename std::result_of<T>::type type1;
                        ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:33:20: note: in instantiation of template class 'Eigen::internal::result_of<Eigen::internal::scalar_difference_op<double> (const double &, const double &)>' requested here
  typedef typename result_of<
                   ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/XprHelper.h:497:56: note: in instantiation of template class 'Eigen::internal::traits<Eigen::CwiseBinaryOp<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>' requested here
template<typename Derived, typename XprKind = typename traits<Derived>::XprKind, typename StorageKind = typename traits<Derived>::StorageKind>
                                                       ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:149:22: note: in instantiation of default argument for 'generic_xpr_base<Eigen::CwiseBinaryOp<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>' required here
  : public internal::generic_xpr_base<CwiseBinaryOp<BinaryOp, Lhs, Rhs> >::type
                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:78:10: note: in instantiation of template class 'Eigen::CwiseBinaryOpImpl<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, Eigen::Dense>' requested here
  public CwiseBinaryOpImpl<
         ^
./matrixIterators/MatrixOps.h:253:57: note: in instantiation of template class 'Eigen::CwiseBinaryOp<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>' requested here
            (m2 + mean.square() * nnz * ((double)n_rows - nnz) / n_rows) / (n_rows - 1);
                                                        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/type_traits:4009:34: note: 'result_of<Eigen::internal::scalar_difference_op<double> (const double &, const double &)>' has been explicitly marked deprecated here
template <class _Callable> class _LIBCPP_DEPRECATED_IN_CXX17 result_of;
                                 ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1066:39: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17'
#  define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED
                                      ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1043:48: note: expanded from macro '_LIBCPP_DEPRECATED'
#    define _LIBCPP_DEPRECATED __attribute__ ((deprecated))
                                               ^
In file included from fragment_utils.cpp:6:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigen.h:25:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigenForward.h:30:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/unsupported/Eigen/../../Eigen/Core:367:
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/Meta.h:320:25: warning: 'result_of<Eigen::internal::scalar_sum_op<double> (const double &, const double &)>' is deprecated [-Wdeprecated-declarations]
  typedef typename std::result_of<T>::type type1;
                        ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:33:20: note: in instantiation of template class 'Eigen::internal::result_of<Eigen::internal::scalar_sum_op<double> (const double &, const double &)>' requested here
  typedef typename result_of<
                   ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/XprHelper.h:497:56: note: in instantiation of template class 'Eigen::internal::traits<Eigen::CwiseBinaryOp<Eigen::internal::scalar_sum_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseUnaryOp<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>>>>' requested here
template<typename Derived, typename XprKind = typename traits<Derived>::XprKind, typename StorageKind = typename traits<Derived>::StorageKind>
                                                       ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:149:22: note: in instantiation of default argument for 'generic_xpr_base<Eigen::CwiseBinaryOp<Eigen::internal::scalar_sum_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseUnaryOp<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>>>>' required here
  : public internal::generic_xpr_base<CwiseBinaryOp<BinaryOp, Lhs, Rhs> >::type
                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:78:10: note: in instantiation of template class 'Eigen::CwiseBinaryOpImpl<Eigen::internal::scalar_sum_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseUnaryOp<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>>, Eigen::Dense>' requested here
  public CwiseBinaryOpImpl<
         ^
./matrixIterators/MatrixOps.h:253:17: note: in instantiation of template class 'Eigen::CwiseBinaryOp<Eigen::internal::scalar_sum_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseUnaryOp<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>>>' requested here
            (m2 + mean.square() * nnz * ((double)n_rows - nnz) / n_rows) / (n_rows - 1);
                ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/type_traits:4009:34: note: 'result_of<Eigen::internal::scalar_sum_op<double> (const double &, const double &)>' has been explicitly marked deprecated here
template <class _Callable> class _LIBCPP_DEPRECATED_IN_CXX17 result_of;
                                 ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1066:39: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17'
#  define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED
                                      ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1043:48: note: expanded from macro '_LIBCPP_DEPRECATED'
#    define _LIBCPP_DEPRECATED __attribute__ ((deprecated))
                                               ^
5 warnings generated.
clang++ -arch x86_64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/Rcpp/include' -I'/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include' -I/opt/R/x86_64/include   -march=native -std=c++17 -DRCPP_EIGEN -DEIGEN_PERMANENTLY_DISABLE_STUPID_WARNINGS -Wno-ignored-attributes -Wno-unknown-pragmas  -fPIC  -falign-functions=64 -Wall -g -O2  -c matrix_io.cpp -o matrix_io.o
In file included from matrix_io.cpp:5:
In file included from ./matrixIterators/ImportMatrixHDF5.h:5:
In file included from ./matrixIterators/StoredMatrix.h:5:
In file included from ./matrixIterators/MatrixIterator.h:14:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigen.h:25:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigenForward.h:30:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/unsupported/Eigen/../../Eigen/Core:367:
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/Meta.h:320:25: warning: 'result_of<Eigen::internal::scalar_product_op<double> (const double &, const double &)>' is deprecated [-Wdeprecated-declarations]
  typedef typename std::result_of<T>::type type1;
                        ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:33:20: note: in instantiation of template class 'Eigen::internal::result_of<Eigen::internal::scalar_product_op<double> (const double &, const double &)>' requested here
  typedef typename result_of<
                   ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/XprHelper.h:497:56: note: in instantiation of template class 'Eigen::internal::traits<Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Matrix<double, 1, -1, 1>>, const Eigen::Block<const Eigen::Map<Eigen::Matrix<double, -1, -1, 0>, 0>, 1, -1, false>>>' requested here
template<typename Derived, typename XprKind = typename traits<Derived>::XprKind, typename StorageKind = typename traits<Derived>::StorageKind>
                                                       ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:149:22: note: in instantiation of default argument for 'generic_xpr_base<Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Matrix<double, 1, -1, 1>>, const Eigen::Block<const Eigen::Map<Eigen::Matrix<double, -1, -1, 0>, 0>, 1, -1, false>>>' required here
  : public internal::generic_xpr_base<CwiseBinaryOp<BinaryOp, Lhs, Rhs> >::type
                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:78:10: note: in instantiation of template class 'Eigen::CwiseBinaryOpImpl<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Matrix<double, 1, -1, 1>>, const Eigen::Block<const Eigen::Map<Eigen::Matrix<double, -1, -1, 0>, 0>, 1, -1, false>, Eigen::Dense>' requested here
  public CwiseBinaryOpImpl<
         ^
./matrixIterators/MatrixOps.h:32:63: note: in instantiation of template class 'Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Matrix<double, 1, -1, 1>>, const Eigen::Block<const Eigen::Map<Eigen::Matrix<double, -1, -1, 0>, 0>, 1, -1, false>>' requested here
                res.col(row_data[i]) += ((double)val_data[i]) * B.row(col);
                                                              ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/type_traits:4009:34: note: 'result_of<Eigen::internal::scalar_product_op<double> (const double &, const double &)>' has been explicitly marked deprecated here
template <class _Callable> class _LIBCPP_DEPRECATED_IN_CXX17 result_of;
                                 ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1066:39: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17'
#  define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED
                                      ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1043:48: note: expanded from macro '_LIBCPP_DEPRECATED'
#    define _LIBCPP_DEPRECATED __attribute__ ((deprecated))
                                               ^
In file included from matrix_io.cpp:5:
In file included from ./matrixIterators/ImportMatrixHDF5.h:5:
In file included from ./matrixIterators/StoredMatrix.h:5:
In file included from ./matrixIterators/MatrixIterator.h:14:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigen.h:25:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigenForward.h:30:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/unsupported/Eigen/../../Eigen/Core:367:
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/Meta.h:320:25: warning: 'result_of<Eigen::internal::scalar_quotient_op<double> (const double &, const double &)>' is deprecated [-Wdeprecated-declarations]
  typedef typename std::result_of<T>::type type1;
                        ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:33:20: note: in instantiation of template class 'Eigen::internal::result_of<Eigen::internal::scalar_quotient_op<double> (const double &, const double &)>' requested here
  typedef typename result_of<
                   ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/XprHelper.h:497:56: note: in instantiation of template class 'Eigen::internal::traits<Eigen::CwiseBinaryOp<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>>>' requested here
template<typename Derived, typename XprKind = typename traits<Derived>::XprKind, typename StorageKind = typename traits<Derived>::StorageKind>
                                                       ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:149:22: note: in instantiation of default argument for 'generic_xpr_base<Eigen::CwiseBinaryOp<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>>>' required here
  : public internal::generic_xpr_base<CwiseBinaryOp<BinaryOp, Lhs, Rhs> >::type
                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:78:10: note: in instantiation of template class 'Eigen::CwiseBinaryOpImpl<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, Eigen::Dense>' requested here
  public CwiseBinaryOpImpl<
         ^
./matrixIterators/MatrixOps.h:246:43: note: in instantiation of template class 'Eigen::CwiseBinaryOp<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>>' requested here
        res.col_stats.row(1) = mean * nnz / n_rows;
                                          ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/type_traits:4009:34: note: 'result_of<Eigen::internal::scalar_quotient_op<double> (const double &, const double &)>' has been explicitly marked deprecated here
template <class _Callable> class _LIBCPP_DEPRECATED_IN_CXX17 result_of;
                                 ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1066:39: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17'
#  define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED
                                      ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1043:48: note: expanded from macro '_LIBCPP_DEPRECATED'
#    define _LIBCPP_DEPRECATED __attribute__ ((deprecated))
                                               ^
In file included from matrix_io.cpp:5:
In file included from ./matrixIterators/ImportMatrixHDF5.h:5:
In file included from ./matrixIterators/StoredMatrix.h:5:
In file included from ./matrixIterators/MatrixIterator.h:14:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigen.h:25:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigenForward.h:30:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/unsupported/Eigen/../../Eigen/Core:367:
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/Meta.h:320:25: warning: 'result_of<Eigen::internal::scalar_square_op<double> (const double &)>' is deprecated [-Wdeprecated-declarations]
  typedef typename std::result_of<T>::type type1;
                        ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseUnaryOp.h:21:20: note: in instantiation of template class 'Eigen::internal::result_of<Eigen::internal::scalar_square_op<double> (const double &)>' requested here
  typedef typename result_of<
                   ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/XprHelper.h:497:56: note: in instantiation of template class 'Eigen::internal::traits<Eigen::CwiseUnaryOp<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>' requested here
template<typename Derived, typename XprKind = typename traits<Derived>::XprKind, typename StorageKind = typename traits<Derived>::StorageKind>
                                                       ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseUnaryOp.h:95:22: note: in instantiation of default argument for 'generic_xpr_base<Eigen::CwiseUnaryOp<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>' required here
  : public internal::generic_xpr_base<CwiseUnaryOp<UnaryOp, XprType> >::type
                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseUnaryOp.h:55:29: note: in instantiation of template class 'Eigen::CwiseUnaryOpImpl<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, Eigen::Dense>' requested here
class CwiseUnaryOp : public CwiseUnaryOpImpl<UnaryOp, XprType, typename internal::traits<XprType>::StorageKind>, internal::no_assignment_operator
                            ^
./matrixIterators/MatrixOps.h:253:24: note: in instantiation of template class 'Eigen::CwiseUnaryOp<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>' requested here
            (m2 + mean.square() * nnz * ((double)n_rows - nnz) / n_rows) / (n_rows - 1);
                       ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/type_traits:4009:34: note: 'result_of<Eigen::internal::scalar_square_op<double> (const double &)>' has been explicitly marked deprecated here
template <class _Callable> class _LIBCPP_DEPRECATED_IN_CXX17 result_of;
                                 ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1066:39: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17'
#  define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED
                                      ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1043:48: note: expanded from macro '_LIBCPP_DEPRECATED'
#    define _LIBCPP_DEPRECATED __attribute__ ((deprecated))
                                               ^
In file included from matrix_io.cpp:5:
In file included from ./matrixIterators/ImportMatrixHDF5.h:5:
In file included from ./matrixIterators/StoredMatrix.h:5:
In file included from ./matrixIterators/MatrixIterator.h:14:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigen.h:25:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigenForward.h:30:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/unsupported/Eigen/../../Eigen/Core:367:
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/Meta.h:320:25: warning: 'result_of<Eigen::internal::scalar_difference_op<double> (const double &, const double &)>' is deprecated [-Wdeprecated-declarations]
  typedef typename std::result_of<T>::type type1;
                        ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:33:20: note: in instantiation of template class 'Eigen::internal::result_of<Eigen::internal::scalar_difference_op<double> (const double &, const double &)>' requested here
  typedef typename result_of<
                   ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/XprHelper.h:497:56: note: in instantiation of template class 'Eigen::internal::traits<Eigen::CwiseBinaryOp<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>' requested here
template<typename Derived, typename XprKind = typename traits<Derived>::XprKind, typename StorageKind = typename traits<Derived>::StorageKind>
                                                       ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:149:22: note: in instantiation of default argument for 'generic_xpr_base<Eigen::CwiseBinaryOp<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>' required here
  : public internal::generic_xpr_base<CwiseBinaryOp<BinaryOp, Lhs, Rhs> >::type
                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:78:10: note: in instantiation of template class 'Eigen::CwiseBinaryOpImpl<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, Eigen::Dense>' requested here
  public CwiseBinaryOpImpl<
         ^
./matrixIterators/MatrixOps.h:253:57: note: in instantiation of template class 'Eigen::CwiseBinaryOp<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>' requested here
            (m2 + mean.square() * nnz * ((double)n_rows - nnz) / n_rows) / (n_rows - 1);
                                                        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/type_traits:4009:34: note: 'result_of<Eigen::internal::scalar_difference_op<double> (const double &, const double &)>' has been explicitly marked deprecated here
template <class _Callable> class _LIBCPP_DEPRECATED_IN_CXX17 result_of;
                                 ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1066:39: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17'
#  define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED
                                      ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1043:48: note: expanded from macro '_LIBCPP_DEPRECATED'
#    define _LIBCPP_DEPRECATED __attribute__ ((deprecated))
                                               ^
In file included from matrix_io.cpp:5:
In file included from ./matrixIterators/ImportMatrixHDF5.h:5:
In file included from ./matrixIterators/StoredMatrix.h:5:
In file included from ./matrixIterators/MatrixIterator.h:14:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigen.h:25:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/RcppEigenForward.h:30:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/unsupported/Eigen/../../Eigen/Core:367:
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/Meta.h:320:25: warning: 'result_of<Eigen::internal::scalar_sum_op<double> (const double &, const double &)>' is deprecated [-Wdeprecated-declarations]
  typedef typename std::result_of<T>::type type1;
                        ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:33:20: note: in instantiation of template class 'Eigen::internal::result_of<Eigen::internal::scalar_sum_op<double> (const double &, const double &)>' requested here
  typedef typename result_of<
                   ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/util/XprHelper.h:497:56: note: in instantiation of template class 'Eigen::internal::traits<Eigen::CwiseBinaryOp<Eigen::internal::scalar_sum_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseUnaryOp<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>>>>' requested here
template<typename Derived, typename XprKind = typename traits<Derived>::XprKind, typename StorageKind = typename traits<Derived>::StorageKind>
                                                       ^
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:149:22: note: in instantiation of default argument for 'generic_xpr_base<Eigen::CwiseBinaryOp<Eigen::internal::scalar_sum_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseUnaryOp<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>>>>' required here
  : public internal::generic_xpr_base<CwiseBinaryOp<BinaryOp, Lhs, Rhs> >::type
                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/RcppEigen/include/Eigen/src/Core/CwiseBinaryOp.h:78:10: note: in instantiation of template class 'Eigen::CwiseBinaryOpImpl<Eigen::internal::scalar_sum_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseUnaryOp<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>>, Eigen::Dense>' requested here
  public CwiseBinaryOpImpl<
         ^
./matrixIterators/MatrixOps.h:253:17: note: in instantiation of template class 'Eigen::CwiseBinaryOp<Eigen::internal::scalar_sum_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_quotient_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double>, const Eigen::CwiseUnaryOp<Eigen::internal::scalar_square_op<double>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>, const Eigen::CwiseBinaryOp<Eigen::internal::scalar_difference_op<double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>, const Eigen::Block<Eigen::Array<double, -1, -1, 0>, 1, -1, false>>>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Array<double, 1, -1, 1>>>>' requested here
            (m2 + mean.square() * nnz * ((double)n_rows - nnz) / n_rows) / (n_rows - 1);
                ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/type_traits:4009:34: note: 'result_of<Eigen::internal::scalar_sum_op<double> (const double &, const double &)>' has been explicitly marked deprecated here
template <class _Callable> class _LIBCPP_DEPRECATED_IN_CXX17 result_of;
                                 ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1066:39: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17'
#  define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED
                                      ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__config:1043:48: note: expanded from macro '_LIBCPP_DEPRECATED'
#    define _LIBCPP_DEPRECATED __attribute__ ((deprecated))
                                               ^
In file included from matrix_io.cpp:9:
./matrixIterators/StoredMatrixTransposeWriter.h:39:35: error: no matching function for call to 'min'
            uint32_t copy_count = std::min(row.size() - loaded, available);
                                  ^~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__algorithm/min.h:39:1: note: candidate template ignored: deduced conflicting types for parameter '_Tp' ('unsigned long long' vs. 'unsigned long')
min(const _Tp& __a, const _Tp& __b)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__algorithm/min.h:50:1: note: candidate template ignored: could not match 'initializer_list<type-parameter-0-0>' against 'unsigned long long'
min(initializer_list<_Tp> __t, _Compare __comp)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__algorithm/min.h:59:1: note: candidate function template not viable: requires single argument '__t', but 2 arguments were provided
min(initializer_list<_Tp> __t)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/__algorithm/min.h:30:1: note: candidate function template not viable: requires 3 arguments, but 2 were provided
min(const _Tp& __a, const _Tp& __b, _Compare __comp)
^
5 warnings and 1 error generated.
make: *** [matrix_io.o] Error 1
ERROR: compilation failed for package ‘BPCells’
* removing ‘/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/BPCells’
Warning message:
In i.p(...) :
  installation of package ‘/var/folders/92/rwb8659d3jl658jf5nmmvw580000gn/T//RtmpTTpNt9/file82b725d082db/BPCells_0.1.0.tar.gz’ had non-zero exit status

sessionInfo() output

> sessionInfo()
R version 4.3.0 Patched (2023-04-27 r84336)
Platform: x86_64-apple-darwin20 (64-bit)
Running under: macOS Monterey 12.6.5

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.3.0 tools_4.3.0    curl_5.0.0     remotes_2.4.2

as(iterablematrix, 'IterableMatrix')

Hi Ben,

Me again. I am seeing BPCells from a different perspective suddenly. I am wondering if it makes sense to wrap our dgCMatrix instances in IterableMatrix wrappers and use the wrapped matrices in place of the dgCMatrices. I suppose it depends on having all of the expected methods. I will run some tests.

So I wondered what happens in the case that I try to wrap an IterableMatrix in an IterableMatrix using R commands like

v1 <- c(1,2,3,4,5,6)
v2 <- c(7,8,9,10,11,12)

m1 <- matrix(v1, nrow=2)
m2 <- matrix(v2, nrow=2)

sm1 <- as(m1, 'dgCMatrix')
sm2 <- as(m2, 'dgCMatrix')

bpm1 <- as(sm1, 'IterableMatrix')
bpm2 <- as(bpm1, 'IterableMatrix')

bpm1 <- as(sm1, 'IterableMatrix')
bpm2 <- as(bpm1, 'IterableMatrix')

> str(bpm1)
Formal class 'Iterable_dgCMatrix_wrapper' [package "BPCells"] with 4 slots
  ..@ mat      :Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
  .. .. ..@ i       : int [1:6] 0 1 0 1 0 1
  .. .. ..@ p       : int [1:4] 0 2 4 6
  .. .. ..@ Dim     : int [1:2] 2 3
  .. .. ..@ Dimnames:List of 2
  .. .. .. ..$ : NULL
  .. .. .. ..$ : NULL
  .. .. ..@ x       : num [1:6] 1 2 3 4 5 6
  .. .. ..@ factors : list()
  ..@ dim      : int [1:2] 2 3
  ..@ transpose: logi FALSE
  ..@ dimnames :List of 2
  .. ..$ : NULL
  .. ..$ : NULL
> str(bpm2)
Formal class 'IterableMatrix' [package "BPCells"] with 3 slots
  ..@ dim      : int [1:2] 2 3
  ..@ transpose: logi FALSE
  ..@ dimnames :List of 2
  .. ..$ : NULL
  .. ..$ : NULL

It appears that the bpm2 lost some important information whereas I expected it to be identical to bpm1.

The Matrix package behave more in line with my expectation, for example,

v <- c(1,2,3,4,5,6)
m <- matrix(v1, nrow=2)
sm1 <- as(m, 'dgCMatrix')
sm2 <- as(sm1, 'dgCMatrix')

str(sm1)
Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
  ..@ i       : int [1:6] 0 1 0 1 0 1
  ..@ p       : int [1:4] 0 2 4 6
  ..@ Dim     : int [1:2] 2 3
  ..@ Dimnames:List of 2
  .. ..$ : NULL
  .. ..$ : NULL
  ..@ x       : num [1:6] 1 2 3 4 5 6
  ..@ factors : list()

str(sm2)
Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
  ..@ i       : int [1:6] 0 1 0 1 0 1
  ..@ p       : int [1:4] 0 2 4 6
  ..@ Dim     : int [1:2] 2 3
  ..@ Dimnames:List of 2
  .. ..$ : NULL
  .. ..$ : NULL
  ..@ x       : num [1:6] 1 2 3 4 5 6
  ..@ factors : list()

Of course, I can test whether or not the input matrix is already an 'IterableMatrix' too.

Ever grateful,
Brent

Dense h5ad matrix to BPCells

Hi,
I find that open_matrix_anndata_hdf5 cannot open h5ad files from Genome-wide Perturb-seq datasets:
https://plus.figshare.com/articles/dataset/_Mapping_information-rich_genotype-phenotype_landscapes_with_genome-scale_Perturb-seq_Replogle_et_al_2022_processed_Perturb-seq_datasets/20029387

Installation problem

Hi! I have recently been updating Seurat V5. I am also very excited about the BPcells ability to handle single-cell data with millions of cells. However, after several trials, I just could not install BPcells on windows 11.
I followed your instruction about pre-installing HDF5 first by running "conda install -c anaconda hdf5", indeed I could see the hdf5 after running "conda list". However, after running "remotes::install_github("bnprks/BPCells", quiet = F)" at the Rstudio, I got the errors shown below:

installing source package 'BPCells' ...
** using staged installation
Testing hdf5 by compiling example program...
tools/h5write.c:19:10: fatal error: hdf5.h: No such file or directory
#include "hdf5.h"
^~~~~~~~
compilation terminated.
Unable to locate libhdf5. Please install manually or edit compiler flags.
ERROR: configuration failed for package 'BPCells'

It seems that R could not find this hdf5.h file despite that I have installed it. I've checked the issues#9 but seems that upgrading Rtools simply do not work for me.
Also, I've also tried to install hdf5 with cMake, according to hdfgroup instructions(https://portal.hdfgroup.org/display/support/Building+HDF5+with+CMake). the R still could not locate libhdf5 either.
I am well aware that this problem is not likely to be caused by BPcells, but I am really looking forward to your kind help.

Comparison with Scarf

Hi Benjamin,

I wanted to applaud your exceptional work on BPCells! It's truly remarkable and will make a significant impact in the world of single-cell analysis.

I hope the manuscript writing is progressing smoothly for you.

I also wanted to bring your attention to Scarf, a Python package for single-cell data analysis. It shares many of the objectives as BPCells, and I believe it would be incredibly interesting to conduct a head-to-head comparison between the two. Scarf was published in Nature Communication (Aug 2022) and went through multiple benchmarks against Scanpy and Seurat, using various parameters.

Best regards,
Parashar

bnprks / bpcells Goto Github PK

bpcells's People

Contributors

Stargazers

Watchers

Forkers

bpcells's Issues

When I am trying to install BPCell, It throws following error

Then I changed hfwrite.c file by including the complete path of the HDF5 library like below #include "C:/Program Files/HDF_Group/HDF5/1.14.1/include/hdf5.h" Then I am getting the following issue

It looks like BPCell is not able to locate "C:\Program Files\HDF_Group\HDF5\1.14.1\lib" path . Can you please help me resolve this ? Where can I declare (by using a global variable or sthng) the above path so that its visible to BPCell. ?

Recommend Projects

Recommend Topics

Recommend Org

Then I changed hfwrite.c file by including the complete path of the HDF5 library like below
#include "C:/Program Files/HDF_Group/HDF5/1.14.1/include/hdf5.h"
Then I am getting the following issue