Comments (2)
Hi @HCMY. Just to get on the same page: this package provides an implementation of the umap algorithm in R/Rcpp and an implementation that launches the python-based 'umap-learn' (the original umap).
The R/Rcpp implementation in this repo relies on a dataset to be loaded as a matrix in memory. If you can coerce your data from whatever source into a matrix, then all is OK. But if you are asking about processing data as a stream or data that is larger than memory, then that is not supported.
My impression is that some users of umap-learn package have mentioned spark, but I have not used that myself. You can ask there for help (?). Also, keep in mind that their advanced capabilities might not be compatible with the R-python interfacing here, so they might not work through this package. If you have success with this, please share! Cheers.
from umap.
Hi @HCMY. Just to get on the same page: this package provides an implementation of the umap algorithm in R/Rcpp and an implementation that launches the python-based 'umap-learn' (the original umap).
The R/Rcpp implementation in this repo relies on a dataset to be loaded as a matrix in memory. If you can coerce your data from whatever source into a matrix, then all is OK. But if you are asking about processing data as a stream or data that is larger than memory, then that is not supported.
My impression is that some users of umap-learn package have mentioned spark, but I have not used that myself. You can ask there for help (?). Also, keep in mind that their advanced capabilities might not be compatible with the R-python interfacing here, so they might not work through this package. If you have success with this, please share! Cheers.
thnaks for your reply, im working on it.
from umap.
Related Issues (20)
- predict() on a umap object with n_components=1 gets two errors -- Looks like missing drop=F HOT 2
- Failed creating initial embedding; using random embedding instead HOT 3
- Intel MKL FATAL ERROR HOT 3
- Differences with Python version? HOT 4
- Add support for umap-learn 0.5 HOT 4
- Sparse Matrix support HOT 4
- missing value where TRUE/FALSE needed HOT 3
- Problem with using custom metric HOT 2
- umap() produces matrix instead of S3 object HOT 2
- method = "python" does not work HOT 1
- when random_state is set automatically in config, it is not sufficient for reproducibility HOT 1
- Citing the package HOT 1
- Type error in optimize_embedding HOT 3
- transforming new data to an embedding HOT 3
- Error with n_components=1 HOT 3
- Number of threads HOT 3
- Allow for supervised/semi-supervised dimension reduction with labels HOT 1
- min_dist not updating with Python backend HOT 3
- predict() generates different predictions if called with multiple points at once versus called with each point individually HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from umap.