Comments (3)
Yes, you are right
from usearch.
For now, its not supported, but its two minor releases away. It won’t be done through add
and will instead use the upcoming merge
feature #84
from usearch.
Let me see if I understand who it would work:
Let's say I have a machine with 128 GB RAM and 300 GB of index data - so more data than would fit into RAM.
It sounds as though I could then assemble such an index like this:
-
Create sub-indices:
- 100 GB -> index1 (create in RAM, then write to disk)
- 100 GB -> index2 (create in RAM, then write to disk)
- 100 GB -> index3 (create in RAM, then write to disk)
-
Use
merge
to merge those indices on disk into one large index on disk, without loading any of them into RAM. -
Use
view
to search that large index, without loading it into RAM.
Is that how it would work?
from usearch.
Related Issues (20)
- Bug: Slow index.add HOT 18
- Bug: Go library build error HOT 4
- Bug: JS keys should be `bigint` HOT 4
- Bug: JavaScript Browser Documentation HOT 3
- Bug: Seg Fault on index.add in Rust HOT 4
- Bug: Python runtime crashes after calling `Index.restore()` HOT 1
- Bug: Python `Index.restore` returns `None` when called with `pathlib.Path` object HOT 1
- Feature: support for retrieving metadata from index file in C interface HOT 2
- Bug: Compilation issue with a defined __AVX512__ without _Float16 support HOT 3
- Feature: possibility of a streaming API? HOT 3
- Bug: RISC-V compilation failure
- Feature: Progress tracking and conditional termination from Python
- Invalid: C must work with synthetic data
- Feature: WAI(WebAssembly Interfaces) for USearch WASM binary
- Bug: `usearch_get` returns wrong value in C99 interface HOT 2
- Documenting Memory Usage HOT 3
- Bug: unable to install with wasmer HOT 1
- Feature: Brute force (KNN) search without index HOT 30
- Bug: cannot catch Runtime exception when index fails to load HOT 4
- Bug: Distance is negative HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from usearch.