Comments (4)
I see. I think the pool is only used in very specific cases (large batch processing tasks).
It seems like the easiest way to fix it is to avoid creating the pool by default. It only affects line 112.
from colbert.
from colbert.
Thank your for this note! I will have it behind a flag.
from colbert.
I would still recommend self.emb2pid.share_memory_()
- its a no-op if its in CUDA.
from colbert.
Related Issues (20)
- Using ColBERTv2 for re-ranking HOT 7
- query without passage in collection
- What is faiss actually used for? HOT 1
- OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.
- IndexUpdater.persist_to_disk incorrectly overwrites metadata["num_embeddings"]
- Indexer throwing error : module 'torch.distributed' has no attribute 'is_initialized' HOT 4
- Finetuning nway parameter HOT 2
- [IndexUpdater.persist_to_disk] Incorrect values of variables for append
- UnicodeDecodeError when loading queries/collection.
- Indexing by using Faiss in ColBERTv1 HOT 1
- Colbert for handling multilingual passages and queries
- Process stuck on Launcher while training using example code
- incremental training on colbert
- [incremental indexing] - IndexUpdater.update_searcher is successful but search result does not show newly indexed passage HOT 1
- ColBERTv1: Faiss_depth usage in retrieval
- [c++] Calling pop() on an empty stack causes "undefined behavior"
- Searcher.search intermittent bug after IndexUpdater.persist_to_disk HOT 2
- Trainer.train() should not take checkpoint as an argument and should use self.config.checkpoint HOT 1
- Clarification on PLAID retrieval HOT 6
- Multiprocessing takes forever after on .get() with mp.Queue() (Possible Deadlock) HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from colbert.