Comments (15)
Not sure, likely the ABI compatibility issue in the runtime. Do you have any stack trace?
from faiss-wheels.
#0 0x00007fffe814e28d in ?? () from /media/donnees/non_sauvegarde/image_embeddings_env/lib/python3.7/site-packages/faiss/_swigfaiss.cpython-37m-x86_64-linux-gnu.so
#1 0x00007fffe8174dc0 in ?? () from /media/donnees/non_sauvegarde/image_embeddings_env/lib/python3.7/site-packages/faiss/_swigfaiss.cpython-37m-x86_64-linux-gnu.so
#2 0x00000000005c8663 in _PyMethodDef_RawFastCallKeywords ()
#3 0x0000000000535990 in ?? ()
#4 0x000000000053c5a1 in _PyEval_EvalFrameDefault ()
#5 0x00000000005c916b in _PyFunction_FastCallKeywords ()
#6 0x0000000000535880 in ?? ()
#7 0x0000000000538713 in _PyEval_EvalFrameDefault ()
#8 0x00000000005365e7 in _PyEval_EvalCodeWithName ()
#9 0x00000000005c9468 in _PyFunction_FastCallKeywords ()
#10 0x0000000000535880 in ?? ()
#11 0x0000000000538713 in _PyEval_EvalFrameDefault ()
#12 0x00000000005365e7 in _PyEval_EvalCodeWithName ()
#13 0x000000000064cbb3 in PyEval_EvalCode ()
#14 0x00000000006402a3 in ?? ()
#15 0x0000000000640357 in PyRun_FileExFlags ()
#16 0x000000000064110a in PyRun_SimpleFileExFlags ()
Not sure if that really helps
from faiss-wheels.
Ah interesting...
It seems I'm using the same issue as #1
In my environment I had tensorflow 2.2.0
If I make an environment with only faiss-cpu, it works
from faiss-wheels.
Do you think a similar fix as #1 (comment) could be possible ?
from faiss-wheels.
Ok no I identified the actual issue.
In order to install tensorflow 2, it's needed to update pip.
So I updated to pip pip-20.1.1
.
Run pip install -U pip
in a venv for this
These commands reproduce my issue :
python3 -m venv env
source env/bin/activate
pip install -U pip
pip install faiss-cpu
python test.py
With test.py content being the thing in the first post
I'm using python 3.7.5 if that can make a difference
Can you reproduce ? any idea why that would happen ?
from faiss-wheels.
There could be many possibilities, but I could not reproduce this on docker ubuntu 20.04 image. It could be bad interaction with multiple package managers (pip and conda), it could be lack of cpu feature (avx2), etc.
from faiss-wheels.
Also, if you import tensorflow as tf
, always do that before importing other pacakges.
from faiss-wheels.
rom1504@rom1504-W35-37ET:~$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 58
model name : Intel(R) Core(TM) i5-3230M CPU @ 2.60GHz
stepping : 9
microcode : 0x21
cpu MHz : 1197.274
cache size : 3072 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear flush_l1d
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit srbds
bogomips : 5188.15
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
Indeed looks like my CPU does not support avx2
What I'm quite confused about is if I do not update pip (so keeping pip 18.1 with python 3.7) then everything works.
If I do update pip to 20.1.1, I'm getting that segfault, even without installing tensorflow.
I don't use conda
from faiss-wheels.
Ah I think I found something.
If I use pip 1.18.1 (working) it's using faiss_cpu-1.6.3-cp37-cp37m-manylinux1_x86_64.whl
whereas if I use pip 20.1.1 (not working) it's using faiss_cpu-1.6.3-cp37-cp37m-manylinux2010_x86_64.whl
Are there some differences in this second file that might explain my result ?
Maybe this second file requires avx2 ?
If that's the case, would it be possible to provide a non avx2 build or do you advise I build it myself ?
(and do you know a way to identify if the issue is missing avx2 ?)
from faiss-wheels.
I tried using python 3.8 instead of 3.7 as per #8 (comment) but this makes no difference in my case
from faiss-wheels.
Tried to build myself but getting some other errors when trying to follow https://github.com/kyamagu/faiss-wheels#linux , I opened a separate issue for this (#17)
I think it may make sense for me to just not support hardware without AVX2 for the lib I'm building, and advising to use google colab in that case.
from faiss-wheels.
manylinux1 and manylinux2010 have different glibc version dependency, and they are usually forward compatible: manylinux1 works on centos 5 environment and above, manylinux2010 works on centos 6 and above. ubuntu 20.04 should be fine with both.
Hmm, it makes sense to support option to disable / enable avx2.
from faiss-wheels.
If you build from source, you can also try the official approach:
cd faiss
./configure
make
make -C python
from faiss-wheels.
So as discussed in the github action PR, the new version will not use avx2 instructions. That's great. I understand it's not possible to change an existing release and you follow faiss versioning which makes sense.
In the mean time, I forked your repo (https://github.com/rom1504/faiss-wheels), and thanks to the use of github action, I was able by just changing NAME in setup.py to release this package https://pypi.org/project/faiss-cpu-noavx2 which completely solves my problem
I will use it until faiss release a new version, at this point I'll be able to go back to faiss-cpu
Again thanks for the support and for building this simple and useful repo, I think at some point you could consider simply doing a PR to faiss main repo, they might just accept it and maintain it in the future.
from faiss-wheels.
@rom1504 Great.
As a side note, faiss maintainers seem not to be interested in supporting PyPI: facebookresearch/faiss#170 (comment)
This is why this unofficial repo exists.
from faiss-wheels.
Related Issues (20)
- pip installs the previous release when tries to build from source HOT 1
- AttributeError: 'MSVCCompiler' object has no attribute 'compiler' when trying to install faiss-cpu on Windows 11 and Python 3.8.10 HOT 1
- Failure to serialize in joblib HOT 17
- Can't install faiss-cpu-noavx2 on colab HOT 3
- Segfault in macOS in the presence of other OpenMP dependent packages HOT 2
- Compatibility issue with faiss-cpu wheels on arm64 vs x86 HOT 9
- faiss-cpu 1.7.4 not working with mkl 2023.1.0 HOT 1
- Running setup.py install for faiss-gpu did not run successfully HOT 2
- inconsistent name: expected 'faiss-gpu', but metadata has 'faiss-cpu' HOT 7
- Faiss-cpu requires numpy, but numpy not installed as a dependency of faiss-cpu
- ERROR: Could not build wheels for faiss-cpu, which is required to install pyproject.toml-based projects in FreeBSD HOT 6
- Support Selector for IDMap HOT 3
- ERROR: Could not build wheels for faiss-cpu, which is required to install pyproject.toml-based projects in linux Alpine HOT 6
- Release v1.7.4 on PyPI HOT 1
- Get a CUDA error when searching in a Faiss index having too small vectors HOT 1
- Release date for faiss-cpu that includes numpy dependency fix? HOT 2
- CPython 3.12 wheels HOT 7
- Cannot install faiss-gpu on Arch Linux HOT 2
- Faiss fails to install on MacOS M2 Macbook HOT 2
- AVX512 support HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from faiss-wheels.