Comments (28)
Is the dtype incorrect (see the error message before the segfault)? I guess np.random.randn returns a double-precision array by default.
Edit: I should have read the pastebin more carefully :-) What happens when you store the query into a variable and then use it as a parameter?
from falconn.
It's more interesting than that. It seems to fail in REPL and within py.test but seems to work when run as a simple script.
For instance: http://pastebin.com/PVmsLrhz fails when run via py.test and works when run via python.
from falconn.
I guess in some cases the temporary float32 array gets garbage-collected?
from falconn.
Does not seem to be the case. But segfault disappears if I try to print anything withing the C++ code...
from falconn.
That sounds like there could be some memory corruption going on. Does valgrind work under cygwin?
from falconn.
No, but I'll run it under Linux tomorrow.
from falconn.
The bug is now somewhat better localized:
- If we just do
get_unique_sorted_candidates
, then everything works - If we do
find_nearest_neighbor
, then everything works as well - But, if we do
find_k_nearest_neighbors
, even withk = 1
, then it fails
from falconn.
The bug disappears if one removes -march=native
!
from falconn.
More specifically, -mavx
seems to trigger it. Depending on whether we have -O0
or not, it either fails in the preprocessing, or in the query.
from falconn.
FFHT does not seem to responsible, even if we run a version without AVX, the code still fails.
from falconn.
Ah, with -O0
it fails in gdb
! Victory!!!!
Error: http://pastebin.com/AbeNXeT0
Backtrace: http://pastebin.com/PbC8q0R0
from falconn.
This is Eigen vs. Cygwin, in the end. The following code works without -mavx
and fails with -mavx
(in both cases compiled with -O0
):
#include <Eigen/Dense>
#include <iostream>
typedef Eigen::VectorXf vec;
int main() {
vec a(8);
vec b(8);
a = a.cwiseProduct(b);
std::cout << a[0] << std::endl;
return 0;
}
from falconn.
Filed a bug report: http://eigen.tuxfamily.org/bz/show_bug.cgi?id=1146
from falconn.
Looks like it's a Cygwin's bug. The following code fails:
#include "immintrin.h"
__m256 ff(float *buf) {
return _mm256_load_ps(buf);
}
int main() {
float *buf = new float[8];
__m256 tt = ff(buf);
return 0;
}
from falconn.
Is the code supposed to work? The array might not be 32-byte aligned.
Also, does the issue go away with the stable Eigen 3.2?
Thanks for looking into this!
On Thursday, January 14, 2016, Ilya Razenshteyn [email protected]
wrote:
Looks like it's a Cygwin's bug. The following code fails:
'''c++
#include
#include "immintrin.h"using namespace std;
__m256 ff(const float *buf) {
return _mm256_load_ps(buf);
}int main() {
float *buf = new float[8];
cout << "before" << endl;
__m256 tt = ff(buf);
cout << "after" << endl;
delete[] buf;
return 0;
}
'''—
Reply to this email directly or view it on GitHub
#30 (comment).
from falconn.
What will you say about this then?
#include "immintrin.h"
__m256 routine(void) {
__m256 aux;
return aux;
}
int main(void) {
void *buf = malloc(1);
__m256 res = routine();
return 0;
}
from falconn.
I don't know about the semantics of returning intrinsic locations. I guess the 1-byte *buf is crucial to make things fail?
from falconn.
Yes it is. Moreover, it's crucial that the pointer buf
is allocated on the stack (rather than as a global variable).
Not sure what's happening there.
from falconn.
I guess buf
somehow messes up the alignment of res
. Maybe the stack frame is 32-byte aligned but buf
is put there before res
, which then leads to the alignment error.
Just to avoid confusion: this works on Linux, correct? I just tested it on my Mac and it doesn't seem to give segmentation faults.
from falconn.
It works on Linux (on towhee, at least). It could have to do with https://cygwin.com/ml/cygwin-apps/2015-11/msg00073.html .
from falconn.
This sounds like AVX is partially broken on Cygwin?
from falconn.
Could be. Let's confirm this, and, if this is the case, declare that we don't support FALCONN for Cygwin. We should also give MinGW (http://www.mingw.org/) a try.
from falconn.
If AVX on Cygwin is indeed not working, I'm also OK with dropping support for Cygwin.
For MinGW, the 64-bit fork might also be worth a look: http://mingw-w64.org/doku.php
I guess the most native compiler for Windows is the Visual Studio compiler. But I don't know how easy it would be for us to support it.
from falconn.
Is MinGW (not the 64-bit fork) actively being developed? http://www.mingw.org/wiki/GCCStatus says "Latest official release of GCC from the MinGW project is GCC 3.4.5."
from falconn.
Looks like this is what happens: https://stackoverflow.com/questions/5983389/how-to-align-stack-at-32-byte-boundary-in-gcc
I guess we have no choice other than to drop the Windows support...
from falconn.
Though, I guess, one can disable AVX within Eigen if one detects Windows (we don't want to disable AVX overall, since FFHT should still work, and using AVX there is crucial). This can probably be done by defining a couple of macroses within falconn_global.h
or something like this.
from falconn.
Yes, we could give that a try. Alternatively, we could try supporting VC++ to get AVX support. I'm not sure what people mainly use for C++ development on Windows these days.
from falconn.
Resolved in 6283f9c
from falconn.
Related Issues (20)
- How to return more points? HOT 2
- windows install falconn failure HOT 1
- Comment. Application to associative memory
- How to save the hash table? HOT 1
- Calculating similarity
- website certificate expired HOT 1
- The similarity Values
- Not returning specified number of neighbors
- Better explanation regarding windows
- something error in params.feature_hashing_dimension parameters HOT 1
- Unable to install python through repository
- The header-only library seems broken HOT 1
- Distance returned is more than 1 and it's also negated when using EuclideanSquared on normalized vectors
- Package is not be able to install inside python docker image
- Detailed steps for running code
- Why can get two vectors' similarity after random projection? HOT 1
- Compile error: error: unknown register name ‘%ymm14’ in ‘asm’
- How to get similar candidate?
- FALCONN KNN alternative to save as pickle? HOT 1
- The website does not load
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from falconn.