Comments (3)
Same issue here. I tried testing a small dataset for validating the code, and a segfault was still encountered.
[LightGBM] [Warning] Unknown parameter per_leaf, min_data
[LightGBM] [Info] Finished loading parameters
num_threads: 1
[LightGBM] [Info] Warning: last line of ./train.data.query has no end of line, still using this line
[LightGBM] [Info] Loading query boundaries...
[LightGBM] [Info] Finished loading data in 0.081986 seconds
position bias_i bias_j i_cost j_cost
0 1 1 0 0
1 1 1 0 0
2 1 1 0 0
3 1 1 0 0
4 1 1 0 0
5 1 1 0 0
6 1 1 0 0
7 1 1 0 0
8 1 1 0 0
9 1 1 0 0
10 1 1 0 0
11 1 1 0 0
[LightGBM] [Info] Total Bins 267
[LightGBM] [Info] Number of data: 4, number of used features: 99
[LightGBM] [Info] Finished initializing training
[LightGBM] [Info] Started training...
Segmentation fault
from unbiased_lambdamart.
I test lightgbm in examples/lambdarank, still seg fault
ldd (Ubuntu GLIBC 2.27-3ubuntu1) 2.27
from unbiased_lambdamart.
Our modified version requires an additional xxx.train.rank file, which indicates the position for each document in a session. Please check the Unbias_LightGBM/examples/lambdarank/rank.train.rank file as an example. If there is no such a file provided, then it will seg fault (I haven't added an exception to deal with it, will add later).
Here is my training log:
[LightGBM] [Info] Finished loading parameters
num_threads_: 48
[LightGBM] [Info] Loading query boundaries...
[LightGBM] [Info] Loading ranks...
[LightGBM] [Info] Loading query boundaries...
[LightGBM] [Info] Finished loading data in 3.523921 seconds
position bias_i bias_j i_cost j_cost
0 1 1 0 0
1 1 1 0 0
2 1 1 0 0
3 1 1 0 0
4 1 1 0 0
5 1 1 0 0
6 1 1 0 0
7 1 1 0 0
8 1 1 0 0
9 1 1 0 0
10 1 1 0 0
11 1 1 0 0
[LightGBM] [Info] Total Bins 6177
[LightGBM] [Info] Number of data: 3005, number of used features: 211
[LightGBM] [Info] Finished initializing training
[LightGBM] [Info] Started training...
eta: 0.666667, pair_cnt_sum: 13543
position bias_i bias_j score lambda high_pair_cnt i_cost j_cost
0 1 1 0 -0.144466 0.0522041 0.00231194 0.00379812
1 1 1 0 0.0290876 0.0607694 0.00212011 0.00182088
2 1 1 0 0.0343925 0.066455 0.001629 0.00127519
3 1 1 0 -0.0087586 0.0580374 0.00107807 0.00116817
4 1 1 0 -0.0157791 0.055453 0.000931196 0.00109352
5 1 1 0 0.00122206 0.057742 0.00101186 0.000999293
6 1 1 0 0.000664673 0.0613601 0.000945256 0.000938418
7 1 1 0 0.0123509 0.0592926 0.00106164 0.000934579
8 1 1 0 0.026347 0.0604002 0.0014762 0.00120515
9 1 1 0 0.00616404 0.0642398 0.00121907 0.00115566
10 1 1 0 0.00750636 0.0605479 0.00112833 0.00105111
11 1 1 0 0.0512683 0.343498 0.00668941 0.00616199
[LightGBM] [Info] Iteration:1, training ndcg@1 : 0.689742
[LightGBM] [Info] Iteration:1, training ndcg@3 : 0.702272
[LightGBM] [Info] Iteration:1, training ndcg@5 : 0.741414
[LightGBM] [Info] Iteration:1, valid_1 ndcg@1 : 0.456952
[LightGBM] [Info] Iteration:1, valid_1 ndcg@3 : 0.551904
[LightGBM] [Info] Iteration:1, valid_1 ndcg@5 : 0.604216
[LightGBM] [Info] 4.448035 seconds elapsed, finished iteration 1
eta: 0.666667, pair_cnt_sum: 13543
position bias_i bias_j score lambda high_pair_cnt i_cost j_cost
0 1 1 -0.00667379 -0.800358 0.0522041 0.0695507 0.0777334
1 0.943892 0.61255 -0.00557382 5.38474 0.0607694 0.139929 0.0569327
2 0.791826 0.483063 -0.0044684 6.70061 0.066455 0.153422 0.0605704
3 0.601332 0.455645 -0.0039316 0.968484 0.0580374 0.0913264 0.0645737
4 0.545394 0.43602 -0.00305987 -3.37869 0.055453 0.0685098 0.0700101
5 0.576454 0.410598 -0.0017311 -1.14106 0.057742 0.0837521 0.0645134
6 0.55087 0.393748 -0.00403243 -3.55345 0.0613601 0.0625244 0.0587793
7 0.595206 0.392674 -0.00263518 -0.0696561 0.0592926 0.0905074 0.0599418
8 0.741502 0.465211 0.000324526 -0.376369 0.0604002 0.0991584 0.064199
9 0.652683 0.452384 0.000514503 -0.510154 0.0642398 0.0777408 0.0563165
10 0.619879 0.424672 0.0016421 -2.99069 0.0605479 0.0638013 0.0567422
11 2.03052 1.38071 -0.0150764 -0.233419 0.343498 0.376195 0.258771
[LightGBM] [Info] Iteration:2, training ndcg@1 : 0.782279
[LightGBM] [Info] Iteration:2, training ndcg@3 : 0.794868
[LightGBM] [Info] Iteration:2, training ndcg@5 : 0.807632
[LightGBM] [Info] Iteration:2, valid_1 ndcg@1 : 0.560762
[LightGBM] [Info] Iteration:2, valid_1 ndcg@3 : 0.576421
[LightGBM] [Info] Iteration:2, valid_1 ndcg@5 : 0.608285
[LightGBM] [Info] 1.654006 seconds elapsed, finished iteration 2
......
from unbiased_lambdamart.
Related Issues (12)
- Jupyter notebook example HOT 4
- AppleClang not supported -- Setup on Mac. HOT 3
- Want to use PythonAPI to train and predict. HOT 2
- Why is sigma = 2? HOT 2
- Question about `position_bins`
- Question about reading "XXX.train" file
- Unbiased_lambdamart ndcg is low than original lambdamart HOT 3
- How to tune hyperparameter when use the lambdamart example HOT 2
- Add LICENSE.md to project root HOT 1
- Reccommend using a submodule+fork for Unbias_LightGBM HOT 7
- Broken Link in README.md HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from unbiased_lambdamart.