Results meaning for CALEngine about hical

[nims11]

Each line corresponds to a judgment. <document_id><space><judgment>. In this repo, we use 3 grades (0 - not relevant, 1 - relevant, 2 - highly relevant).

[jieralice13]

Is there anywhere that explains the output for CALEngine (especially bmi_cli).
Sample output for bmi:

064578 0
047277 2
050307 2
050457 2
047817 1
047799 2
273177 0
273551 2
047347 0
273611 0
086457 0
080529 2
085197 2
...

Thanks (sorry to bother)

Hi @jeanchilger, may I ask how did you get multiple output for one topic? When I run bmi_cli, I only get one line output.

I used the sample-dataset, by using the athome4.topic.sample as query, and athome4.qrel.sample as qrel, and my command is like below.

THREADS=4
JOBS=10
EXEC=./bmi_cli
RESULT_DIR=results
DOC_FEATURES=../data/athome4_sample.bin
PARA_FEATURES=../data/athome4_para_sample.bin
QREL=../data/athome4.qrels
QUERIES=../data/athome4.query
MAXEFFORT=0
MAXEFFORTFACTOR=3.1
TRAINITERATIONS=200000

./bmi_cli --doc-features "$DOC_FEATURES" --judgment-logpath "$SUBRESULT_DIR" --qrel "$QREL" --query "$QUERIES"
--threads $THREADS --jobs $JOBS --training-iterations "$TRAINITERATIONS"
--mode BMI_DOC --max-effort "$MAXEFFORT" --max-effort-factor "$MAXEFFORTFACTOR" &> "$SUBRESULT_DIR/log"