Issue on running --phase train about zingg

[sonalgoyal]

How many positive and negative pairs have you labeled? The logs say "All labels are the same value and fitIntercept=true, so the coefficients will be zeros. Training is not needed." which means that there are possibly no positive pairs?

[premsmac2021]

+-------+---------+------+-----------+---------------------+--------------+--------+--------------+--------+-------------+---------+------------+----------+----------+-------------+-----------+---+------+--------+--------+----+----+-----------+------------+-----------+--------+
|fname |TALUKNAME|GPNAME|VILLAGENAME|SCHEMENAME |SERIESYEARNAME|BENFCODE|SPOUSENAME |BENFNAME|HUSBAND |CASTENAME|RELIGIONNAME|GENDERNAME|OCCUPATION|TOTALFAMILYNO|ADDRESS |AGE|AADHAR|VERIFIED|PHONENUM|VC |VI |VRATIONCARD|NEWACCOUNTNO|GOLDENRECNO|z_source|
+-------+---------+------+-----------+---------------------+--------------+--------+--------------+--------+-------------+---------+------------+----------+----------+-------------+-----------+---+------+--------+--------+----+----+-----------+------------+-----------+--------+
|RAJAMMA|Mysuru |Varuna|Chikkahalli|Basava Housing Scheme|2015 2016 |265342 |CHIKKAMUTTAIAH|rajam ma|cikkamuttayya|SC |Hindu |Female |Labour |3 |CHIKKAHALLI|36 |null |No |null |null|null|null |85034963709 |5910841 |test |
|RAJAMMA|Mysuru |Varuna|Chikkahalli|Basava Housing Scheme|2015 2016 |265342 |CHIKKAMUTTAIAH|rajam ma|cikkamuttayya|SC |Hindu |Female |Labour |3 |CHIKKAHALLI|36 |null |No |null |null|null|null |85034963709 |5910841 |test |
+-------+---------+------+-----------+---------------------+--------------+--------+--------------+--------+-------------+---------+------------+----------+----------+-------------+-----------+---+------+--------+--------+----+----+-----------+------------+-----------+--------+

Please select from the following choices
No, they do not match : 0
Yes, they match       : 1
Not sure              : 2

To exit               : 9

Please enter your choice [0,1,2 or 9]: 1

Record pair 11 out of 22 records to be labelled by the user.
Zingg predicts the records ARE NOT KNOWN IF MATCH with a similarity score of 0.00

[sonalgoyal]

sorry I did not understand your last comment

[premsmac2021]

Above is the positive pair. but issue message says that there are possibly no positive pair.
and not producing the trained records in tmp folder.

[sonalgoyal]

How many positive pairs do you have? You need at least 20-30 pairs for the classifier to get trained on

[premsmac2021]

Record pair 21 out of 22 records to be labelled by the user.
Zingg predicts the records ARE NOT KNOWN IF MATCH with a similarity score of 0.00

What is the score ? If record
No, they do not match
or
Yes, they match means?

[sonalgoyal]

sorry, I do not understand the question above. In the initial round, Zingg does not give any score to the pairs. Once you label a few pairs, Zingg learns similarities and starts refining the matching predictions and scores. Does that help?

[premsmac2021]

Can you please refer this.

Record not matching
Please select from the following choices
No, they do not match : 0
Yes, they match : 1
Not sure : 2

To exit               : 9

Please enter your choice [0,1,2 or 9]: 0

Record pair 17 out of 20 records to be labelled by the user.
Zingg predicts the records MATCH with a similarity **score of 1.00**

---

Record Matching:-
Please select from the following choices
No, they do not match : 0
Yes, they match : 1
Not sure : 2

To exit               : 9

Please enter your choice [0,1,2 or 9]: 1

Record pair 18 out of 20 records to be labelled by the user.
Zingg predicts the records DO NOT MATCH with a similarity **score of 0.27**

---

Please select from the following choices
No, they do not match : 0
Yes, they match       : 1
Not sure              : 2

To exit               : 9

Please enter your choice [0,1,2 or 9]: 1

Record pair 19 out of 20 records to be labelled by the user.
Zingg predicts the records MATCH with a similarity **score of 1.00**

Here I've given 2 scenarios, 1st one record doesn't match 2nd & 3rd one records are matching. But score's are showing different.

Do you understand?

[sonalgoyal]

The labelling phase scores are indicative and get refined as you add more samples. How many pairs have you labelled so far? How many positives?

[premsmac2021]

Out of 21, 15 Positives

Positive means same record?

[sonalgoyal]

Yes, positive means they are same. How many total rounds have you run? Have you labelled 15 pairs as yes out of total 21 or is it in this round?

[premsmac2021]

labelled 15 pairs as yes out of total 21.

[sonalgoyal]

Ok, that’s pretty good to find so many matches in the first round of labelling itself. Do a few more rounds till you have about 30-49 matches and then try trainMatch

[premsmac2021]

Yes, will do that..

[premsmac2021]

what is the length of header name?

I've given this length.
MEMBER_OCCUPATION_ID_DESCRIPTION

[sonalgoyal]

Is there a problem?

[premsmac2021]

didn't accept the "_" underscore

[sonalgoyal]

oh, looks like a bug, opened #50 for that. thanks for reporting. please add any other observations related to that issue there.

[sonalgoyal]

Is the training issue sorted @premsmac2021 - if so, can you please close this issue or let me know?