Remove regex DS rules; they might actually be hurting about dd-genomics

[ajratner]

If you remove these, could you perhaps just comment them out so they're
easy to put back in (e.g. for data-programming testing :) )?

Also, curious to hear more detailed thoughts on this in person

On Thu, Feb 11, 2016 at 9:53 PM Colossus [email protected] wrote:

—
Reply to this email directly or view it on GitHub
#346.

[Colossus]

Yeah I just commented them

[chrismre]

How are we doing in quality =)

On Thu, Feb 11, 2016 at 9:56 PM Colossus [email protected] wrote:

Yeah I just commented them

—
Reply to this email directly or view it on GitHub
#346 (comment)
.

[Colossus]

Basically I think they were good initially, but at this point our SV set is so good that these rules pick up too much, in particular when it comes to picking out wrong pairs.

[ajratner]

Awesome thanks!

Okay this will be cool to look into further, is interesting!

On Thu, Feb 11, 2016 at 9:56 PM Colossus [email protected] wrote:

Yeah I just commented them

—
Reply to this email directly or view it on GitHub
#346 (comment)
.

[Colossus]

90% precision as usual and 30% recall, with the big fight against picking out wrong pairs in progress. The current weapons of choice, all of which are being evaluated, are:

• Hand-crafted factors to lower likelihood of extracting certain wrong pairs based on known patterns (e.g.: GP-GP pattern; introduce negative factor for middle P-G)
• Mixing ddlib and treedlib; with a bunch of novel features in treedlib targeting the word sequence between gene and pheno
• hopefully (suggestion to Alex) creating classes of treedlib features based on the length of the word sequence/dep path between gene and pheno

[Colossus]

We had a bunch of trouble with the expressivity of DDLib factors, but all of these have been fixed by Feiran and Jaeho

[ajratner]

Generated some good new ideas for treedlib today which I'll be putting in
(at least to have option of experimenting with) soon :)

On Thu, Feb 11, 2016 at 9:59 PM Colossus [email protected] wrote:

90% precision as usual and 30% recall, with the big fight against picking
out wrong pairs in progress. The current weapons of choice, all of which
are being evaluated, are:

• Hand-crafted factors to lower likelihood of extracting certain wrong
  pairs based on known patterns (e.g.: GP-GP pattern; introduce negative
  factor for middle P-G)
• Mixing ddlib and treedlib; with a bunch of novel features in
  treedlib targeting the word sequence between gene and pheno
• hopefully (suggestion to Alex) creating classes of treedlib features
  based on the length of the word sequence/dep path between gene and pheno

—
Reply to this email directly or view it on GitHub
#346 (comment)
.

[Colossus]

so I just compared ddlib pure VS ddlib for single-relationship-candidate sentences and treedlib for multi-relationship-candidate sentences and it appears there is hardly any difference in precision and recall unfortunately (runs on different sets produce slightly different results, but none of them seem decisively better) ... Based on the results, however, I'm going with ddlib for single-relationship and treedlib for multi-relationship (treedlib DID seem to do slightly better there)

I'm still hoping for the

• word sequence features, which I'm sure would get us rid of one class of "wrong-pair-errors" (which is the class where there's simply no "good" word in between gene and pheno mentio)
• and hand-crafted factors, which will beat sentences that list a bunch of GP mentions in a row

[ajratner]

For your first bullet point, I'm assuming you mean 'where there's no good
word on the dep path between', as per our convo today?

Either way, will have some stuff soon in treedlib that will hopefully help
here- will keep you updated!

On Thu, Feb 11, 2016 at 10:04 PM Colossus [email protected] wrote:

so I just compared ddlib pure VS ddlib for single-relationship-candidate
sentences and treedlib for multi-relationship-candidate sentences and it
appears there is hardly any difference in precision and recall
unfortunately (runs on different sets produce slightly different results,
but none of them seem decisively better) ... Based on the results, however,
I'm going with ddlib for single-relationship and treedlib for
multi-relationship (treedlib DID seem to do slightly better there)

I'm still hoping for the

• word sequence features, which I'm sure would get us rid of one class
  of "wrong-pair-errors" (which is the class where there's simply no "good"
  word in between gene and pheno mentio)
• and hand-crafted factors, which will beat sentences that list a
  bunch of GP mentions in a row

—
Reply to this email directly or view it on GitHub
#346 (comment)
.

[Colossus]

well not exactly, I mean "where there's no good word in the word sequence in between"

[Colossus]

Psalm 45:13

[ajratner]

Ok & lol

On Thu, Feb 11, 2016 at 10:08 PM Colossus [email protected] wrote:

Psalm 45:13

—
Reply to this email directly or view it on GitHub
#346 (comment)
.