paubellot / netbenchmark Goto Github PK

I confirm that following commit e07b557 fixes the problem when true network evaluated against true network produced wrong AUROC value.

However, there is one more issue, which is a subject for discussion. If you take true network and replace all the zeros with the same non-negative value you would still get nearly perfect precision, while based on my understanding you should get very poor precision:

syntren300.net.nozeros <- syntren300.net
syntren300.net.nozeros[syntren300.net.nozeros==0] <- 0.9

e <- evaluate(syntren300.net.nozeros,syntren300.net,extend=0,sym=FALSE)
aupr(e)
auroc(e)
e <- evaluate(syntren300.net.nozeros,syntren300.net,extend=no.edges,sym=FALSE)
aupr(e)
auroc(e)

Moreover, randomly replacing a row with zeros does not change aupr value at all:

syntren300.net.nozeros <- syntren300.net
syntren300.net.nozeros[syntren300.net.nozeros==0] <- 0.9
syntren300.net.nozeros[4, ] <- 0

e <- evaluate(syntren300.net.nozeros,syntren300.net,extend=0,sym=FALSE)
aupr(e)
auroc(e)
e <- evaluate(syntren300.net.nozeros,syntren300.net,extend=no.edges,sym=FALSE)
aupr(e)
auroc(e)

I believe this could be an issue with minet package. The other package, which I tried with the very same inferred and true networks and which doesn't seem to use minet is networkBMA. It outputs very poor aupr (although I had other problems with that package).