Improper handling of NA/NaN values by knn about nabor

[jefferis]

Hi Cornel,

Thanks for the bug report. As you say the present situation is not ideal, but I am not sure of the right approach to take. The explanation for the current behaviour is that it is determined entirely by the underlying libnabo. The authors of that package have decided not to use a dedicated value to signal an invalid index in the results (see this issue.), instead returning 0, which the knn function then returns as 1 (because we use R-style 1-indexing rather than C style 0-indexing).

Using your example code above, if we look at the result we can still see that there is information in the result to indicate that something is not right:

> knn(targets, query, k=1)
$nn.idx
     [,1]
[1,]    1
[2,]    1
[3,]    3

$nn.dists
     [,1]
[1,]  Inf
[2,]  Inf
[3,]    0

So the value of nn.dists is Inf when there is no valid match. It would be possible to convert nn.idx to NA for non-finite nn.dists.

As a minimum the present behaviour for inputs containing NA values should be documented. Best,

Greg.

[cmpop1]

Hi Greg,

Thanks for replying so quickly and for clarifying this issue. Given your explanation, I think it's probably not a good idea to modify the underlying behaviour, since that'd be the responsibility of the libnabo developers. That said, it may be useful to have an extra (optional) argument in knn to request conversion of indices to NA? On the other hand, such argument may be deprecated in the future potentially causing some minor headaches (I still consider this issue to be a bug in, not a feature of, libnabo).

Documenting NA handling would probably be sufficient, since it's easy enough to clean the output, or the input before passing it to knn.

Cheers,

Cornel