Typo in Algorithm 21.9 about decisionmaking

[mykelk]

In case this link is not accessible to @tawheeler

I am failing to understand the weighting steps in Algorithm 21.9. The relevant steps are

# say w = [1,2,3]
w /= sum(w)
w = [1 - wi for wi in w]
# w would be [0.83, 0.67, 0.5] here

I'm not sure if I understood this correctly, but I believe this would result in a weight vector that sums to $|S| - 1$ instead of $1$, which doesn't make sense. May I ask if there might be a typo here?

[tawheeler]

I think that is indeed a mistake. It should be:

dists = [<L1 distances>]
w = normalize([1/(d+eps) for d in dists], 1)

We can confirm with Kyle if we want to, but this is how kernel smoothing does it (section 8.3).

I'll commit a fix. Thank you!

[tawheeler]

@Tom-CCC, we typically add contributors to our acknowledgements section. If you would like to be added, could you please share your first and last name?

[mykelk]

I'll double check with Kyle real quick.

[Tom-CCS]

Sure, My name is Shengtong Zhang.

[mykelk]

Great! We're still digging into this. The reference implementation is here:
https://github.com/trey0/zmdp/blob/master/src/pomdpBounds/SawtoothUpperBound.cc#L46

[mykelk]

We have updated the implementation in Alg. 21.9 and reposted the PDF and acknowledged you. We also streamlined the text in Sec. 21.6. Thank you for bringing this to our attention!