the return value maybe wrong? in load.py about pytorch-unet

[milesial]

Hi, this only changes the array order, the content is the same I believe. The list is shuffled anyway as part of the training in split_train_val (utils/utils.py). In this case I loop over the i inside the loop over ids, so the result is something like
[(id1, 0), (id1, 1), (id2, 0), (id2, 1), ...].

Edit: I'm wrong, the order of the for loops is left to right

[KingofAsianPopJC]

@milesial I find this problem as well, we hope the return of this function (split_ids) is like the form :
[(id1, 0), (id1, 1), (id2, 0), (id2, 1), ...]. The code "((id, i) for id in ids for i in range(n))" can get this output. And your version "((id, i) for i in range(n) for id in ids)" is supposed to output like this:
[(id1, 0), (id2, 0),... ,(id1, 1), (id2, 1) ...], however it just output [(id1, 0), (id2, 0),...] , without (id1, 1), (id2, 1) ...

[milesial]

That's strange ! When n=2, on python3, all went fine for me. If the order is wrong that does not matter because it's shuffled but if half the values are missing that's a problem...

If I run this code [(id, i) for i in range(2) for id in ['id1', 'id2', 'id3']] I get [('id1', 0), ('id2', 0), ('id3', 0), ('id1', 1), ('id2', 1), ('id3', 1)], which is fine. Is it the same on your setup ?

[KingofAsianPopJC]

@milesial I think I find the reason of this strange result. The code "((id, i) for i in range(n) for id in ids)" can expand as this :
for i in range(n):
for id in ids:
.....
In the first loop of ids(when n = 0), the iter ids has been iterated out, so in the next loop(when n = 1), there's no element in iter ids. And this result the output with only half the value [(id1, 0), (id2, 0),...] .
However, the code "((id, i) for id in ids for i in range(n))" just iterate the ids once, so it works well.
Of course, above is my own opinion, maybe not correct.

[milesial]

Ooh you're right ! Nice catch, indeed ids is a generator so it runs out. This can be fixed by either returning a list in the function get_ids, or copying the ids generator before iterating over it.

[KingofAsianPopJC]

Thanks for your reply! I agree with your solution for this. Maybe the developers of Python can alter the mechanism of the iterator so that it won't iterate in "for...in..." code, only if we run 'next()' or "next"
^_^

[milesial]

Haha it would break many projects, even my train loop relies on the fact that the train iterator finishes.