Why the edges of the colorized optical flow look jaggy? about raft

[zachteed]

Yes, you are correct, results can be improved with upsampling. Last month, we started experimenting with ways of upsampling the flow fields while maintaining the simple structure of our network. The most recent version of our paper, which will appear on arxiv later this week, includes an upsampling module and predicts full resolution flow fields. Our code will be updated in a few days.

Note, the link I posted earlier is no longer activate, because the results on public leaderboards have been replaced with results from our full resolution model.

Here are some qualitative results using the updated module:

[zachteed]

The edge artifacts are due to the fact that optical flow is being predicted at 1/8 resolution, then upsampled using bilinear interpolation (see "Flow Prediction" section 3.3 of our paper). You can verify that edges are present on our Sintel submission (http://sintel.is.tue.mpg.de/hero?flow_type=Flow&method_id=2285&metric_id=0&selected_pass=0) which match the results reported in the paper.

[complexfilter]

Thanks for the clarification. Wouldn't the predicted optical flow be more accurate if you use guided image upsampling to upsample the optical flow guided by the original image? In that case, the aliasing artifacts near the edge will be removed.

[complexfilter]

Congratulations on your recent progress! I am looking forward to your updated code and paper!

[complexfilter]

Out of curiosity, have you completed the update of your training code. I notice you uploaded train.py file. Is the file the most recent one? Also, is there any no clue on the training parameters?

[zachteed]

I've just uploaded the training code / instructions.

[Guptajakala]

Yes, you are correct, results can be improved with upsampling. Last month, we started experimenting with ways of upsampling the flow fields while maintaining the simple structure of our network. The most recent version of our paper, which will appear on arxiv later this week, includes an upsampling module and predicts full resolution flow fields. Our code will be updated in a few days.

Note, the link I posted earlier is no longer activate, because the results on public leaderboards have been replaced with results from our full resolution model.

Here are some qualitative results using the updated module:

This is awesome! Out of curiosity, have you tried upsampling by deconv and how does it compare with the convex combination upsampling?