Wrong video downloaded about twitter-video-dl

[inteoryx]

Thank you very much for all this. Excellent bug report and, even better, it came with a fix. I have pulled in your fix and updated the test cases file. Seems to all be working now. I really appreciate your contribution.

[Perkinss27]

I'm working on a fix, I'll share it here later this day or tomorrow.

[Perkinss27]

Fix commited to my fork.
Code might be messy, and we might have to do further checking before assuming it works correctly.
It's an early fix that worked for the case I explained earlier.
For more details about what's modified and what's added please read my comment.

[inteoryx]

Awesome, thanks very much for finding this issue and submitting a fix. There is an issue with your fix which is that it can download lower resolution than the max available, for example try your code with https://twitter.com/Queen__Deshda/status/1452467693517426697 - it downloads a lower resolution version of the video than the mainline. Ideally I would like to keep the max resolution behavior.

When I have a chance I will try to update your code to keep the max resolution version and merge it. Or, if you want to take that on, that would be great. From the tests directory you can run "python TestVideoDownloader.py" and it will catch errors like this.

I really do appreciate your PR and will merge it soon once this issue is ironed out - thanks again.

[Perkinss27]

Hi, thank you for your feedback. I took time to find what causes the bug you describe here and it looks like it does not come from my update. The bug occurs because sometime ext_tw_pattern and amplitude_pattern do not find the highest resolution video link.
I looked into the json data received from requesting https://twitter.com/Queen__Deshda/status/1452467693517426697 and manually found those 3 video links (+ 1 container) :

{
    "bitrate": 256000,
    "content_type": "video/mp4",
    "url": "https://video.twimg.com/ext_tw_video/1452467633727627271/pu/vid/480x270/mJYI0tdF0rk4Boxq.mp4?tag=12"
},
{
    "bitrate": 2176000,
    "content_type": "video/mp4",
    "url": "https://video.twimg.com/ext_tw_video/1452467633727627271/pu/vid/1280x720/-Rk-UWbHoRufnCUr.mp4?tag=12"
},
{
    "bitrate": 832000,
    "content_type": "video/mp4",
    "url": "https://video.twimg.com/ext_tw_video/1452467633727627271/pu/vid/640x360/JSZJP1JdQJDhuEdr.mp4?tag=12"
}

In this case only https://video.twimg.com/ext_tw_video/1452467633727627271/pu/vid/480x270/mJYI0tdF0rk4Boxq.mp4?tag=12 and https://video.twimg.com/ext_tw_video/1452467633727627271/pu/vid/640x360/JSZJP1JdQJDhuEdr.mp4?tag=12 are matched by your regular expression, so I started comparing links and spotted a difference.
In fact, the highest resolution link contains '-' in its field just after its resolution (1280x720) : -Rk-UWbHoRufnCUr. Comparing these fields for the 3 video links we have :

mJYI0tdF0rk4Boxq
-Rk-UWbHoRufnCUr
JSZJP1JdQJDhuEdr

Next step was to look at how this field is handled by your regular expression :

ext_tw_pattern = re.compile(r'(https://video.twimg.com/ext_tw_video/(\d+)/pu/vid/(\d+x\d+)/\w+.mp4\?tag=\d+)')

By quickly looking at it, ext_tw_pattern handles the previously described field with the \w+ statement.
According to google : \w (lowercase w) matches a "word" character: a letter or digit or underbar [a-zA-Z0-9_].
So the bug occurs because \w does not match a "word" with a '-'.
I then updated the regular expression to :

ext_tw_pattern = re.compile(r'(https://video.twimg.com/ext_tw_video/(\d+)/pu/vid/(\d+x\d+)/[^.]+.mp4\?tag=\d+)')

replacing \w by [^.] which means anything except a '.' .
And ran a quick test comparing deprecated regular expression and the updated one on the 3 above links.

Bingo, highest resolution matched ! The new pattern should fix that bug.

So I changed both ext_tw_pattern and amplitude_pattern because they were using \w.
You can see that fix on my fork's last commit.