Formerly live https://youtu.be/4BL65HElOPg fails to download [same "no such column: error" issue as #186 ?] about calibre-web

[deldesir]

means the video got stuck due to unavailable fragments

Dozens of "Some fragments are taking longer than expected to download" messages need to be explained better in "Tasks" view for everyday users:

1. What does this really mean?

This is an unrecoverable error happened mysteriously with some videos. During the download some fragments just gone missing and the download is stuck. Nothing to see with Calibre-Web, xklb and yt-dlp. The expected fragments just don't leave YouTube servers.

2. Being told dozens of times to "please wait" is not believable — is this truly the best that can be done?

Fixed in #223

3. Conversely, worst case if there's absolutely nothing that can be done in this case and such cases, the root cause yt-dlp issue(s) need to be mentioned!

There is nothing we can do actually. This is an external issue yt-dlp cannot do anything about

4. Finally, what can be done about (1) download.py: "sqlite3.OperationalError: no such column: error" after "TypeError: 'NoneType' object is not subscriptable" (2) editbooks.py: SAWarning: Object of type not in session, add operation along 'Authors.books' won't proceed #186 ?

I have done my best to get rid of the warning editbooks.py: SAWarning: Object of type <Books> not in session, add operation along 'Authors.books' won't proceed. There is still work to do to ship a successful PR.

[holta]

@deldesir

Is it possible the video's long length/duration (1:56:49) might be part of the problem?

[holta]

https://dpaste.com/4D4SEN6F6 Ubuntu 24.04 here:

Lines 1110 and 1111 pasted in below — this issue appears to be the same as:

• #186

[2024-07-05 23:59:45,842] ERROR {cps.tasks.download:132} An error occurred during the subprocess execution: 'NoneType' object is not subscriptable
[2024-07-05 23:59:45,843] ERROR {cps.services.worker:202} no such column: error
Traceback (most recent call last):
  File "/usr/local/calibre-web-py3/cps/tasks/download.py", line 93, in run
    requested_file = conn.execute("SELECT path FROM media WHERE webpath = ? AND path NOT LIKE 'http%'", (self.media_url,)).fetchone()[0]
                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/calibre-web-py3/cps/services/worker.py", line 199, in start
    self.run(*args)
  File "/usr/local/calibre-web-py3/cps/tasks/download.py", line 133, in run
    self.message = f"{self.media_url_link} failed to download: {self.read_error_from_database()}"
                                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/calibre-web-py3/cps/tasks/download.py", line 149, in read_error_from_database
    error = conn.execute("SELECT error FROM media WHERE webpath = ?", (self.media_url,)).fetchone()[0]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: no such column: error

[holta]

@deldesir do you understand this bug @nzola reported about a week ago?

[deldesir]

Yes, that means the video got stuck due to unavailable fragments.

[holta]

means the video got stuck due to unavailable fragments

Dozens of "Some fragments are taking longer than expected to download" messages need to be explained better in "Tasks" view for everyday users:

1. What does this really mean?
2. Being told dozens of times to "please wait" is not believable — is this truly the best that can be done?
3. Conversely, worst case if there's absolutely nothing that can be done in this case and such cases, the root cause yt-dlp issue(s) need to be mentioned!
4. Finally, what can be done about #186 ?

[holta]

There is nothing we can do actually. This is an external issue yt-dlp cannot do anything about

1. Ok. So please now provide a link in "Tasks" view to a yt-dlp (or similar!) ticket explaining the context. Thanks.

2. Explain here why the error message is being repeated, so everyone can understand the root cause, and what it means.

   Whereas suppressing error messages is almost always neither useful nor healthy.

3. Where is the work below? Thanks!

I have done my best to get rid of the warning editbooks.py: SAWarning: Object of type <Books> not in session, add operation along 'Authors.books' won't proceed. There is still work to do to ship a successful PR.