How many uploadFiles do we receive for already uploaded files? about td

[iiscosd3]

Let's imagine that we have a ~4gb file, and a premium subscription on the account. So we are trying to upload the file using preliminary wrappers, we do perform the request, wait until the file is loaded, and chunk are saved on the server-side/cdn whatever, close the database, then we open it again, let's say 10 minutes later, we are trying to upload it, but due to the tdlib cache we just run into a weird situation where the filds 'is_uploading_active' are being set as true in the response to the original 'preliminaryUploadFile', even though the file is already uploaded and the uploaded_size matches the size, and expected_size fields values, while 'is_uploading_completed' is being set to false.

Then, we do receive 3 updateFiles, last 2 of which are identical, so even if i am going to use a deferred promise its still remaining unreliable? And both are set with is_uploading_active, is_uploading_completed - false, so how do we exactly determine when we should deal with the file now?

preliminaryUploadFile

As you can see its still stating 'is_uploading_active' as true and the other field as false etc.

res {
  _: 'file',
  id: 1229,
  size: 4151822324,
  expected_size: 4151822324,
  local: {
    _: 'localFile',
    path: '/media/root/732d2c75-7703-405f-ab61-22daf460bdf51/root/Videos/2023-12-17_08.22.44.mp4',
    can_be_downloaded: false,
    can_be_deleted: false,
    is_downloading_active: false,
    is_downloading_completed: true,
    download_offset: 0,
    downloaded_prefix_size: 4151822324,
    downloaded_size: 4151822324
  },
  remote: {
    _: 'remoteFile',
    id: '',
    unique_id: '',
    is_uploading_active: true,
    is_uploading_completed: false,
    uploaded_size: 4151822324
  }
}

updateFile # 1

'is_uploading_active: true, is_uploading_completed: false,'

{
  _: 'updateFile',
  file: {
    _: 'file',
    id: 1229,
    size: 4151822324,
    expected_size: 4151822324,
    local: {
      _: 'localFile',
      path: '/media/root/732d2c75-7703-405f-ab61-22daf460bdf51/root/Videos/2023-12-17_08.22.44.mp4',
      can_be_downloaded: false,
      can_be_deleted: false,
      is_downloading_active: false,
      is_downloading_completed: true,
      download_offset: 0,
      downloaded_prefix_size: 4151822324,
      downloaded_size: 4151822324
    },
    remote: {
      _: 'remoteFile',
      id: '',
      unique_id: '',
      is_uploading_active: true,
      is_uploading_completed: false,
      uploaded_size: 4151822324
    }
  }
}

updateFile # 2

'is_uploading_active: false, is_uploading_completed: false,'

{
  _: 'updateFile',
  file: {
    _: 'file',
    id: 1228,
    size: 4151822324,
    expected_size: 4151822324,
    local: {
      _: 'localFile',
      path: '/media/root/732d2c75-7703-405f-ab61-22daf460bdf51/root/Videos/2023-12-17_08.22.44.mp4',
      can_be_downloaded: false,
      can_be_deleted: false,
      is_downloading_active: false,
      is_downloading_completed: true,
      download_offset: 0,
      downloaded_prefix_size: 4151822324,
      downloaded_size: 4151822324
    },
    remote: {
      _: 'remoteFile',
      id: '',
      unique_id: '',
      is_uploading_active: false,
      is_uploading_completed: false,
      uploaded_size: 4151822324
    }
  }
}

updateFile # 3

'is_uploading_active: false, is_uploading_completed: false,'

{
  _: 'updateFile',
  file: {
    _: 'file',
    id: 1229,
    size: 4151822324,
    expected_size: 4151822324,
    local: {
      _: 'localFile',
      path: '/media/root/732d2c75-7703-405f-ab61-22daf460bdf51/root/Videos/2023-12-17_08.22.44.mp4',
      can_be_downloaded: false,
      can_be_deleted: false,
      is_downloading_active: false,
      is_downloading_completed: true,
      download_offset: 0,
      downloaded_prefix_size: 4151822324,
      downloaded_size: 4151822324
    },
    remote: {
      _: 'remoteFile',
      id: '',
      unique_id: '',
      is_uploading_active: false,
      is_uploading_completed: false,
      uploaded_size: 4151822324
    }
  }
}

So there's no other updates besides these ones, no matter how long you wait for.

Should we simply rely on expected_size === uploaded_size condition instead?
And please, do not reply as 'Updates updateFile will be used to notify about upload progress and successful completion of the upload. The file will not have a persistent remote identifier until it is sent in a message.', it can't be applied in current context since we simply do not even know when the file is uploaded to send the message, even though some clients ignore it completely and send files as is even before they were uploaded. Such as inline music bots etc. Shall we replicate their behaviour? Yet i need to know what to rely on after restarts.

[iiscosd3]

Even though #2784 describes it as expected behaviour, we still need an additional, internal field to determine the status, otherwise its pointless in general, since yeah, whatever, 'is_uploading_completed' might be related to even regular updates, where you need to know whether you can start downloading the media off the message or not, but for uploading process itself keeping it on 'false' is idek... Especially inside of a 'updateFile', maybe either a new field needs to be implemented or simply a new event, designed specifically for uploading process so updateFile remain to the dowloading/updating process of incoming messages?

[iiscosd3]

In telegram android it has a delegate https://github.com/DrKLO/Telegram/blob/master/TMessagesProj/src/main/java/org/telegram/messenger/FileUploadOperation.java#L588-L614
and the condition sets:
https://github.com/DrKLO/Telegram/blob/master/TMessagesProj/src/main/java/org/telegram/messenger/FileUploadOperation.java#L482-L489
and
https://github.com/DrKLO/Telegram/blob/master/TMessagesProj/src/main/java/org/telegram/messenger/FileUploadOperation.java#L373-L375...

Everything is overcomplicated and needs to be wrapped inside of the tdlib instead: https://github.com/DrKLO/Telegram/blob/a906f12aaec2768969c77650a7e4b377baa6cf2a/TMessagesProj/src/main/java/org/telegram/messenger/FileUploadOperation.java#L250, as an example.

[iiscosd3]

Is this one reliable? https://github.com/TGX-Android/Telegram-X/blob/ccefce2b595e0654769df7d415d0d9bb5e83a456/app/src/main/java/org/thunderdog/challegram/data/TD.java#L3306

[levlam]

All updates are correct. preliminaryUploadFile never completes upload of a file even all data is uploaded.
You never need to use preliminaryUploadFile before sending it in a message if you have complete file beforehand.

[iiscosd3]

All updates are correct. preliminaryUploadFile never completes upload of a file even all data is uploaded. You never need to use preliminaryUploadFile before sending it in a message if you have complete file beforehand.

Ah... Sure. So its mainly for internal usage/video/voice notes?

[iiscosd3]

What about inputFileGenerated? What is it used for? It says something something that it has to be preserved between application restarts, but what does it exactly do? Is it used in some kind of 'reset sent media cache', or tdlib simply performs some kind of basic resize/trim actions on... basic file types? I need to preserve uploaded file 'id' or somehow reupload it in the other chat, without memorizing the unique_file_id and file_id. So how exactly do we do that if having issues with 'is_uploading_completed', working a little bit different in these scenarios.

[levlam]

Video and voice notes can be uploaded during recording without sending a message, and preliminaryUploadFile can be used for this.

See #126 for examples of inputFileGenerated usage.

[iiscosd3]

Alright.