errors on new backup. about duplicati

[Jojo-1000]

What did you do to get here? It looks like you run database recreate with existing backup files? If that is the case, please check that your encryption is configured the same. If you set no encryption but the files are encrypted, the symptoms are the same (#5007).

If that is not what you did, you need to give more information about your setup and the configuration.

[Commissar0617]

I did not use any encryption on this backup. the DB recreate was my attempt to get it working, but i got the same errors when i tried to manually run it after freshly creating.

[Jojo-1000]

The files you have are definitely encrypted (.aes extension). If this is a new backup, are there still files from a different backup in the same folder? The earliest versions you have in the log are from last year.

You need a separate folder for every backup. If that is not the reason, please describe in more detail what steps you already tried and what your setup looks like. Otherwise I can only keep guessing.

[Commissar0617]

The files you have are definitely encrypted (.aes extension). If this is a new backup, are there still files from a different backup in the same folder? The earliest versions you have in the log are from last year.

You need a separate folder for every backup. If that is not the reason, please describe in more detail what steps you already tried and what your setup looks like. Otherwise I can only keep guessing.

yeah, that explains it. i was using an existing folder lol

[schmidhorst]

I have two backup drives with backups, which have been created on a Synology NAS via Duplicati in an Docker container and stored on external USB drives. I am using Duplicati - 2.0.7.1_beta_2023-05-25.
The backups have been created via mono and Duplicati.CommandLine.exe with an bash script with aes encryption and zip compression. One drive with BTRFS and one with ext4.

Restoring files while the disks connected to the Synology works well.
If I am attaching the USB drives to a Laptop with pure Ubuntu, all works also well.

But if I am attaching the drives to my Windows 10 computer and using either the btrfs.sys driver https://github.com/maharmstone/btrfs or for ext4 the wsl (https://learn.microsoft.com/en-us/windows/wsl/wsl2-mount-disk) to mount the USB drives (to D: or to \wsl.localhost\Ubuntu-20.04\mnt\wsl\PHYSICALDRIVE3p1) and then trying to use "Direct restore from backup files ...", the step 3 "Select files" is causing this error:
2024-01-05 22:13:35 +01 - [Warning-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-FileProcessingFailed]: Failed to process file: duplicati-20230330T000740Z.dlist.zip.aes
{
"MainOperation": "Repair",
"ParsedResult": "Warning",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "2024-01-05T21:13:35.1028443Z",
"BeginTime": "2024-01-05T21:13:27.9053351Z",
"Duration": "00:00:07.1975092",
"MessagesActualLength": 7,
"WarningsActualLength": 1,
"ErrorsActualLength": 0,
"Messages": [
"2024-01-05 22:13:28 +01 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: Die Operation Repair wurde gestartet",
"2024-01-05 22:13:28 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()",
"2024-01-05 22:13:34 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (801 Bytes)",
"2024-01-05 22:13:34 +01 - [Information-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-RebuildStarted]: Rebuild database started, downloading 1 filelists",
"2024-01-05 22:13:34 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20230330T000740Z.dlist.zip.aes (2,94 MB)",
"2024-01-05 22:13:34 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20230330T000740Z.dlist.zip.aes (2,94 MB)",
"2024-01-05 22:13:35 +01 - [Information-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-RecreateOrUpdateOnly]: Recreate/path-update completed, not running consistency checks"
],
"Warnings": [
"2024-01-05 22:13:35 +01 - [Warning-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-FileProcessingFailed]: Failed to process file: duplicati-20230330T000740Z.dlist.zip.aes"
],
"Errors": [],
"BackendStatistics": {
"RemoteCalls": 2,
"BytesUploaded": 0,
"BytesDownloaded": 3079629,
"FilesUploaded": 0,
"FilesDownloaded": 1,
"FilesDeleted": 0,
"FoldersCreated": 0,
"RetryAttempts": 0,
"UnknownFileSize": 0,
"UnknownFileCount": 0,
"KnownFileCount": 0,
"KnownFileSize": 0,
"LastBackupDate": "0001-01-01T00:00:00",
"BackupListCount": 0,
"TotalQuotaSpace": 0,
"FreeQuotaSpace": 0,
"AssignedQuotaSpace": 0,
"ReportedQuotaError": false,
"ReportedQuotaWarning": false,
"MainOperation": "Repair",
"ParsedResult": "Success",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "0001-01-01T00:00:00",
"BeginTime": "2024-01-05T21:13:27.9053351Z",
"Duration": "00:00:00",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null
}
}

Same problem if I am using Duplicati in a VMware virtual Ubuntu machine on the Windows laptop.

The "Test connection" before is reporting "Success Connection worked!". With the correct passphrase the list of backup dates is generated well, but instead of a list of files the above error occurs. I have tried with and without the option no-local-blocks. No difference.

My folder with the backup is containing several different versions of my backup, but it's all the same source folders, the same passphrase.
I have also tried to copy the whole folder with the backup to the internal NTFS drive and to run the restore from there. This produces the same error.
I have installed https://www.aescrypt.com to my Windows. Decryption works well with this tool. And the generated ZIP file can be unzipped then in the Windows File Explorer without problems.
If I am decrypting all the files with an batch loop and the command line version aescrypt.exe, deleting the *.aes files, then the file restore with Duplicati GUI is working well! There is only for each ZIP file a warning like
"Warning-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-RewritingFilenameMapping]: Unable to find volume duplicati-b3f93ddf88b4c468cb263604d26e97162.dblock.zip.aes, but mapping to matching file duplicati-b3f93ddf88b4c468cb263604d26e97162.dblock.zip"

[Jojo-1000]

@schmidhorst Please create a new issue for your problem, because it is different from this instance.

If you open the live log (on the about page) with level information in a second tab while you do the direct restore, you should be able to see more descriptive error messages about the failed files.

It would also be helpful if you are able to reproduce your problem with a simple dummy backup that is freshly created, to rule out problems that might have accumulated over time.