Nested folders do not work as I understand... about steamdownloader

[klukule]

Hey there,

hard to debug without more data, it would be nice if you could share more exact details about what game and what file is being made etc...
Feel free to send it to my email that you can find on my profile. I'll see what I can do tomorrow

[UrbanSide]

@klukule Thank you very much for your communication, I will send it during the day (it’s already night here)
P.S The game I tested was Conan Exiles

[UrbanSide]

@klukule I sended you email with all files and info)

[klukule]

Sorry for the delay with fixing this, I've been quite busy IRL, so I just got to it... I have pushed a fix that solves this specific issue :)

[UrbanSide]

Sorry for the delay with fixing this, I've been quite busy IRL, so I just got to it... I have pushed a fix that solves this specific issue :)

Thx,i go testing)

[UrbanSide]

[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' bec
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\swiftshader' because a file or directory with the same name already exists.teamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' bec
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\resources' because a file or directory with the same name already exists.\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' bec
│ [VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\resources' because a file or directory with the same name already exists.teamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' bec
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.ster\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a f
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' bec
│ ████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ec
[VERIFY] File failed to verify after 3 attemptseady exists.                                                   00:00:00[V
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.er\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' bec
[VERIFY] File failed to verify after 3 attemptseady exists.er\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
0,00% Launcher\FuncomLauncher.exe                                                                           00:00:01[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\resources' because a file or directory with the same name already exists.s.er\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\resources' because a file or directory with the same name already exists.\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.er\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.ster\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a f
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' bec
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a
[VERIFY] File failed to verify after 3 attemptsownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' b
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.er\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' bec
                                                                                                                    [VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' bec
[VERIFY] File failed to verify after 3 attemptseady exists.teamDownloader\bin\Debug\net7.0\cs2\Engine' because a file or
[VERIFY] File failed to verify after 3 attemptss.oader-master\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.er\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] File failed to verify after 3 attemptss.oader-master\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.master\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] File failed to verify after 3 attemptss.                                                           00:00:00[VER
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.ster\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a f
[VERIFY] File failed to verify after 3 attemptssts.sts.master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.er\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] File failed to verify after 3 attemptss.r-master\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a file or
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.master\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] File failed to verify after 3 attemptss.oader-master\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.er\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.er\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] File failed to verify after 3 attemptss.oader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.er\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] File failed to verify after 3 attemptss.oader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.er\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] File failed to verify after 3 attemptss.oader-master\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] File failed to verify after 3 attemptss.oader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.                                                 00:00:00[VER
[VERIFY] File failed to verify after 3 attemptsder-master\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a file or
100,00% ConanSandbox\Binaries\Win64\BattlEye\EULA\zh.txt                                                    00:00:00[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.master\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a fil
[VERIFY] File failed to verify after 3 attemptss.xists.master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Launcher\locales' because a file or directory with the same name already exists.er\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] File failed to verify after 3 attempts exists.master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a file or directory with the same name already exists.oader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] File failed to verify after 3 attempts exists.master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
0,00% TheColdEmbrace-Main.pak                                                                               00:00:01[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a file or directory with the same name already exists.dy exists.master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\Engine' because a file or directory with the same name already exists.xists.master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.                                                     00:00:00[VER
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
3,33% Verify Progress (Queued: 612)                                                                           00:00:01[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.r-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] File failed to verify after 3 attempts exists.
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] File failed to verify after 3 attempts exists.
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
 a file or directory with the same name already exists.
[VERIFY] File failed to verify after 3 attemptsnloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.
[VERIFY] File failed to verify after 3 attempts60432)                                                       00:00:00[VER
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.
[VERIFY] File failed to verify after 3 attempts
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.
[VERIFY] File failed to verify after 3 attempts
[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.
[VERIFY] File failed to verify after 3 attempts
100,00% Launcher\d3dcompiler_47.dll (4194304 - 4287584)                                                     00:00:00[VERIFY] Exception - Cannot create 'D:\SteamDownloader-master\SteamDownloader\bin\Debug\net7.0\cs2\ConanSandbox' because a file or directory with the same name already exists.
[VERIFY] File failed to verify after 3 attempts

[UrbanSide]

And to be honest, among all this progress, I don’t really understand how to track how many% the game itself has been downloaded?
And apparently it will automatically download all the mods that are attached to the manifest?

[klukule]

The issue probably stems from the fact that the download you already have is already corrupted from previous versions, I've tested it myself on clean download and everything worked just fine, regarding progress monitoring there are 4 parts of the progress:
Parsing - reading the manifetsts and building the structure
Verify - goes over existing files and checks what parts need to be downloaded
Download - download threads themself
Write - Progress of writing

Due to the queue based nature of download it's hard to keep track of overall progress... especially since total workload is not known beforehand... since only during the verify we find out what needs to be downloaded and written...

But, I think that looking at the "Verify" can provide rough overview on how stuff is going

[UrbanSide]

Yes, with a clean installation, I see that things are working. Thank you, I will further figure out how to put this together with my project)

[UrbanSide]

I was just inspired by the Tek Launcher project and the author said that he uses the SteamKit method and somehow tracks everything through it. But I still couldn’t figure out how)

[UrbanSide]

And for some reason it just wildly clogs up the RAM

[klukule]

I'm not familiar with Tek Launcher so I can't tell how they keep track of stuff... I presume they do stuff in single threaded fashion which is easier to track...

And yes, this program takes quite a lot of RAM as it does a lot of things, you could try passing in the arguments to use lower amount of threads and play with the combination of them... it will slow down the downloads, but it will also lower the resource use

[UrbanSide]

I'm not familiar with Tek Launcher so I can't tell how they keep track of stuff... I presume they do stuff in single threaded fashion which is easier to track...

And yes, this program takes quite a lot of RAM as it does a lot of things, you could try passing in the arguments to use lower amount of threads and play with the combination of them... it will slow down the downloads, but it will also lower the resource use

Yes, I would be glad even if it downloaded the file by file in the queue)) It would be easier to track the process)
Thanks, I'll look into it

[klukule]

In that case you could probably limit the Verify thread count to 1 as that would go file by file, well... almost, it would most likely do couple of files at once since verify does it's stuff until the download queue is not full, but with that you could probably also go in the code and limit the queue sizes to be small enough for it to be almost synchronized...

But as I've said, this whole process is a three-way balancing act... since you're balancing speed, usage of computer resources and in your cae also ease of use

[UrbanSide]

In that case you could probably limit the Verify thread count to 1 as that would go file by file, well... almost, it would most likely do couple of files at once since verify does it's stuff until the download queue is not full, but with that you could probably also go in the code and limit the queue sizes to be small enough for it to be almost synchronized...

But as I've said, this whole process is a three-way balancing act... since you're balancing speed, usage of computer resources and in your cae also ease of use

Hmmm, thanks, I'll think about how to do this)