Hexen save file too large for read parameters about save-game-copier

[RevQuixo]

I guess the the TLDR of the above:

1. Timestamp is wrong
2. Hexen not having a comment in Slot 1 may prevent it from being loadable in game
3. otherwise the save file might be working in SGC..so progress!

[slinga-homebrew]

It looks like my maximum save size is 128k, I'll up that. However the "Failed to read file size" error message only happens when jo_backup_get_file_size() fails. Not sure why it's failing. Will take some time to debug.

Thanks again for the bug reports.

[slinga-homebrew]

I tested writing a 256k save. jo_backup_save_file_contents() is reporting success but writing 0 bytes. Will have to looked at Jo source.

[slinga-homebrew]

The bug is jo_backup_save_file_contents() takes the save length as an unsigned short when it needs to take the length as a unsigned int. I can make a PR and submit to Jo Engine.

[slinga-homebrew]

I submitted a PR with the fix on the Jo Engine side: johannes-fetz/joengine#44.

[slinga-homebrew]

I think it's fixed here: https://github.com/slinga-homebrew/Save-Game-Copier/releases/tag/3.2.2. I am able to copy a 256k save game. I verified listing works as well. Re-open if you still have an issue.

[bumfacekilla]

Just tried copying a Hexen save from memory cart to MODE (tried two different carts) and still get the same JO Engine error as before, SGC will not show the list of saves. Deleting the Hexen save from the cart allows SGC to list the rest of files as normal.

[slinga-homebrew]

Can you tell me the exact size of the save if possible? I made a 256k test save and it works fine. I guess I could get Hexen and give it a shot....

[slinga-homebrew]

And just to be clear, you can't list the save on the cart? So it has nothing to do with Mode?

[slinga-homebrew]

The Hexen save is working for me: https://imgur.com/a/Qxu1k5u.

[bumfacekilla]

Strange, I wonder what the issue may be. It's odd that your save is only showing as 481 blocks and mine as 3801. I believe the save file size must be dynamic, as you can save at any point in the game from the pause menu. My save for testing was made from the pause menu as soon as the first level begins.

Just did a bit more testing and took some pictures to better describe the issue. Tested with the PAL and US versions to see if it made any difference, but it didn't. As a side note, the save is cross-compatible between regions which is true of many Saturn games.

So, if there is a Hexen save file on the card, the Saturn BIOS can read and list just fine, but SGC bluescreens. Delete the Hexen save from the Saturn BIOS and SGC reads the rest of what's on the card just fine.

[slinga-homebrew]

I'm stumped pretty bad. I build a debug version with extra logging to help diagnose: https://github.com/slinga-homebrew/Save-Game-Copier/releases/tag/3.2.2.1.

Can you tell me if this error message occurs?

• jo_core_error("fileinfo failed with %d", result);

Don't worry about the other debugging messages.

[bumfacekilla]

Just tested with the debug build, the error "fileinfo failed with %d" does appear, but with "0" as the value for "%d". There were three notifications in total:

• In saturnListSaveFiles( ):
  list found: 6 count: 255

• In jo-backup-get-file-size( )
  fileinfo failed with 0

• In saturnListSaveFiles( )
  Failed to read file size!!

The first message indicates that it is aware of the presence of all saves on the cart, including the Hexen saves. At the time of testing, I had 6 saves on the cart, 2 for Hexen both at 3801 blocks and 4 more saves, the same saves as those shown in the previous comment.

I deleted the 2nd of the two Hexen saves and tested again, the results were the same except that the first message read "list found: 5 count: 255" instead of "found: 6".

[slinga-homebrew]

Thanks for the screenshots. The "fileinfo failed with 0" let's me know it's JO_BACKUP_DRIVER_GET_FILE_INFO() that's failing.

I made another debug build to test out: https://github.com/slinga-homebrew/Save-Game-Copier/releases/tag/3.2.2.2. This should have two fileinfo prints. One with the error value, and one with the name of the save it failed to open. Can you please share those results? Thanks again.

[bumfacekilla]

This is a long report, but I think I have made some valuable observations :)

The first time I tested with build 3.2.2.2, the results were the following messages:

• In saturnListSaveFiles( ) :
  list found: 5 count: 255

• In jo-backup-get-file-size( ) :
  fileinfo failed with 0

• In jo-backup-get-file-size( ) :
  fileinfo failed $_

• In saturnListSaveFiles ( ) :
  Failed to read files size!!

However, I noticed something odd. I attempted to read the cartridge again, and $_ became $_ 6. So, I attempted several more times in quick succession. Each time I did, I noticed that the _ in the $_ part of the message was gradually becoming a fuller square (something like a progress bar) each time I attempted, with a different random character appearing after the _.

Once the square had become close to full on the 10th attempt, it actually read the contents of the cart properly. It also seems that the correct details for the file can be read, but for some reason multiple passes were required to get that full info.

On my first two attempts at copying the save to the MODE SD, I had believed that the process had hung but noticed when checking in Windows that the save on my SD was larger on my second attempt which I had left running for longer than the first before powering off.

On my 3rd attempt, I was able to successfully copy the save to the MODE SD, this process was very long (as expected with a save of this size). I timed the process, to copy the entire save took approx 12m50s.

I then rebooted my Saturn, deleted the save on the cartridge from the Saturn BIOS and was then able to restore the save via SGC which worked perfectly except for one small issue. When accessing the save in game, the text field is blank. I believe this is probably related to issue #21, "SGC does not backup save file COMMENTS...". The save will still load correctly when chosen, regardless of the blank comment. Immediately saving over the file again in-game fixes the comment.

It's also worth noting that even under normal circumstances the text field for the save file in-game and the comment field in the Saturn BIOS show completely different text, but seem to somehow be related in terms of data preservation.

Hope this info helps :)

[slinga-homebrew]

Man this is confusing. I created yet another debug build with more logging: https://github.com/slinga-homebrew/Save-Game-Copier/releases/tag/3.2.2.3.

The "fileinfo failed $_" worries me. That should say HEXENSAV001. So either 1) Jo Engine is giving me bogus save filenames or 2) I'm corrupted the filenames somehow. This new build should help me figure that out.

Thanks again for debugging.

[bumfacekilla]

Thanks for the quick updates :)

I just did some testing with 3.2.2.3. When I started, I realised that any having saves on the cart other than the one for Hexen was going to make things take longer due to the extra debugging info, so I deleted those to make things a bit quicker. If it would have been helpful to have some other saves on the cart too, let me know and I will put a couple back on there and go again.

When I first attempted to read the cartridge memory, these were the messages displayed:

• In saturnListSaveFiles ( ) :
  list found: 55 count: 255
  (Maybe worth noting that the list count is wrong and the dynamic memory usage result was reporting 255967%)

• In saturnListSaveFiles ( ) :
  sat before 0) HEXENSAV001

• In jo_backup_get_file_size( ) :
  jo before fn: HEXENSAV001

• In saturnListSaveFiles ( ) :
  sat after 0) HEXENSAV001

• In saturnListSaveFiles ( ) :
  sat before 1) $|

• In jo_backup_get_file_size( ) :
  jo before fn: $|
  (the vertical line after the $ was thicker at this stage)

• In jo_backup_get_file_size( ) :
  jo after failed (101207492) fn

• In saturnListSaveFiles ( ) :
  Failed to read file size!!

So I made a second attempt to read the cartridge, and the messages were as follows:

• In saturnListSaveFiles ( ) :
  list found: 1 count: 255
  (On this 2nd attempt, the list count was correct and dynamic memory usage was reporting a more normal 20%)

• In saturnListSaveFiles ( ) :
  sat before 0) HEXENSAV001

• In jo_backup_get_file_size( ) :
  jo before fn: HEXENSAV001

• In saturnListSaveFiles ( ) :
  sat after 0) HEXENSAV001

• In saturnListSaveFiles ( ) :
  list returned: 1

• In listSaves_draw( ) :
  numSaves is 1

After this I was able to transfer the save back and forth as I had with 3.2.2.2 but then encountered another issue.

With the newly created HEXENSAV001 copied back to my cartridge, accessing the save menu in-game caused a redscreen crash. To make certain things were good with HEXENSAV001, I compared the byte count and MD5 in Windows with the info given by SGC and they matched.

I deleted HEXENSAV001 from my memory card and restored HEXENSAV002 (which I created yesterday using 3.2.2.2), with this save on the cart I experienced the same as yesterday, the save worked but was just missing it's description text. Having both saves on the cart also causes the redscreen crash.

Hexen has only two save slots in the in-game save menu, I am uncertain if this redscreen error occurring with Save 1 but not 2 has something to do with the first save slot being occupied without a comment/save description being present and Hexen being unable to process that scenario. When just the 2nd is save present, the first slot shows text to inform the player that the slot is empty, but still allows you to load the descriptionless slot 2 save.

[slinga-homebrew]

I think I fixed a bug in Jo Engine. Do you mind testing another debug build? https://github.com/slinga-homebrew/Save-Game-Copier/releases/tag/3.2.2.4.

No changes to SGC, I just changed a size parameter in Jo Engine.

[bumfacekilla]

The change seems to have fixed the filesize reading, but introduces a new minor issue :)

The first attempt at reading the cart took me straight through to the file list. The messages were the following:

• In saturnListSaveFiles ( ) :
  list found: 1 count: 255

• In saturnListSaveFiles ( ) :
  sat before 0) HEXENSAV001

• In jo_backup_get_file_size( ) :
  jo before fn: HEXENSAV001

• In saturnListSaveFiles ( ) :
  sat after 0) HEXENSAV001

• In saturnListSaveFiles ( ) :
  list returned: 1

• In listSaves_draw( ) :
  numSaves is 1

I was again able to backup the save to the MODE SD, but noticed something strange when checking the backup in Windows, the date stamp is now incorrect. I checked my Saturn's internal clock (it was correct) and tested with two more saves to be certain, one from internal memory and one from cart. These were copied today, at around 12:45 pm, but the date modified in Windows shows 01/03/2020 at around 23:20pm. Until 3.2.2.4, the date/time information had always been correct.

Now when Hexen is running, trying to access the save menu with the HEXENSAV001 restored to slot 1 no longer causes a redscreen crash, but the game interface becomes very unresponsive and will not load the save. It takes several seconds for the menu to appear and then the description fields for both save slots are empty, even though HEXENSAV002 is not occupied at this time and would usually show text to indicate that the slot is empty. It then takes several more seconds to be able to move the cursor, attempting to load the slot 1 save returns the player to the previous menu.

As with yesterdays attempts, restoring HEXENSAV002 to slot 2 and leaving slot 1 unoccupied works fine, regardless of the missing description text.

[slinga-homebrew]

Thank you for your debugging help, it was quite valuable in diagnosing the issue as I could not reproduce it locally. I have tagged a new stable release: https://github.com/slinga-homebrew/Save-Game-Copier/releases/tag/3.3.0.

Regarding the timestamp and the comment it should get fixed when I get around to implementing #21.