DisplayMagician Origin library parsing misses games about displaymagician

[terrymacdonald]

Hi Parvel, The manifest files are files included with the origin launcher, and you shouldn't need to edit them. They are part of that game launcher, and DisplayMagician simply tries to read them to find games. Can you please tell me why you want to edit those manifest files? Thanks Terry

…

On Fri, 8 Dec 2023, 15:45 Pavel Krestyannikov, ***@***.***> wrote: How do I edit the manifest file?) (mfst) — Reply to this email directly, view it on GitHub <#43 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLU5LEAFXW77WSLVXCG44LYIJ5NNAVCNFSM5FH6FIN2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBUGY2DMOJVGY2Q> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[terrymacdonald]

Okay @DragRedSim - so I only have one game, and that uses the new format. The current logic only works with that format. For reference, my Origin client is Version 10.5.104.48966 - 769862.

I should be able to make the version 3.0 format work fine as it contains a runtime XML item, but I think I'll struggle with the BF1942 (which is manifestVersion="2.1"). There doesn't appear to be any runtime listed within the installer.xml for the BF1942 game. Are you able to do some detective work on that Battlefield 1942 install directory to see where the runtime is listed? It could be in some other files in there or as you say within the registry.

Up until now I've been doing the quick and dirty text file parsing rather than ingesting XML, and that is mainly because XML parsing is SSSLLLOOOWWWWW and uses a LOT of memory. If I can get text parsing working then I will, as it will be way faster for parsing a large number of games as this happens on startup at the moment with DisplayMagician. But if that doesn't pan out, then XML it is :(.

Anyway, I'll get to work on the v3.0 XML parsing, and if you can get back to me with the v2.1 information I'll add that in too. I'll probably need to give you a custom build to test for me too once we're done :). Hope that's ok.

Thanks
Terry

[terrymacdonald]

Hi @DragRedSim,

I've restructured the Origin detection logic now and it's a lot more robust. It should handle manifest format v4.0+ games, v3.x games, but won't handle the v2.1 games until you find me some places in those games where there is a GameExePath :).

I've attached a new build for you to download and test. I think you'll be able to just install this over the top of your existing DisplayMagician v2.0.1 and it should overwrite it.... If not you'll need to uninstall the old one and install this one.

DisplayMagicianSetup-v2.0.1.71.zip

As mentioned above, I really need you to check Battlefield 1942 and see if the game executable location is found anywhere within the game folder or if it's in registry. I need that information for the v2.1 manifest format detection, otherwise those games just won't get detected.

As always, if you have files to attach please reply in your web browser and attach the files directly to the reply, as github strips attachments sent via the email gateway.

Thanks
Terry

[DragRedSim]

Hey there,
Sorry about the delay. I'm a little snowed under with Things^TM piling up faster than I can handle them.
Probably the easiest way to handle this is to take it back to the basics. The flow path of the current system is:

• Origin Local Content Directory - get all .mfst files within game-based subdirectories of that.
• .mfst file: read the string, parse out the dipinstallpath location, URLDecode that string.
• Installation directory: check __Installer subdirectory for installerdata.xml file,
• installerdata.xml - read it through, get a path with either an absolute file path, or a combination of a registry key and an .exe name.
• • Registry: extract the path fron the registry key, and concat it with the exe name.
• Finally, add the game to the library with the following information: gameID, gamename, game EXE path, game icon path (calculated as from the EXE at this point).

Here's the thing - at step 3, we already have the installation directory, and anything else beyond that is effectively just confirmation of the information we already have. Therefore, I propose an alternate program flow:

• Origin Local Content Directory - get all .mfst files within game-based subdirectories of that.
• .mfst file: read the string, parse out the dipinstallpath location, URLDecode that string.
• Installation directory: check __Installer subdirectory for installerdata.xml file,
• installerdata.xml - read it through, get the installManifest/filePath value at the end of the file (normally /Support/mnfst.txt, this is a reference based on the installation directory collected from the dipinstallpath value in the .mfst file.)
• Within the mnfst.txt file is a list of strings, each stating one file that is installed as part of the game package. If we then filter that to the files in the root directory which are .exes (by use of a regex that excludes any string that contains a backslash), there is a high probability that the first one returned is the game .exe. Alternately, if there is a file with a matching file of the same name with a .par extension, that seems to be the game exe.

Although this should be suitable for all Origin games, it may be better to lock this workflow to only those games that do not have a path string within their installerdata.xml to work with (therefore, anything with a manifest version less than 3.0); this would help reduce the chances of false-positives on detecting the game EXEs. I say less than 3.0 since I even found a version 1.0 manifest today in my collection! It also has a mnfst.txt file, so should be safe in that regard.

The key thing to note here is that the __Installer/installdata.xml file is in the game install location - so we already have a path, we just need to know which exe to run from that path!

I also came across another method, documented at https://github.com/erri120/GameFinder#origin, which utilises the id string within the .mfst file as part of an API call to an Origin server, to get information about the package; this information does contain the classic registry install path. This may be an alternate option, however it's dependent on calling out to an external server.

Attached is an example of the BF1942 folder structure, with the exe location relative to the installerdata.xml and mnfst.txt files. BF1942.zip This structure seems to be mirrored across all Origin games, including those without a parsable path in the installerdata.xml.

I do apologise for not having been able to test the debug build just yet, but I wanted to tackle the manifest lookup first.

[terrymacdonald]

Hi David, The v2.1 doesn't have the Filepath in it. The v4 and v3 files do. I try to identify the specific exe that starts the game definitively, by finding it is a config file or in registry. The problem is that the v2.1 file doesn't contain the exe for the game. The current code works exactly as you described in order to specifically and definitively match the game exe. For reliability reasons I want to be sure the game exe is the right exe to minimise bugs for my users. I really need to find another place to get the Filepath for a v2.1 installed game (e.g. 1942). Origin must have a way it processes it. I don't want to change the logic drastically for the future, as the current one will work fine for v3 and V4 manifest files with the recent changes. I don't know what guarantee there is that the game exe is the first one in the list, and I think it's too risky to use that strategy. There must be a 'way' that origin itself finds the game exe when it runs. If I had a game old enough to be installed as V2.1 I'd use process monitor to see what origin was attempting to read. Unfortunately I only have one v4.0 manifest game. I'm not sure if I'm going to spend time to build a new processing pipeline for an old file format when a user can manually add the game exe as an 'application executablec in the section above the game selector in the edit shortcut form in order to get around this problem. I decided at the start of building DisplayMagician that I wasn't going to build any online connectivity to log into games libraries. I don't want to store logins and passwords for users access to their online accounts. I was very focused on only reading what was on the person's SSD, as I only need to know what games are installed and usable now. There are other platforms for full library access, like GOG, and those work well for that problem. For those reasons, I'll have a closer look at the information you sent through, but I'm not confident that I'll be able or willing to add game parsing for v2.1 manifest files or earlier with the information I have at present. I'll make a definitive decision in a few days after some more sleuthing. I really am thankful for your help. I feel very lucky to be in a position to at least take a little annoyance out of people's days during a global pandemic. Thanks Terry

…

On Wed, 6 Oct 2021, 03:36 David Ross Smith, ***@***.***> wrote: Hey there, Sorry about the delay. I'm a little snowed under with Things^TM piling up faster than I can handle them. Probably the easiest way to handle this is to take it back to the basics. The flow path of the current system is: - Origin Local Content Directory - get all .mfst files within game-based subdirectories of that. - .mfst file: read the string, parse out the dipinstallpath location, URLDecode that string. - Installation directory: check __Installer subdirectory for installerdata.xml file, - installerdata.xml - read it through, get a path with either an absolute file path, or a combination of a registry key and an .exe name. - - Registry: extract the path fron the registry key, and concat it with the exe name. - Finally, add the game to the library with the following information: gameID, gamename, game EXE path, game icon path (calculated as from the EXE at this point). Here's the thing - at step 3, we already have the installation directory, and anything else beyond that is effectively just confirmation of the information we already have. Therefore, I propose an alternate program flow: - Origin Local Content Directory - get all .mfst files within game-based subdirectories of that. - .mfst file: read the string, parse out the dipinstallpath location, URLDecode that string. - Installation directory: check __Installer subdirectory for installerdata.xml file, - installerdata.xml - read it through, get the installManifest/filePath value at the end of the file (normally /Support/mnfst.txt, this is a reference based on the installation directory collected from the dipinstallpath value in the .mfst file.) - Within the mnfst.txt file is a list of strings, each stating one file that is installed as part of the game package. If we then filter that to the files in the root directory which are .exes (by use of a regex that excludes any string that contains a backslash), there is a high probability that the first one returned is the game .exe. Alternately, if there is a file with a matching file of the same name with a .par extension, that seems to be the game exe. Although this should be suitable for all Origin games, it may be better to lock this workflow to only those games that do not have a path string within their installerdata.xml to work with (therefore, anything with a manifest version less than 3.0); this would help reduce the chances of false-positives on detecting the game EXEs. I say less than 3.0 since I even found a version 1.0 manifest today in my collection! It also has a mnfst.txt file, so should be safe in that regard. The key thing to note here is that the __Installer/installdata.xml file is in the game install location - so we already have a path, we just need to know which exe to run from that path! I also came across another method, documented at https://github.com/erri120/GameFinder#origin, which utilises the id string within the .mfst file as part of an API call to an Origin server, to get information about the package; this information does contain the classic registry install path. This may be an alternate option, however it's dependent on calling out to an external server. Attached is an example of the BF1942 folder structure, with the exe location relative to the installerdata.xml and mnfst.txt files. BF1942.zip <https://github.com/terrymacdonald/DisplayMagician/files/7286711/BF1942.zip> This structure seems to be mirrored across all Origin games, including those without a parsable path in the installerdata.xml. I do apologise for not having been able to test the debug build just yet, but I wanted to tackle the manifest lookup first. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#43 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLU5LAWHA7FORQGWJCP7MLUFMEPBANCNFSM5FH6FINQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[terrymacdonald]

@DragRedSim So a bit of time testing and reviewing your extracted directory info, and it looks like the main exe will have a matching .par file. So I've taken the liberty of building some logic that tries that, and if it can't find a par file it will just choose the first exe in that mnsft.txt file. It might work, so I'd love you to test it. Please ignore the previously posted test version, and use this one instead!

DisplayMagicianSetup-v2.0.1.72.zip

Install and give it a go. It may work :)

I'll keep this new code in the mainline codebase until I hear from you if it finds the v2.1 games or not. I'm not too worried about the code as it will be less and less used in the future, and it has a lot of robust error checking around it that should skip the game if there is something odd about it. It means that we'll cope with Origin games with a manifest version from v2.0+ (which is good enough in my opinion).

Please test when you can and we'll adjust the code based on the results of testing.

Thanks
Terry

[terrymacdonald]

Ok - I believe this is fixed in the codebase, so I'm going to close this issue for now. If you still have issues after the release of DisplayMagician v2.1.0 then please feel free to reopen this issue.

[terrymacdonald]

Closing issue as fixed in codebase.

[terrymacdonald]

@DragRedSim The Origin parsing should now be fixed, and the fixed are included in this test build. I'm confident enough to provide you with test version 2.1.0.35 for you to test. It should handle cloned windows, NVIDIA surround, weird layouts (even a combined surround with extra single display works!) and it works after multiple reboots.

I'd really love you to install this version, and to especially test the different displays and that it parses most of your Origin library. I really want you to try and break it :D. I'm hoping 🤞 that it will also work on Origin manifest v2.1 games, but I'm not too worried if it doesn't at this stage. I really want to the v3.0 and 4.0 manifest games to work though.

DisplayMagicianSetup-v2.1.0.35-test.zip

One important thing to note - it will require you to create new Display Profiles as part of the upgrade. It turns out I needed a bit more information than was available in the DisplayProfiles_2.0.json format I created earlier. So this new 2.1 version creates a DisplayProfiles_2.1.json instead.

Hopefully everything works, and then I can release a new DisplayMagician version and move on to adding other features!

Thanks
Terry

[bymagnum]

How do I edit the manifest file?) (mfst)

[bymagnum]

Hi Parvel, The manifest files are files included with the origin launcher, and you shouldn't need to edit them. They are part of that game launcher, and DisplayMagician simply tries to read them to find games. Can you please tell me why you want to edit those manifest files? Thanks Terry
…
On Fri, 8 Dec 2023, 15:45 Pavel Krestyannikov, @.> wrote: How do I edit the manifest file?) (mfst) — Reply to this email directly, view it on GitHub <#43 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLU5LEAFXW77WSLVXCG44LYIJ5NNAVCNFSM5FH6FIN2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBUGY2DMOJVGY2Q . You are receiving this because you modified the open/close state.Message ID: @.>

Hello, friend!

I accidentally came here through a search.

I am an experienced programmer in another field. I have a camera that generates mfst footage. And I need to edit mfst, but it doesn't open for me, wherever I tried to open this file, but edit bytes, it's very bad, I'm looking for an option to edit this file with something, for example python, php, js ... or another language. Is it possible to edit such a file in any language?