prismaticflower / swbf-unmunge Goto Github PK

Current code in master assumes that material information is always stored in a struct of constant size. I now know this to be incorrect. A more thorough investigation into the format across both games is needed to truly fix this. For now in the abstraction branch I will add a primitive check to prevent crashing.

After I've finished refactoring the codebase this will most likely be my next project.

Title explains itself, please consider porting to Linux, it would be greatly helpful.

I haven't taken the time to look closely at it yet but it is most likely being caused by subtle differences in format between the games.

Hopefully only minor changes are needed to fix it. Also it would be cool to add a way for the output .ter file to be in the format used by SWBF1's Zero Editor. That will most likely require adding an -output_format switch or something though.

Currently scripts are simply extracted into munged .SCRIPT files. It would be great to be able to decompile these into raw Lua scripts.

Was chatting with AnthonyBF2 on HUD last night and he pointed out that the HUDs are not unmunging totally correctly.

Looks like I missed this one when adding support for multiple level of details in models. get_model_name in handle_model.cpp will need updating along with msh_builder and it's lod handling.

Looking at BF2 sound\shell.lvl (and BF1 too)
The 1st 2 arguments to 'Layer' unmunge as large numbers instead of strings.

original (shell.snd)

SoundLayered()
{
    Name("shell_transition2");
    Layer("shell_transition2_left",  "shell_transition2_left", 0.0, 0.5);
    Layer("shell_transition2_right", "shell_transition2_right", 0.0, 0.5);
}

unmunged:

soundlayered()
{
	Name("shell_transition2");
	Layer(76831842304, 76831842304, 0, 0.500000);
	Layer(-494586690535424, -494586690535424, 0, 0.500000);
}

bes2_detail_1.tga should be bes2_detail_1.png
Not a huge issue, but its there.

Side note, I've been getting quite the mileage out of the latest pre release gltf features

From the report access violation is causing it. Which is strange because if the tool was going to cause an access violation I would expect it to do that regardless of OS. Maybe it isn't the OS doing it, I don't have Windows 7 handy to find out.

A possible candidate for the cause would be the refracted File_saver causing it.

Currently they're completely ignored. I haven't looked into the audio format at all but I know the config files for the sound just go right through configmunge. (Though it uses -hashstrings so the files would always need manual editing afterwards.)

I believe others in the past have extracted audio files from the game it shouldn't be too hard to learn the format the game stores them in.

Hey just wanted to notice you that the newest version of the tool was no longer working for W7 (1.2.1 work)

I know well enough that W7 is no longer supported by microsoft and all that stuff but BF1 is an old game and it run better on older OS
If you don't have the time to look into this i perfectly understand, but if it's possible to fix it that would be awsome

Thx for your work

Currently I (poorly) use the bounding box from the munged information. It would be nice to instead generate a bounding box for a msh file based off the extent of the visible geometry, this should give better results when using the model in Zero Editor.

For example, this:

[WeaponClass]

ClassLabel = "com_weap_inf_smg_n7hurricane"
GeometryName = "m9_tempest.msh"

[Properties]

HUDTag = "m9_tempest"
GeometryName = "m9_tempest"
HighResGeometry = "m9_tempest"
OrdnanceName = "ssv_weap_inf_smg_n7hurricane_ord"

should be this:

[WeaponClass]

ClassParent = "com_weap_inf_smg_n7hurricane"
GeometryName = "m9_tempest.msh"

[Properties]

HUDTag = "m9_tempest"
GeometryName = "m9_tempest"
HighResGeometry = "m9_tempest"
OrdnanceName = "ssv_weap_inf_smg_n7hurricane_ord"

Crashes on (PC worlds):[cor1.lvl, end1.lvl, geo1.lvl, mus1.lvl, nab2.lvl, pol1.lvl, spa1.lvl, spa2.lvl, spa3.lvl, spa4.lvl,
spa8.lvl, spa9.lvl, spa_sky.lvl, tan1.lvl, uta1.lvl]

Hangs on (PC worlds):[tat3.lvl ]

Crashes on (PC):[core.lvl, ingame.lvl]
Crashes on (PC Side):[gun.lvl, jed.lvl, tur.lvl]

I looked into these a little and don't yet understand why all the crashing happens.
But I do think these crashes should get fixed before new features come in.

Suspecting it's probably something in the art assets, an unexpected compression or something of the sort (but who knows.) The crash is not limited to one single .lvl file, but the most recent one I attempted to open was actually a side .lvl from a PS2 beta build for BF2. Reason why I'm blaming the art assets is because there are half-extracted textures, possibly models too that remain broken and unviewable in the output folder. At the moment I'm not too worried about the art, but I'd like to retrieve and examine the .odf portions of some classes as they were set up during development, before getting settled into the final BF2 setting.
This isn't a terribly important thing and therefore doesn't warrant any hurry if there's no time available to spare on this tool, I simply just like seeing how things were during games' development histories. Still, it would be cool if the PS2 .lvl files (and its beta build's, if they're even any different) could be fully extracted. Or even just the code sections, really. Whichever is easier to make happen.

"If you for some reason do want to build it on Linux or something feel free to get in touch I am happy to help point out what bits of the codebase are non-portable and what could be done."

Soooo...what bits of the codebase are non-portable and what can be done?

If it's relevant what my system is:

Distro --> Ubuntu 18.04.1 LTS
Kernel --> 4.15.0-33-generic
Graphics --> Nvidia GTX 750TI with 396.51
gcc --> version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)

It might be a good idea (albeit a ambitious one) to rewrite targeting webassemly, making the tool work in the browser, and able to be used as a module instead of standalone, which would make for some interesting browser modding tools. Then we could compile on any platform with any browsers supporting wasm (and even remote compilation) with no need for Windows.

I'm interested in doing this, but I'm not sure I can handle it on my own.

Originally posted by @RepComm in #14 (comment)

Send me an email if you want me to dig through the levelpack/munge source code and tell you what it's doing. I reply to every email that doesn't get caught by the junk filter. -- [email protected]

I also have the BF1 shaders I haven't released yet but am working on it as part of bigger project to create a BF1 master branch that mirrors the original dev environment once I sort out what the final files looked like. I have at least two branches of my own planned for it, one that includes an HD remaster using the red shell with updated movies, shaders and lots of corrections that I've been working on. I also have the original BF1 fonteditor I was going to release with it.

A lot of that we didn't release because of disk space and constraints of the addons. I'd like to release everything, but if I can sort out an installer that pulls from the lvls and uses the assets to recreate folders and extract the files rather than download them I figure I might not have to, so I was looking into your tool or possibly creating my own. One thing I didn't keep was the international VO, so I'm hoping I can find an iso and a way to extract those to make the dev environment complete.

Thanks for all your hard work, it's great and makes it possible for me to legally release things people have figured out how to extract. If there's anything I can do to assist let me know.

Barriers are extracted with incorrect rotation, the function get_barrier_corners in handle_world.cpp is responsible. Not the most important thing to get correct as barriers are easily recreated but it would be nice to fix.

The SWBF1 munger applies a transformation to the rotation of bones that SWBFII doesn't. (Or something.) I spent a good couple hours trying to work it but math isn't my strong point and I was just burning myself out on it. I may take another crack at it after a break.

Also the z-coordinate of the position is flipped for the SWBF1 bones, but that was easy to figure out.

The information appears to be stored in a chunk separate from the main planning info. This would mean having to pass about a "planning builder" of some sort to properly recover it, given the increase code complexity I don't think it is worth it.

That said it could be neat to at least recover information about which connections are meant to be dynamic and dump it into a text for the user.

Currently only their geometry, colour, texture settings, water settings are extracted.

To correctly extract everything it requires more study of both the terrain format Zero Editor uses and the terrain format SWBF's engine uses. A fairly nontrivial task but it would be nice to get done at some point.

This isn't an actual issue, but I thought it would be useful for the people wanting to run on linux.

This runs successfully on Ubuntu x86 64bit gnome desktop using steam proton 6.3-5
I don't know if graphics drivers need to be specific (probably not), but I'm using Nvidia server driver 460.

Configuration:

• Requires steam app for linux installed
• Requires adding Proton 6.3-5 : Navigate to steam Library in the app, search for Proton 6.3-5 and install it / wait for download to finish
• Download latest SWBF unmunge binaries, extract them to some folder (mine is in ~/Downloads/swbf-unmunge)
• Create a .desktop file so you can import into steam later:

I named mine swbf-unmunge.desktop, heres the content:

[Desktop Entry]
Name=swbf-unmunge
Comment=unmunge
Exec=/home/jon/Downloads/swbf-unmunge/swbf-unmunge.exe -file /home/jon/Downloads/swbf-unmunge/file.lvl -version swbf -outversion swbf -imgfmt png -platform pc -verbose -mode extract
Icon=steam
Terminal=true
Type=Application

You need to change the line Exec= to the location of your installed swbf-unmunge.exe path
After the exe comes the arguments for the file you want to run it on. You'll need to follow the swbf-unmunge readme for the proper arguments, as well as point to the proper file (I just rename whatever LVL like rep.lvl to file.lvl for convenience).

Add the swbf-unmunge.desktop file to steam:

• Go to steam library
• Bottom left corner: Add Game
• "Add non-steam game"
• "Browse..." and find your swbf-unmunge.desktop file
• "Add selected programs"
• Click on the newly added app in your library
• Open manager by clicking on gear icon on far right
• Open "Properties"
• Click on Compatibility
• Check "Force use of specific steam play compat tool"
• Select the proton version you installed earlier

Now you can start the app and it will execute the command line options you added.
If you want to edit the command line options you can now set them in the app's properties in the shortcut tab, where it is labelled as Launch options.

Example: stock Yav1.lvl on Xbox will give "Unknown VBUF type encountered", and the beta version gives "Chunk magic number mismatch"
We determined before that BF1 doesn't really care too much about strictly following its guidelines on the magic number of 81 (and will still load ingame), so maybe something a little more flexible for VBUF_type::geometry could be used for BF1. I have very little knowledge of how the TER file handles this, so suppressing the errors just gives a seemingly blank terrain.

Feel free to reply with a report on any file that fails to be extracted in someway. Crashes, broken output, etc. Include what's broken about the extraction and if the file isn't in the base game a link to it.

List