makiah / botw-sbfres-to-fbx

⚔ A repo which employs a number of useful libraries I found across the net along with some custom-written Python and MAXScript to facilitate batch export of Nintendo SBFRES compressed files to FBX files (for use in Unity, for instance). DISCLAIMER: I do not condone the unauthorized replication of files provided by employing this script!

Automated SBFRES Extractor for Zelda BotW

Disclaimer

Avoid distributing the resulting models from implementing this tool, as they are copyrighted by Nintendo and you will be subject to such laws. Personal use ONLY!

Proof of Concept

Initial Setup

First, you'll need the models found by extracting your Wii U disc's content. This can be done with a number of existent tools, so avoid torrenting the disc image (support this awesome game peeps!).
Then, clone this repository in SourceTree or extract the downloaded ZIP archive. Then run cd /extracted/project/copy, and finally, python3 RunMe.py. This should take care of everything, and after about a day or so (it'll probably take that long), you can open the FBX files in fbxextraction/Database. For a more in-depth analysis of how it works, look at the next section.

Functionality

Note: this tool will only work on Windows because of limited platform support for certain tools.

For each section, you'll need Python 3 so I suggest installing that first.

PACK Extraction

Searches through the entirety of the game directories and finds PACK files, then extracts them to a directory in the same path.

Thanks to: NWPlayer123 for SARCExtract.py.

Additional requirements: Python 2.7.

SBFRES grouping

Super simple, just takes all discovered SBFRES files (including extracted ones) and moves them to a Compilation subdirectory.

Additional requirements: None.

Texture Extraction

A bit more complex: uses yaz0dec to extract the model and tex1 sbfres files, and then uses an assortment of libraries to extract and convert the textures to PNG format.

Thanks to: Random Talking Bush from the VG Resource for his QuickBMS scripts, NWPlayer123 for TexConv2, and the creators of ImageMagick.

Additional requirements: None (everything included in Libraries).

Model Extraction

Uses RTB's BFRES extraction MAXScript via 3DS Max to render the model and load the previously extracted textures, then uses some MAXScript I wrote to automate the process via a startup script argument.

Please note for this section: models may appear disfigured or malformed in the Windows viewer for some reason, but they seem to typically render fine in a game engine.

You'll probably have to coax the extraction through DgnObj items, since they're often forms this script can't handle (just close 3DSMax on error and the script will ignore this section and proceed onward).

Thanks to: Random Talking Bush for the extraction script.

Additional requirements: 3DS Max 2015/16.

Animation Extraction (WiP)

Extracts Maya animations (.anim files) in each animation sbfres via an automated fork of Smash Forge, alongside the corresponding .Animation.sbfres in one database. Then, takes the completed models from fbxextraction and loads them in Maya, applies an animation to them, and exports the skeleton to a different set of database folders. I couldn't figure out how to embed multiple tracks into a single FBX file that was readable by Unreal or Unity without doing frame splitting, and this was the next best thing.

Thanks to: KillzXGaming for his terrific work on Smash Forge that enabled animation extraction.

Additional requirements: Maya 2016.

botw-sbfres-to-fbx's Issues

Crashing in sveral different point

The option 4 for BFRES extraction is always crashing for me after a few minutes. In several different points too, i've seen so far:

• when "Calling Libraries\szstols\yaz0dec.exe", with yaz0dec.exe getting stuck in a way Task Manager can't end the process

• when "Calling convertGTX.bat", with TexConv2.exe getting stuck in a way Task Manager can't end the process

• when "Calling convertPGN.bat", with magick.exe getting stuck in a way Task Manager can't end the process

Edit: just adding that the crashes are random. Whenever they happen, i just clean the Database folder and start again, and it will keep going. I'm actually going forward with it partially, moving the BRFES to another foward and cleaning the SBFRES from the Compilation folder!

I'm on a Windows 7 setup with python 3.7.2 trying to do a full models rip. cheers!

Can't understand the readeam for my situation

What I'm doing is moding a mod . So yea I'd like some clarification cause I'm new to the moding comunity (sorry if this doesnt belong in the isues tab)

Script Makes Duplicate FBX's

Script works well for what I need, but there seems to be a flaw right now where if the Bfres file creates multiple fbx files they will all simply be duplicates of the first instead of what they are supposed to be.

For example, when I open up the armor files in a folder, they all are just the head armor.

I can simply use the import script manually, but I don't know what the script does to the file to prep it for fbx.

UnicodeDecodeError: 'gbk' codec can't decode byte 0xd7 in position 1580: illegal multibyte sequence

Phase 4: Model Exports
Traceback (most recent call last):
File "runme.py", line 66, in
LoadAndExportExtractedModels.loadAndExportModels()
File "E:\BotW-SBFRES-to-FBX\fbxextraction\LoadAndExportExtractedModels.py", line 48, in loadAndExportModels
data = file.readlines()
UnicodeDecodeError: 'gbk' codec can't decode byte 0xd7 in position 1580: illegal multibyte sequence

Possible normal inversion

Not sure why this is happening but it explains a couple rendering issues:

Scrolling through the meshes in the extracted Max results in this nightmarish appearance of Link's face, in which the texture behind the forefront texture is visible in front of the face texture. This is how Windows renders the exported model as well.

After a bit of digging I've determined it also happens for meshes in Unreal, looking at the bear model's face results in transparency through the muzzle. From what I can tell this is caused by normal inversion, but checking "flip normals" in Max didn't seem to have an effect. Perhaps this is a GTX conversion mishap?

Possibly roll back MAXScript to R4?

Exporting the result of the R4 script load currently gives out a textured model when viewed in Windows/Unreal, but R6 warns that "Color Correction" is not supported outside of Max and then viewed models are textured grey by default for certain textures. Perhaps roll back to R4 in the event that R6 has no additionally useful features?

#5 issue

When ever i get to animation extraction, it just crashes.

FldObj terrain textures missing for some exports

I think this might be rectifiable by exporting Terrain.Tex1.sbfres and making it accessible to RTB's script.

Recommendation: Stick to FBX for primary model output

Hey again! Just want to express an opinion of mine about the move from FBX exporting to OBJ exporting you're working on, and while it might be a good thing to have another format to convert to, I would not recommend that you replace the FBX functionality altogether, as there are several instances in which using OBJ instead of FBX would detract from the experience of the end user.

1. Lack of hierarchy support: due to the OBJ format lacking 1st party support for skinning or mesh deformation (among other things), exporting in OBJ format will cause there to be no skeleton data for the models. Most modelers would want this feature retained for things like animation, so keep it in mind.
   AND
2. Lack of support for normal, specular, and bump maps: these are often what give BOTW's models their characteristic appearance and are needed to have truly accurate rips of said models.

If you are really bent on including this functionality, then include it in a separate branch or add said functionality alongside the already existing FBX exporting capability; not replacing FBX exporting outright, as FBX supports more features than the more limited (but more accessible) OBJ specification.

Hope this helps!

Import Animations for Main Character

Hi there,

I'm trying to import animations for the main character. I tried exporting the Link.bfres in 3DS Max to FBX and then importing the file in Maya. It works fine, I can see the skeleton and the skin.

Next I tried importing an .anim file I got from Smash Forge. Here the character collapses inside and plays a weird animation.

Doing the same procedure with much more simpler characters (e.g. Frog) does work and plays fine.

Has anyone gotten the main character work with animations?
Any help is welcome.

Current BFRES extractor is obsolete

It's been a while since I've had time to push to this repo, but the extractor used by this project (developed here) hasn't been updated in a while, although it seems that RandomTalkingBush has made changes. Might take a while to fix.

Merge object mesh

I'm noticing that a consistent issue within Unreal and 3rd party model viewers is that tiny objects appear at the root position of the scene. I'm going to attempt to merge all models into a single mesh from within the MAXscript to rectify this.

could not export fbx model in DLC

Not working with 3ds Max 2019

It seems that the Python extraction script doesn't work with 3ds Max 2019.
When executing the MAXScript it throws the error "-- Unable to convert: undefined to type: Integer64"
This happens everytime the CompleteAutomation.py calls the MAXScript.

Add support for files that have both Tex1 and Tex2 image sbfres and files that have only Tex2

There are certain models in The Legend of Zelda: Breath of the Wild that use more than one texture archive in the form of Tex1 and Tex2. There are also other models in said game that use just the Tex2 archive strangely enough. So if this project is still being developed for, can this functionality please be added? It would be very helpful as extracting the models from this game is time-consuming as it is. If you need samples to work with, I can provide a few models if necessary.

Thanks for any efforts in advance.
Jako

3ds max error

When I try to load your ms script to 3ds max, it gives me "Unable to convert undefined to type: Integer 64"

Armor models are duplicates

This is actually a big issue which I somehow just noticed: all of the models of the armor objects are the exact same model, definitely an issue with the edited MAXscript.

Textures not exporting

The texture extractor isn't outputting/extracting any texture files. The command window states that it can't find the texture, despite that it seems to be otherwise exporting the meshes fine from what I've seen so far.
Just a quick picture of the command window when exporting bfres models to fbx:

(I have a switch version of the romfs, for reference, if that matters.)

Found texture is True and found bfres is False

Upon phase 4 (Model exports) i just get "Found texture is True and found bfres is False"
Textures export just fine, but models dont