jkanderson / soulsformats

A .NET library for reading and writing FromSoftware file formats.

SoulsFormats

A .NET library for reading and writing various FromSoftware file formats, targeting .NET Framework 4.6 and .NET Standard 2.1.
Dark Souls, Demon's Souls, Bloodborne, and Sekiro are the main focus, but other From games may be supported to varying degrees.
A brief description of each supported format can be found in FORMATS.md, with further documentation for some formats.

Usage

Objects for most formats can be created with the static method Read, which accepts either a byte array or a file path. Using a path is preferable as it will be read with a stream, reducing memory consumption.

BND3 bnd = BND3.Read(@"C:\your\path\here.chrbnd");

// or

byte[] bytes = File.ReadAllBytes(@"C:\your\path\here.chrbnd");
BND3 bnd = BND3.Read(bytes);

The Write method can be used to create a new file from an object. If given a path it will be written to that location with a stream, otherwise a byte array will be returned.

bnd.Write(@"C:\your\path\here.chrbnd");

// or

byte[] bytes = bnd.Write();
File.WriteAllBytes(@"C:\your\path\here.chrbnd", bytes);

DCX (compressed files) will be decompressed automatically and the compression type will be remembered and reapplied when writing the file.

BND3 bnd = BND3.Read(@"C:\your\path\here.chrbnd.dcx");
bnd.Write(@"C:\your\path\here.chrbnd.dcx");

The compression type can be changed by either setting the Compression field of the object, or specifying one when calling Write.

BND3 bnd = BND3.Read(@"C:\your\path\here.chrbnd.dcx");
bnd.Write(@"C:\your\path\here.chrbnd", DCX.Type.None);

// or

BND3 bnd = BND3.Read(@"C:\your\path\here.chrbnd.dcx");
bnd.Compression = DCX.Type.None;
bnd.Write(@"C:\your\path\here.chrbnd");

Finally, DCX files can be generically read and written with static methods if necessary. DCX holds no important metadata so they read/write directly to/from byte arrays instead of creating an object.

byte[] bndBytes = DCX.Decompress(@"C:\your\path\here.chrbnd.dcx");
BND3 bnd = BND3.Read(bndBytes);

// or

byte[] dcxBytes = File.ReadAllBytes(@"C:\your\path\here.chrbnd.dcx");
byte[] bndBytes = DCX.Decompress(dcxBytes);
BND3 bnd = BND3.Read(bndBytes);

Writing a new DCX requires a DCX.Type parameter indicating which game it is for. DCX.Decompress has an optional out parameter indicating the detected type which should usually be used instead of specifying your own.

byte[] bndBytes = DCX.Decompress(@"C:\your\path\here.chrbnd.dcx", out DCX.Type type);
DCX.Compress(bndBytes, type, @"C:\your\path\here.chrbnd.dcx");

// or

byte[] bndBytes = DCX.Decompress(@"C:\your\path\here.chrbnd.dcx", out DCX.Type type);
byte[] dcxBytes = DCX.Compress(bndBytes, type);
File.WriteAllBytes(@"C:\your\path\here.chrbnd.dcx", dcxBytes);

Special Thanks

To everyone below, for either creating tools that I learned from, or helping decipher these formats one way or another. Please yell at me on Discord if I missed you.

• albeartron
• Atvaark
• B3LYP
• horkrux
• HotPocketRemix
• katalash
• Lance
• Meowmaritus
• Nyxojaele
• Pav
• SeanP
• thefifthmatt
• Wulf2k
• Yoshimitsu

soulsformats's Issues

MDO Format listed for KF1 is actually from Sword of Moonlight

KF1 doesn't use this file format, it's from the Sword of Moonlight King's Field maker tool which was used to remake KF1 by FromSoft themselves.

I know King's Field formats aren't your primary target, but a few rough specs for them are in my repository 'Psycpros'. Most of the formats are either just standard PsyQ formats or modified variants, like .MIM and .MO which use 'MIMe' routines.

Sorry for being a pain :)

Add descriptors for GX## and parser for GXMD

Hello!

I recently had a problem with DS3 materials and mapped out layout of almost all GX## GXItems. I think, it would like be beneficial if this data was included in SoulsFormats for others to use.

Data sources

All the results and claims I make below are based on Data from DS1, DS2, DS3, Bloodborne, Sekiro, and Elden Ring. I collected the material information (Material + GXLists) of all FLVER files (.flver, or .flv) in those games. All the raw data can be found here.

I want to thank Rayan(Dahoom) for providing me with the Bloodborne data.

The GX## data

The mapping includes:

• The Unk04 for all GX## (where ## are digits). Turns out, that this number is determined by the GXItem ID.
  Correction: Unk04 is unique for all GX## except GX00. For GX00, there exists 100 (used in BB, DS3, SK, ER), 101 (used in SK), and 102 (used in SK, ER). The data format of all these GX00 variants are different. This is why I now believe that Unk04 is a version number for GX## (where ## are digits). If 100, 101, and 102 are read as v1.0, v1.1, and v1.2, then those differences make some sense. This also explains why most Unk04 values are 100 across BB, DS3, SK, and ER.
• The layout and corresponding DS3 debug menu settings of almost all GXItems's data bytes.

The data bytes are especially useful. It's obvious that those bytes are just a list of either int32 or float32 values. Turns out, the number of those values + their type (int or float) is determined by the GXItem ID + Unk04. So I created a mapping of almost all GXItem data values to their corresponding DS3 debug menu entries. This mapping includes:

• The name of the corresponding setting in DS3's Debug menu.
• The type of the value (int32/float32/bool/enum).
• The ranges of the values. If the data type is an int or float, I included min and max bounds if the debug menu had them. So we know that e.g. BlendMaskIntensity is a float between 0 and 100. I included this information so that others can potentially build UI tooling around this data. If the value is an enum, I also included a mapping of all possible values to their names.

DS3, SK, and ER also have GXMD GXItems (items with "GXMD" as their ID). They are rare, and their bytes have a custom format. The bytes essentially contain a map<ID, Value> where ID is an int and Value is either a float, float2, float3, or float5. Other value types are likely also possible, but I only found GXMD with those types. Unfortunately, I don't really understand GXMD. The strange thing about it is that the same ID can have different types of data. E.g. the ID 95 has a float2 value in some files and a float3 value in others. No idea what that is about.

So I don't have a mapping for GXMD, but I did write a parser for it.

Data

Here is a JSON with descriptors for almost all GX## (where ## are digits) items. The descriptors follow this format, so you can use JsonSerializer.Deserialize to parse the JSON.

Here is a JSON list of all Unk04 values I found for GXMD items in DS3, SK, and ER.

Example

This is the descriptor (simplified) for GX03 v1.0:

{
	"ID": "GX03",
	"Unk04": 100,
	"Items": [
		{
			"Name": "[Emissive] EmissiveIntensity",
			"Type": 2,
			"Min": 0,
			"Max": 8
		},
		{
			"Name": "Unknown",
			"Type": 0
		},
		{
			"Name": "Unknown",
			"Type": 0
		},
		{
			"Name": "Unknown",
			"Type": 0
		}
	]
}

This data layout (Items) of the GX item. The data of every GX03 item always contains 4 values. The first value is a float between 0 and 8 that is called "[Emissive] EmissiveIntensity" in the debug menu. The 3 remaining values are unused.

Almost

I said "almost" a few times. That's because a few GX## items are not mapped or only partially mapped.

• I did not find the GXItem values for the settings "[Shadow] DisableCasting" and "[Shadow] CullingMode".
• GX15 is only partially mapped. It controls SSS (sub surface scattering) settings, and messing with them crashed my game, so I stopped investigating them.
• GX16-GX19 are not mapped. DS3 contains no file with these GXItems, so I didn't map them. They likely also control SSS settings.
• GX10 and GX11 are not mapped. Same reason as before, no file contains them.

Apart from that, everything is mapped. All remaining "Unknown" items (example) are unused values. They do not appear in the debug menu and all files in DS3 set them to 0. It's likely that From added a few vacant items for future extensions. I believe that this can already be seen in GX07 where the ScrollSpeeds for Normal3 are suspiciously last.

GXMD format

While the parser also knows the format, I wanted to write it down here. The GXMD bytes have the following format:

The bytes are either int32 or float32 values (meaning that the total number of bytes is a multiple of 4).

The stream of values starts with an int32 that is the number of items. This number is always present. After this, only items follow. Each item start with 2 int32s. The first int is the id of the item. The second int is the data type. Depending on the data type, some number of floats follow. The stream ends after all items have been read. There is no ending 0 or similar.

It's important to note that item ids are unique, but may appear in any order. That's why I said that GXMD data is essentially a map.

Correction: What I said above is correct for DS3. In SK and ER, From extended the GXMD format. Each item can now be optionally followed by a "flag". This flag is a boolean value that is kinda like a second value for that item. While an unflagged item has the format id, dataType, data, a flagged item has the format id, dataType, data, id, dataType=0, data=0|1. I have no idea what those flags do in SK and ER. They are also somewhat rare.

Adding it to SoulsFormats

This data is applicable to Bloodborne, Dark Souls 3, Sekiro, and Elden Ring. DS1 PTDE does not use GXItems. DS2 has a weird GXItem format that I don't really understand, but it seems to be similar. I have not tested DS1 Remastered and Demon Souls.

I do not plan on making a PR, but feel free to ask me anything if you have questions.

All my code and data is licensed under CC0, so take whatever you want.

TAE3.Animation Doesn't have a public constructor

as the title says, TA3.Animation lacks a constructor. Even a copy constructor.
TAE3 itself also lacks a constructor.

Cannot read DSR map HKX files

Firstly, thanks for this awesome set of tools! I'm using this with Dark Souls Remastered. I know that some file formats have changed between DS1 and DSR, but I'm only focusing on map models.

I can successfully load character models and map piece models, but I'm having trouble loading map collision models. Here's what I'm trying:

Logic:

string ROOT_INPUT_FOLDER = "C:\\Program Files (x86)\\Steam\\steamapps\\common\\DARK SOULS REMASTERED\\";
string inputBhdFile = ROOT_INPUT_FOLDER + "map\\m10_02_00_00\\h10_02_00_00.hkxbhd";
string inputBdtFile = ROOT_INPUT_FOLDER + "map\\m10_02_00_00\\h10_02_00_00.hkxbdt";
BXF3 bxf = BXF3.Read(inputBhdFile, inputBdtFile); // this works
BinderFile file = bxf.Files[0]; // file.Name is "m10_02_00_00\\h0000B2A10.hkx.dcx"
byte[] bytes = DCX.Decompress(file.Bytes); // this works
SoulsFormats.HKX hkx = SoulsFormats.HKX.Read(bytes, HKX.HKXVariation.HKXDS1); // this throws InvalidDataException

Output Exception message:
Read UInt32: 0xAC230300 | Expected: 0x57E0E057 | Ending position: 0x4

Incorrect header info when writing EDGE DCX files (Demon's Souls)

The chunk offsets at 0x74, 0x84, etc. are relative to the beginning of the file when they should be relative to the beginning of the compressed data (I think the value at 0x74 is always 0). The resulting files cause the game to hang indefinitely when loading. Subtracting the length of the headers from each of these offset values in a hex editor fixed the load issue entirely.

I actually encountered this using Yabber.DCX, but I'm pretty sure the issue is here.

item.msgbnd.dcx binder files are empty for Magic*.fmg

Firstly, great job, loving the Elden Ring support!

So, most of the binder files inside item.msgbnd.dcx seem fine, but for some reason, MagicName.fmg, MagicInfo.fmg, MagicCaption.fmg are empty?

string gameRoot = "C:\\Program Files (x86)\\Steam\\steamapps\\common\\ELDEN RING\\Game";
BND4 bnd = BND4.Read(gameRoot + "\\msg\\engus\\item.msgbnd.dcx");
foreach (BinderFile binderFile in bnd.Files)
{
    FMG fmg = FMG.Read(binderFile.Bytes);
    Console.WriteLine(binderFile.Name + " has " + fmg.Entries.Count + " entries.");
}

Output:

N:\GR\data\INTERROOT_win64\msg\engUS\GoodsName.fmg has 4207 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\WeaponName.fmg has 3142 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\ProtectorName.fmg has 629 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\AccessoryName.fmg has 764 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\MagicName.fmg has 1 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\NpcName.fmg has 531 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\PlaceName.fmg has 1045 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\GoodsInfo.fmg has 4215 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\WeaponInfo.fmg has 39 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\ProtectorInfo.fmg has 625 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\AccessoryInfo.fmg has 764 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\GoodsCaption.fmg has 4225 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\WeaponCaption.fmg has 3143 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\ProtectorCaption.fmg has 629 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\AccessoryCaption.fmg has 764 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\MagicInfo.fmg has 1 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\MagicCaption.fmg has 1 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\GemName.fmg has 263 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\GemInfo.fmg has 252 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\GemCaption.fmg has 252 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\GoodsDialog.fmg has 122 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\ArtsName.fmg has 201 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\ArtsCaption.fmg has 201 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\WeaponEffect.fmg has 83 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\GemEffect.fmg has 1 entries.
N:\GR\data\INTERROOT_win64\msg\engUS\GoodsInfo2.fmg has 1081 entries.

Make SetValue(ReadOnlySpan<char>) function for PARAM.Cell

I propose to add new function to PARAM.Cell class that takes ReadOnlySpan as first argument and it converts it to type of its parafdef and assigns it to value property just as setter of Value does.
I don't want to make allocations (new string(...)) just to set cell to a value so I can relief some pressure from GC and yield better performance.
If this proposal is accepted I can make a PR with this change included.

System.IO.InvalidDataException:“Read ASCII: ?:? | Expected ASCII: BDF4”

I will export the models from “Sekiro”：
The folder ：

I have run this code

BXF4 bxf = BXF4.Read(@"E:\Sekiro\data2.bhd", @"E:\Sekiro\data2.bdt");

Its give me Exception :

How can I fix that?

Elden Ring support

Any plans to add support for Elden Ring? At minimum, maybe just add comments in the README and FORMATS. For example, "some formats appear to be the same as DS3, but not planning to enhance this anytime soon." ?