aers / ffxivclientstructs Goto Github PK
View Code? Open in Web Editor NEWResources for reverse-engineering the FFXIV client's native classes.
License: MIT License
Resources for reverse-engineering the FFXIV client's native classes.
License: MIT License
Status like ruin4 now always show their stack count as 0. But their status.pram shows their correct stacks. Is this a bug?
ffxiv_idarename.py> Running...
File "W:\bullshit\ghidra\ffxiv\FFXIVClientStructs\ida\ffxiv_idarename.py", line 619
def recurse_tree(current_node: Node, current_class: FfxivClass, all_bases: set[str]):
^
SyntaxError: mismatched input ':' expecting RPAREN
ffxiv_idarename.py> Finished!
Seems to be this commit breaking it - type hinting isn't in py2, which Ghidra uses.
Current offset exceeded the next field's offset (0x228 > 0x1E0): Client::Graphics::Render::RenderTargetManager.OffscreenRenderTarget_1
Current offset exceeded the next field's offset (0x18E8 > 0x18C8): Client::UI::AddonWeeklyBingo.SecondChanceButton
Current offset exceeded the next field's offset (0x8 > 0x1): Client::Game::Character::Character::ForayInfo.Element
Current offset exceeded the next field's offset (0x4280 > 0x3E90): Client::UI::Agent::AgentMap.MiniMapMarkerArray
Current offset exceeded the next field's offset (0x8F8 > 0x8F1): Client::Graphics::Scene::Human.Sex
Current offset exceeded the next field's offset (0x1A0 > 0x198): Client::Game::UI::Buddy.Pet
Unhandled type: FFXIVClientStructs.FFXIV.Client.System.Memory.ICreatable
I think the ForayInfo is an issue in how the program calculates offsets.
WeeklyBingo has buttons in the wrong struct i think? but the offsets need to be recalculated.
I'm not sure whats going on with BuddyPet.
It was recently discovered that AtkTextNode stores the pointer to the original text buffer used when calling SetText. The string overload generator handles converting C# strings (UTF-16) to strings the game expects (generally, null-terminated UTF-8). It relies on the fact that the game doesn't take ownership of the string buffer passed to the native function and temporarily allocates a buffer to store the converted UTF-8 string in, usually on the stack. The game then copies the string into its own objects and the string overload method can allow the buffer to be freed. Since AtkTextNode stores a pointer to the original text buffer, this results in the node storing a pointer to the stack.
While people have been using this function for years with no visible issues, it is incredibly unsafe for us to be storing a stack pointer in a native game object like this, so we can't provide string overloads for SetText anymore.
If you use SetText you will need to allocate the UTF-8 string buffer yourself and free it when your plugin is done using the TextNode.
The sample code below shows how to allocate and pass the buffer. It uses the same methods ClientStructs' string overloads uses to convert a string to UTF8.
byte* strBuffer;
unsafe void SetText(AtkTextNode* node, string text)
{
if (strBuffer != null) // free buffer if you've already got one
System.Runtime.InteropServices.NativeMemory.Free(strBuffer);
int strLen = System.Text.Encoding.UTF8.GetByteCount(text); // get length of string as UTF-8 bytes
strBuffer = System.Runtime.InteropServices.NativeMemory.Alloc(strLen + 1); // need one extra byte for the null terminator
Span<byte> bufferSpan = new(strBuffer, strLen + 1); // wrap buffer in a span so you can use GetBytes
System.Text.Encoding.UTF8.GetBytes(text, bufferSpan); // convert string to UTF-8 and store in your buffer
bufferSpan[strLen] = 0; // add null terminator to the end of your string
node->SetText(strBuffer);
}
You will also need to call Free
when you are no longer using the TextNode (if you're destroying it, or your plugin is unloading, for example). You can also use other C# allocation functions (AllocHGlobal) or the game's native UI allocator (via IMemorySpace) if you want.
You may be worried about allocating a new buffer every time you set the text. First - don't prematurely optimize. That said, the way to avoid this is to re-use the buffer.
The code examples below weren't directly tested, likely have typos, and probably need some more work to be complete.
byte* strBuffer;
int strBufferLen;
unsafe void SetText(AtkTextNode* node, string text)
{
int strLen = System.Text.Encoding.UTF8.GetByteCount(text); // get length of string as UTF-8 bytes
if (strBuffer == null || strLen + 1 > strBufferLen) // reallocate buffer if it doesn't already exist or is too small
{
System.Runtime.InteropServices.NativeMemory.Free(strBuffer);
strBuffer = System.Runtime.InteropServices.NativeMemory.Alloc(strLen + 1); // need one extra byte for the null terminator
strBufferLen = strLen + 1;
}
Span<byte> bufferSpan = new(strBuffer, strLen + 1); // wrap buffer in a span so you can use GetBytes
System.Text.Encoding.UTF8.GetBytes(text, bufferSpan); // convert string to UTF-8 and store in your buffer
bufferSpan[strLen] = 0; // add null terminator to the end of your string
node->SetText(strBuffer);
}
You could wrap this logic in a class:
unsafe class AtkTextNodeBufferWrapper
{
private byte* strBuffer;
private int bufferLen;
public byte* GetBuffer => strBuffer;
public void SetBuffer(string text)
{
int strLen = System.Text.Encoding.UTF8.GetByteCount(text); // get length of string as UTF-8 bytes
if (strBuffer == null || strLen + 1 > strBufferLen) // reallocate buffer if it doesn't already exist or is too small
{
System.Runtime.InteropServices.NativeMemory.Free(strBuffer);
strBuffer = System.Runtime.InteropServices.NativeMemory.Alloc(strLen + 1); // need one extra byte for the null terminator
strBufferLen = strLen + 1;
}
Span<byte> bufferSpan = new(strBuffer, strLen + 1); // wrap buffer in a span so you can use GetBytes
System.Text.Encoding.UTF8.GetBytes(text, bufferSpan); // convert string to UTF-8 and store in your buffer
bufferSpan[strLen] = 0; // add null terminator to the end of your string
}
public void FreeBuffer()
{
System.Runtime.InteropServices.NativeMemory.Free(strBuffer);
bufferLen = 0;
}
}
Also consider pre-allocating a buffer length you know is likely to store the longest text you're setting, especially if its a small amount of text.
Another option is to use the game's Utf8String
class. This class has internal storage for a string of length 64 or smaller and won't reallocate unless your strings are larger.
Utf8String.FromString(str)
allows you to allocate an unmanaged Utf8String*
which you can then save. You can update the string text with SetString
. The Utf8String's StringPtr
can be safely passed to SetText.
PulseActionBarSlot
Crashed the game!
I don't know the reverse engineering. And I guess this is because the index of vfunc
has changed.
FFXIVClientStructs/ida/sigdata.json
Line 9459 in 1f07ec0
I joint the fate in game, but FateManager.Instance()->FateJoined
is 0
. And the FateManager.Instance()->CurrentFate
is Zero
.
AtkTextNode->AtkResNode
for example. This generator would need info from all other generators so first combine all generators into a single generator (relatively easy) beforehand.Anything I'm forgetting. Feel like we've talked about other stuff before.
I feel like some way to indicate which structs have been 'validated' after a patch (anything that didnt change size is probably safe, and anything that changed size and has had its offsets verified) would be nice. There's no doubt numerous "broken" (outdated) offsets and having some way of tracking that would be useful.
https://github.com/xivapi/ffxiv-datamining/blob/master/csv/Status.csv
At the very least, its better terminology for buffs AND debuffs.
I realize this is a bit OT, but I am suddenly unable to attach ida to the running game after the 6.1 patch. I was wondering if you have experienced this issue and, if so, have you discovered what anti-debug method they are using.
Is BalloonType uint or byte?
Current offset exceeded the next field's offset (0xEC > 0xE9): Client::UI::Agent::BalloonInfo.byte
namespace FFXIVClientStructs.FFXIV.Client.Game;
public enum BalloonType : uint
{
Timer = 0, // runs on a simple timer and disappears when the timer ends
Unknown = 1 // the non-timer mode, not sure what its called or where its used
}
namespace FFXIVClientStructs.FFXIV.Client.Game;
public struct Balloon
{
...
[FieldOffset(0x8)] public BalloonType Type; // <- uint usage?
[FieldOffset(0xC)] public BalloonState State;
...
}
namespace FFXIVClientStructs.FFXIV.Client.UI.Agent;
public unsafe struct BalloonInfo
{
...
[FieldOffset(0xE4)] public int BalloonId; // <- byte usage?
[FieldOffset(0xE8)] public BalloonType Type;
...
}
For structs with explicit set struct size, there is no error for fields that esceed the struct size
PS: I vaguely recall some discussion about this, but as I did not find any written info on this I'll open an issue for either fixing this or making a note in a readme
Was playing around with Raycasts a bit and it looks like some offsets have been changed:
Distance
is at 0x48
nowObject
is at 0x50
Flags
is at 0x40
Opened this issue so someone with better knowledge of the struct can take a look cause I don't really know how the Flags or the Object should look like.
This could be added to the README.md pip dependencies I think
Hey @wolfcomp,
as far as I understand, excel definitions are now maintained in EXDSchema, but ffxiv-exdgetters.py still uses Saint Coinach definitions.
Please update the script when you have some time.
Thank you very much! ๐โโ๏ธ
namespace FFXIVClientStructs.FFXIV.Client.UI;
public struct AddonContextMenu
{
[FieldOffset(0x0)] public AtkUnitBase AtkUnitBase;
[FieldOffset(0x160)] public unsafe AtkValue* AtkValues; // Does this belong in base?
[FieldOffset(0x1CA)] public ushort AtkValuesCount; // // Does this belong in base?
}
It looks like the c structs are a bit out of date with the csharp source, so I tried building/running your CExporter and got the following:
PS C:\Users\miked\Source\repos\FFXIVClientStructs\ida\CExporter\bin\Debug\net5.0> ./CExporter.exe Unhandled exception. System.ArgumentException: Field passed in is not a marshaled member of the type 'FFXIVClientStructs.FFXIV.Client.UI.AddonEnemyList'. (Parameter 'fieldName') at System.Runtime.InteropServices.Marshal.OffsetOf(Type t, String fieldName) at CExporter.FieldInfoExtensions.GetFieldOffset(FieldInfo finfo) in C:\Users\miked\Source\repos\FFXIVClientStructs\ida\CExporter\Program.cs:line 424 at CExporter.Exporter.<>c.<ProcessStruct>b__10_1(FieldInfo finfo) in C:\Users\miked\Source\repos\FFXIVClientStructs\ida\CExporter\Program.cs:line 145 at System.Linq.EnumerableSorter
2.ComputeKeys(TElement[] elements, Int32 count)
at System.Linq.EnumerableSorter1.ComputeMap(TElement[] elements, Int32 count) at System.Linq.EnumerableSorter
1.Sort(TElement[] elements, Int32 count)
at System.Linq.OrderedEnumerable1.GetEnumerator()+MoveNext() at System.Linq.Lookup
2.Create(IEnumerable1 source, Func
2 keySelector, IEqualityComparer1 comparer) at System.Linq.GroupedEnumerable
2.GetEnumerator()
at CExporter.Exporter.ProcessStruct(Type type, StringBuilder header) in C:\Users\miked\Source\repos\FFXIVClientStructs\ida\CExporter\Program.cs:line 147
at CExporter.Exporter.ProcessType(Type type, StringBuilder header) in C:\Users\miked\Source\repos\FFXIVClientStructs\ida\CExporter\Program.cs:line 105
at CExporter.Exporter.Export(GapStrategy strategy) in C:\Users\miked\Source\repos\FFXIVClientStructs\ida\CExporter\Program.cs:line 84
at CExporter.Program.Main(String[] _) in C:\Users\miked\Source\repos\FFXIVClientStructs\ida\CExporter\Program.cs:line 19`
namespace FFXIVClientStructs.FFXIV.Client.UI.Agent;
public unsafe partial struct AgentMap
{
....
[FieldOffset(0x3AA0)] public fixed byte UnkArray2[0xA8 * 12]; // <-- until 0x4280, exceeds MiniMapMarker
[FieldOffset(0x3E90)] public fixed byte MiniMapMarkerArray[0x40 * 100]; // 100 * MiniMapMarker
....
}
namespace FFXIVClientStructs.FFXIV.Component.GUI;
public unsafe partial struct AtkComponentNumericInput
{
....
[FieldOffset(0x0)] public AtkComponentInputBase AtkComponentInputBase; // Size 0xF0
[FieldOffset(0xC8)] public AtkTextNode* AtkTextNode; // Does this belong inside the base?
....
}
do you know if theres data somewhere in the window to know how many in the Index are The non click headers? like are there no collision nodes for them, and if so is there a way to determine what there specific indexes are? like maybe an array of non collision headers and their indexes? is that possible?
Not sure if this is the right place, but I'd like to know if any of you FFXIV reversing experts would be willing to help me understand it's software architecture a bit more. Particularly, I'd love to have a better understanding the role of Agents and how they work, how the FFXIV UI system works, how the UI retrieves the data it displays from the server, and how to manipulate the controls on the UI. Even a basic interaction diagram of those components would be very helpful.
If this information already exists somewhere, then a link to where I can find it could be appreciated. Otherwise, if you don't want to have the discussion here, feel free to email me at [email protected].
Thanks!
I want to create structures for the FFXIV Korean version.
Can you explain how to create a data.yml to use in IDA?
I know how much you love updating your diff scripts. This seems sane enough, although if you want it in json instead, i can make that happen. The idea is simple enough, take the actual data out of the py script into a yml file.
version: 2020.12.29.0000.0000
globals:
0x14169A2A0: g_HUDScaleTable
functions:
0x140059670: FFXIVString_ctor # empty string ctor
classes:
Client::Game::Control::TargetSystem::ListFeeder:
GroupManager:
funcs:
0x140BB2600: Create
Common::Configuration::ConfigBase:
inherits_from: Client::System::Common::NonCopyable
vtbl: 0x14164E260
funcs:
0x140068C30: ctor
Common::Configuration::UIConfig:
inherits_from: Common::Configuration::ConfigBase
vtbl: 0x14164E280
Client::System::Framework::Framework:
vtbl: 0x14164F4B8
vfuncs:
1: Setup
funcs:
0x14008EA40: ctor
With 603b211 I added versioning to Client Structs, but I failed to realize GitVersion fails to work with detached heads which makes building anything that isn't fully updated with Client Structs fails to build.
I'd like to keep versioning as I believe it will be helpful, but I'm unsure of a better way to do it.
Any ideas?
In FFXIVClientStructs.FFXIV.Client.Game.Status
, the fields StackCount
and Param
are actually one combined field, at least in the case of food and potions.
The u16
starting at offset 2
contains the ItemFood
row for consumed food and potion status effects (i.e. "Well Fed" and "Medicated"). I imagine for status effects with stacks, it's conceivable that it's actually just a u16
.
Hi, you're probably still working on updating FFXIVClientStructs but just wanted to let you know this enum seems to be off by one now.
My plugin code which is now hitting BGM instead of master.
/// <summary>
/// Mutes/Unmutes the sound
/// </summary>
/// <param name="enabled"></param>
public static unsafe void SetMasterSoundEnable(bool enabled)
{
Framework.Instance()->SystemConfig.CommonSystemConfig.ConfigBase.ConfigEntry[(int)ConfigOption.IsSndMaster].SetValueUInt((uint)(enabled ? 0 : 1));
}
public static unsafe bool GetMasterSoundEnable()
{
if (Framework.Instance()->SystemConfig.CommonSystemConfig.ConfigBase.ConfigEntry[(int)ConfigOption.IsSndMaster].Value.UInt == 0)
PluginLog.Debug("Enabled");
return (Framework.Instance()->SystemConfig.CommonSystemConfig.ConfigBase.ConfigEntry[(int)ConfigOption.IsSndMaster].Value.UInt == 0);
}
I seem to be unable to build the project. I get the following errors (there's over 2500 of them, I'm only including a few):
AtkUnitBase.cs(15, 65): [CS0246] The type or namespace name 'FixedSizeArray32<>' could not be found (are you missing a using directive or an assembly reference?)
UserFileEvent.cs(13, 66): [CS0246] The type or namespace name 'FixedSizeArray12<>' could not be found (are you missing a using directive or an assembly reference?)
ActionManager.cs(153, 25): [CS8795] Partial method 'ActionManager.StartCooldown(ActionType, uint)' must have an implementation part because it has accessibility modifiers.
RaptureAtkModule.cs(124,32): Error CS8377 : The type 'RaptureAtkModule.ItemCache' must be a non-nullable value type, along with all fields at any level of nesting, in order to use it as parameter 'T' in the generic type or method 'InheritsAttribute<T>'
RaptureAtkModule.cs(129,32): Error CS8377 : The type 'RaptureAtkModule.ItemCache' must be a non-nullable value type, along with all fields at any level of nesting, in order to use it as parameter 'T' in the generic type or method 'InheritsAttribute<T>'
I'm using the project solution file and Rider 2024.1.4. Is there something else I should be using instead?
FYI, the unknown long at 0x1A0 in FFXIVClientStructs.FFXIV.Group.Partymember
is the players Content ID (errr I think that's what it's called anyway; your eight-byte identifier).
What would a class-based approach look like?
Do we need to need to specify the size of a vtable at all? When I was calculating things, the only xref in a vtable is vf0. The next xref is generally the start of a new vtable. Is this the rule or just common? Do vtables continue past what look like xrefs to data (dwords, bytes, etc)?
@vtbl(0x14169AE50, size=3) # Cycle through __dict__, basically treating this like an enum
class Component__GUI__AtkResNode(Component__GUI__AtkEventTarget):
dtor = 1 # also lose the nice layout orientation of all the addresses in a line, unless we turn off formatting, aligning the = signs.
ctor = 0x1404CC6B0"
@vtbl(0x14169AEC8) # More verbose, harder to maintain?
class Component__Gui__AtkCollisionNode(Component__GUI__AtkResNode):
@vfunc(1)
def dtor(): ...
@func(0x14053F820)
def ctor(): ...
It looks like a class, but its still basically working the vtables. However the automatic sizing would open up some opportunities I think. Especially if we can identify the first vtbl object regularly. Then they could all be labled something dumb like vtbl_Unknown001.
Trying to find out if I'm inside or not, get a confusing error. Can't find any documentation so any help would be appreciated.
Code:
FFXIVClientStructs.Interop.Resolver rt = FFXIVClientStructs.Interop.Resolver.GetInstance;
rt.SetupSearchSpace();
rt.Resolve();
bool isInside = FFXIVClientStructs.FFXIV.Client.Game.Housing.HousingManager.Instance()->IsInside();
Error:
Function pointer for HousingManager.Instance is null. The resolver was either uninitialized or failed to resolve address with signature E8 ?? ?? ?? ?? 8B 56 7C ?? ?? ?? ?? ?? ?? ?? ??.'
I apologize for opening this but I don't see anything in the repo about the kind of contributions you accept and this has recently come up and I wanted to ask/clarify.
Concerns have been raised with me about some of the things I've recently contributed (and were accepted), in that they are potentially dangerous (or even exploitable).
From my point of view, this project is meant to be an as complete as possible reverse engineering of FFXIV and encapsulates the collective knowledge about the client. There are already things either implemented (or documented in data.yml) which are inherently dangerous and can be used by nefarious developers to create cheats etc (and actively are).
I'm generally of the opinion that knowledge is not harmful and that if people choose to misuse or abuse it, that's on them.
Do you have any official guidance on what is acceptable to document/implement in this project and not?
Should I continue to push things regardless of what they are?
Hello,
I am getting quite a few errors when I run this macro. They are of the sort:
7FF67FB4E810: can't rename byte as 'vtbl_Client::UI::AddonTooltip' because this byte can't have a name (it is a tail byte).
7FF67FB675A8: can't rename byte as 'vtbl_Client::UI::AddonCharaSelectWorldServer' because this byte can't have a name (it is a tail byte).
7FF67FBB5F98: can't rename byte as 'vtbl_Client::UI::AddonWeeklyPuzzle' because this byte can't have a name (it is a tail byte).
7FF67FBBD2A0: can't rename byte as 'vtbl_Client::UI::AddonMJIRecipeNoteBook' because this byte can't have a name (it is a tail byte).
and
Error: Function at 0x7FF67EA5E730 had unexpected name "" during naming of Client::Game::UI::RecipeNote.CancelCrafting
Error: Function at 0x7FF67EA5F690 had unexpected name "" during naming of Client::Game::UI::RecipeNote.IsRecipeUnlocked
Error: Function at 0x7FF67EA5FC00 had unexpected name "" during naming of Client::Game::UI::RecipeNote.FirstRecipeIndex
Error: Function at 0x7FF67EA5FBC0 had unexpected name "" during naming of Client::Game::UI::RecipeNote.GetRecipeByIndex
Error: Function at 0x7FF67EA5FC40 had unexpected name "" during naming of Client::Game::UI::RecipeNote.GetSelectedRecipe
Error: Function at 0x7FF67EA607A0 had unexpected name "" during naming of Client::Game::UI::RecipeNote.Initialize
probably hundreds of them. I did put the ida.cfg file in place. Is there something else I must do to not get these errors? Or is this normal?
Thanks.
Example steps to reproduce:
(*AgentMap.Instance()).CurrentOffsetX
and (*AgentMap.Instance()).CurrentOffsetY
These numbers should be 704 and 704. Perhaps something changed in Dawntrail to make them negative?
This is currently breaking Dalamud's GetMapCoordinates()
method: goatcorp/Dalamud#1916
imo the source generator is quite hard to debug and use, especially if you'd like to view what their output looks like on GitHub.
I think it would be better to convert it to a standalone application that produces files that can be committed into the repository.
Open to arguments otherwise, though.
Is there any resource that generally describes the FFXI software architecture and how all the various objects interact? I'd be particularly interested in generally understanding how the Raptor GUI toolkit works, as well as the general flow of how addon event handling works.
Using AddonHudLayoutScreen for example.
How are vfuncs accessed while still retaining access to the vtbl? Currently the vtbl is in AtkUnitBase.
AddonHudLayoutScreen should get its own vtbl for its own specific functions. Since we don't necessarily have knowledge of what is overridden like in IDA, it'll likely need to inherit all of AtkUnitBase's names too.
Should vfuncs be differentiated from funcs?
Should we get into the including delegate business?
[VTable(0x1418106A0)]. // This I could like
[Func("AddonOverlayMouseMovedEvent", 0x141023730)] // not a fan of looking these up by string
public unsafe struct AddonHudLayoutScreen { <stuff> }
public unsafe struct AddonHudLayoutScreen
{
...
public static class VTable
{
public static readonly IntPtr Address = new IntPtr(0xdeadbeef);
public static IntPtr this[int index] => Address + 8*index; // Although you cant have static indexers.
}
public static class Functions
{
public static readonly IntPtr AddonOverlayMouseMovedEventAddress = new IntPtr(0x141023730);
}
public static class Delegates
{ // How would these be stored, as plaintext in the data.yml?
public delegate void AddonOverlayMouseMovedEventDelegate(stuff);
}
}
Title says it all. Here is a link to a SS of the issue:
https://i.imgur.com/p60lwvW.png
I have the latest version of FFXIV 6.x installed.
Does anyone know if there exists a data structure for spearfishing?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.