Status like ruin4 now always show their stack count as 0. But their status.pram shows their correct stacks. Is this a bug?

ffxiv_idarename.py> Running...
  File "W:\bullshit\ghidra\ffxiv\FFXIVClientStructs\ida\ffxiv_idarename.py", line 619
    def recurse_tree(current_node: Node, current_class: FfxivClass, all_bases: set[str]):
                                ^
SyntaxError: mismatched input ':' expecting RPAREN
ffxiv_idarename.py> Finished!

Seems to be this commit breaking it - type hinting isn't in py2, which Ghidra uses.

Current offset exceeded the next field's offset (0x228 > 0x1E0): Client::Graphics::Render::RenderTargetManager.OffscreenRenderTarget_1
Current offset exceeded the next field's offset (0x18E8 > 0x18C8): Client::UI::AddonWeeklyBingo.SecondChanceButton
Current offset exceeded the next field's offset (0x8 > 0x1): Client::Game::Character::Character::ForayInfo.Element
Current offset exceeded the next field's offset (0x4280 > 0x3E90): Client::UI::Agent::AgentMap.MiniMapMarkerArray
Current offset exceeded the next field's offset (0x8F8 > 0x8F1): Client::Graphics::Scene::Human.Sex
Current offset exceeded the next field's offset (0x1A0 > 0x198): Client::Game::UI::Buddy.Pet
Unhandled type: FFXIVClientStructs.FFXIV.Client.System.Memory.ICreatable

I think the ForayInfo is an issue in how the program calculates offsets.
WeeklyBingo has buttons in the wrong struct i think? but the offsets need to be recalculated.
I'm not sure whats going on with BuddyPet.

Issue

It was recently discovered that AtkTextNode stores the pointer to the original text buffer used when calling SetText. The string overload generator handles converting C# strings (UTF-16) to strings the game expects (generally, null-terminated UTF-8). It relies on the fact that the game doesn't take ownership of the string buffer passed to the native function and temporarily allocates a buffer to store the converted UTF-8 string in, usually on the stack. The game then copies the string into its own objects and the string overload method can allow the buffer to be freed. Since AtkTextNode stores a pointer to the original text buffer, this results in the node storing a pointer to the stack.

While people have been using this function for years with no visible issues, it is incredibly unsafe for us to be storing a stack pointer in a native game object like this, so we can't provide string overloads for SetText anymore.

Plugin Authors

If you use SetText you will need to allocate the UTF-8 string buffer yourself and free it when your plugin is done using the TextNode.

The sample code below shows how to allocate and pass the buffer. It uses the same methods ClientStructs' string overloads uses to convert a string to UTF8.

byte* strBuffer;

unsafe void SetText(AtkTextNode* node, string text)
{
    if (strBuffer != null) // free buffer if you've already got one
        System.Runtime.InteropServices.NativeMemory.Free(strBuffer);
    int strLen = System.Text.Encoding.UTF8.GetByteCount(text); // get length of string as UTF-8 bytes
    strBuffer = System.Runtime.InteropServices.NativeMemory.Alloc(strLen + 1); // need one extra byte for the null terminator
    Span<byte> bufferSpan = new(strBuffer, strLen + 1); // wrap buffer in a span so you can use GetBytes
    System.Text.Encoding.UTF8.GetBytes(text, bufferSpan); // convert string to UTF-8 and store in your buffer
    bufferSpan[strLen] = 0; // add null terminator to the end of your string
    node->SetText(strBuffer);
}

You will also need to call Free when you are no longer using the TextNode (if you're destroying it, or your plugin is unloading, for example). You can also use other C# allocation functions (AllocHGlobal) or the game's native UI allocator (via IMemorySpace) if you want.

Excessive Allocations

You may be worried about allocating a new buffer every time you set the text. First - don't prematurely optimize. That said, the way to avoid this is to re-use the buffer.

The code examples below weren't directly tested, likely have typos, and probably need some more work to be complete.

byte* strBuffer;
int strBufferLen;

unsafe void SetText(AtkTextNode* node, string text)
{
    int strLen = System.Text.Encoding.UTF8.GetByteCount(text); // get length of string as UTF-8 bytes
    if (strBuffer == null || strLen + 1 > strBufferLen) // reallocate buffer if it doesn't already exist or is too small
    {
        System.Runtime.InteropServices.NativeMemory.Free(strBuffer);
        strBuffer = System.Runtime.InteropServices.NativeMemory.Alloc(strLen + 1); // need one extra byte for the null terminator
        strBufferLen = strLen + 1;
    }
    Span<byte> bufferSpan = new(strBuffer, strLen + 1); // wrap buffer in a span so you can use GetBytes
    System.Text.Encoding.UTF8.GetBytes(text, bufferSpan); // convert string to UTF-8 and store in your buffer
    bufferSpan[strLen] = 0; // add null terminator to the end of your string
    node->SetText(strBuffer);
}

You could wrap this logic in a class:

unsafe class AtkTextNodeBufferWrapper
{
    private byte* strBuffer;
    private int bufferLen;

    public byte* GetBuffer => strBuffer;

    public void SetBuffer(string text)
    {
        int strLen = System.Text.Encoding.UTF8.GetByteCount(text); // get length of string as UTF-8 bytes
        if (strBuffer == null || strLen + 1 > strBufferLen) // reallocate buffer if it doesn't already exist or is too small
        {
             System.Runtime.InteropServices.NativeMemory.Free(strBuffer);
            strBuffer = System.Runtime.InteropServices.NativeMemory.Alloc(strLen + 1); // need one extra byte for the null terminator
            strBufferLen = strLen + 1;
        }
        Span<byte> bufferSpan = new(strBuffer, strLen + 1); // wrap buffer in a span so you can use GetBytes
        System.Text.Encoding.UTF8.GetBytes(text, bufferSpan); // convert string to UTF-8 and store in your buffer
        bufferSpan[strLen] = 0; // add null terminator to the end of your string
    }

    public void FreeBuffer()
    {
        System.Runtime.InteropServices.NativeMemory.Free(strBuffer);
        bufferLen = 0;
    }
}

Also consider pre-allocating a buffer length you know is likely to store the longest text you're setting, especially if its a small amount of text.

Another option is to use the game's Utf8String class. This class has internal storage for a string of length 64 or smaller and won't reallocate unless your strings are larger.

Utf8String.FromString(str) allows you to allocate an unmanaged Utf8String* which you can then save. You can update the string text with SetString. The Utf8String's StringPtr can be safely passed to SetText.

PulseActionBarSlot Crashed the game!

I don't know the reverse engineering. And I guess this is because the index of vfunc has changed.

In an attempt to fix a bugged config in RSR i found that NameplateIconId does not populate properly for Hunt bill or Atma weapon relic book targets

I joint the fate in game, but FateManager.Instance()->FateJoined is 0. And the FateManager.Instance()->CurrentFate is Zero.

Generators

• Static VTables
  Make vtables available as addresses without needing an instance of the object first. Also provide vtable size via attribute argument. Would be integrated with the virtual function generator and generate a properly-sized vtable struct.
• Inheritance
  [Inherits] attribute that generates struct members from the parent into the child so all fields and methods show as available without having to do AtkTextNode->AtkResNode for example. This generator would need info from all other generators so first combine all generators into a single generator (relatively easy) beforehand.

IDA

• Reformat data.yml
  There are some constructs we don't really support well. Might switch off yaml to something "simpler" since we don't need yaml features.
• CExporter
  Will likely rewrite this using Roslyn syntax analysis rather than reflection. In addition to exporting types, export method signatures.
• Rewrite import script
  Import types and use method signatures to set function argument types appropriately. Also want to pull in some other forms of smart renaming (rename EXD getters via the root, maybe rename unknown agents and addons automatically with their IDs and uld names, more ideas here???)

Other

Anything I'm forgetting. Feel like we've talked about other stuff before.

I feel like some way to indicate which structs have been 'validated' after a patch (anything that didnt change size is probably safe, and anything that changed size and has had its offsets verified) would be nice. There's no doubt numerous "broken" (outdated) offsets and having some way of tracking that would be useful.

https://github.com/xivapi/ffxiv-datamining/blob/master/csv/Status.csv

At the very least, its better terminology for buffs AND debuffs.

I realize this is a bit OT, but I am suddenly unable to attach ida to the running game after the 6.1 patch. I was wondering if you have experienced this issue and, if so, have you discovered what anti-debug method they are using.

Is BalloonType uint or byte?
Current offset exceeded the next field's offset (0xEC > 0xE9): Client::UI::Agent::BalloonInfo.byte

namespace FFXIVClientStructs.FFXIV.Client.Game;
public enum BalloonType : uint
{
    Timer = 0, // runs on a simple timer and disappears when the timer ends
    Unknown = 1 // the non-timer mode, not sure what its called or where its used
}

namespace FFXIVClientStructs.FFXIV.Client.Game;
public struct Balloon
{
    ...
    [FieldOffset(0x8)] public BalloonType Type; // <- uint usage?
    [FieldOffset(0xC)] public BalloonState State;
    ...
}

namespace FFXIVClientStructs.FFXIV.Client.UI.Agent;
public unsafe struct BalloonInfo
{
    ...
    [FieldOffset(0xE4)] public int BalloonId;  // <- byte usage?
    [FieldOffset(0xE8)] public BalloonType Type;
    ...
}

For structs with explicit set struct size, there is no error for fields that esceed the struct size

PS: I vaguely recall some discussion about this, but as I did not find any written info on this I'll open an issue for either fixing this or making a note in a readme

Was playing around with Raycasts a bit and it looks like some offsets have been changed:

• Distance is at 0x48 now
• I'm pretty sure Object is at 0x50
• I think Flags is at 0x40

Opened this issue so someone with better knowledge of the struct can take a look cause I don't really know how the Flags or the Object should look like.

This could be added to the README.md pip dependencies I think

Hey @wolfcomp,

as far as I understand, excel definitions are now maintained in EXDSchema, but ffxiv-exdgetters.py still uses Saint Coinach definitions.

Please update the script when you have some time.
Thank you very much! 🙇‍♂️

It looks like the c structs are a bit out of date with the csharp source, so I tried building/running your CExporter and got the following:

PS C:\Users\miked\Source\repos\FFXIVClientStructs\ida\CExporter\bin\Debug\net5.0> ./CExporter.exe Unhandled exception. System.ArgumentException: Field passed in is not a marshaled member of the type 'FFXIVClientStructs.FFXIV.Client.UI.AddonEnemyList'. (Parameter 'fieldName') at System.Runtime.InteropServices.Marshal.OffsetOf(Type t, String fieldName) at CExporter.FieldInfoExtensions.GetFieldOffset(FieldInfo finfo) in C:\Users\miked\Source\repos\FFXIVClientStructs\ida\CExporter\Program.cs:line 424 at CExporter.Exporter.<>c.<ProcessStruct>b__10_1(FieldInfo finfo) in C:\Users\miked\Source\repos\FFXIVClientStructs\ida\CExporter\Program.cs:line 145 at System.Linq.EnumerableSorter2.ComputeKeys(TElement[] elements, Int32 count)
at System.Linq.EnumerableSorter1.ComputeMap(TElement[] elements, Int32 count) at System.Linq.EnumerableSorter1.Sort(TElement[] elements, Int32 count)
at System.Linq.OrderedEnumerable1.GetEnumerator()+MoveNext() at System.Linq.Lookup2.Create(IEnumerable1 source, Func2 keySelector, IEqualityComparer1 comparer) at System.Linq.GroupedEnumerable2.GetEnumerator()
at CExporter.Exporter.ProcessStruct(Type type, StringBuilder header) in C:\Users\miked\Source\repos\FFXIVClientStructs\ida\CExporter\Program.cs:line 147
at CExporter.Exporter.ProcessType(Type type, StringBuilder header) in C:\Users\miked\Source\repos\FFXIVClientStructs\ida\CExporter\Program.cs:line 105
at CExporter.Exporter.Export(GapStrategy strategy) in C:\Users\miked\Source\repos\FFXIVClientStructs\ida\CExporter\Program.cs:line 84
at CExporter.Program.Main(String[] _) in C:\Users\miked\Source\repos\FFXIVClientStructs\ida\CExporter\Program.cs:line 19`

do you know if theres data somewhere in the window to know how many in the Index are The non click headers? like are there no collision nodes for them, and if so is there a way to determine what there specific indexes are? like maybe an array of non collision headers and their indexes? is that possible?

Not sure if this is the right place, but I'd like to know if any of you FFXIV reversing experts would be willing to help me understand it's software architecture a bit more. Particularly, I'd love to have a better understanding the role of Agents and how they work, how the FFXIV UI system works, how the UI retrieves the data it displays from the server, and how to manipulate the controls on the UI. Even a basic interaction diagram of those components would be very helpful.

If this information already exists somewhere, then a link to where I can find it could be appreciated. Otherwise, if you don't want to have the discussion here, feel free to email me at [email protected].

Thanks!

I want to create structures for the FFXIV Korean version.
Can you explain how to create a data.yml to use in IDA?

I know how much you love updating your diff scripts. This seems sane enough, although if you want it in json instead, i can make that happen. The idea is simple enough, take the actual data out of the py script into a yml file.

version: 2020.12.29.0000.0000

globals:
  0x14169A2A0: g_HUDScaleTable

functions:
  0x140059670: FFXIVString_ctor  # empty string ctor

classes:
  Client::Game::Control::TargetSystem::ListFeeder:
  
  GroupManager:
    funcs:
      0x140BB2600: Create
  Common::Configuration::ConfigBase:
    inherits_from: Client::System::Common::NonCopyable
    vtbl: 0x14164E260
    funcs:
      0x140068C30: ctor
  Common::Configuration::UIConfig:
    inherits_from: Common::Configuration::ConfigBase 
    vtbl: 0x14164E280
  Client::System::Framework::Framework:
    vtbl: 0x14164F4B8
    vfuncs:
      1: Setup
    funcs:
      0x14008EA40: ctor

With 603b211 I added versioning to Client Structs, but I failed to realize GitVersion fails to work with detached heads which makes building anything that isn't fully updated with Client Structs fails to build.

I'd like to keep versioning as I believe it will be helpful, but I'm unsure of a better way to do it.

Any ideas?

In FFXIVClientStructs.FFXIV.Client.Game.Status, the fields StackCount and Param are actually one combined field, at least in the case of food and potions.

The u16 starting at offset 2 contains the ItemFood row for consumed food and potion status effects (i.e. "Well Fed" and "Medicated"). I imagine for status effects with stacks, it's conceivable that it's actually just a u16.

Hi, you're probably still working on updating FFXIVClientStructs but just wanted to let you know this enum seems to be off by one now.

My plugin code which is now hitting BGM instead of master.

        /// <summary>
        /// Mutes/Unmutes the sound
        /// </summary>
        /// <param name="enabled"></param>
        public static unsafe void SetMasterSoundEnable(bool enabled)
        {
            Framework.Instance()->SystemConfig.CommonSystemConfig.ConfigBase.ConfigEntry[(int)ConfigOption.IsSndMaster].SetValueUInt((uint)(enabled ? 0 : 1));
        }

        public static unsafe bool GetMasterSoundEnable()
        {
            if (Framework.Instance()->SystemConfig.CommonSystemConfig.ConfigBase.ConfigEntry[(int)ConfigOption.IsSndMaster].Value.UInt == 0)
                PluginLog.Debug("Enabled");
            return (Framework.Instance()->SystemConfig.CommonSystemConfig.ConfigBase.ConfigEntry[(int)ConfigOption.IsSndMaster].Value.UInt == 0);
        }

I seem to be unable to build the project. I get the following errors (there's over 2500 of them, I'm only including a few):

AtkUnitBase.cs(15, 65): [CS0246] The type or namespace name 'FixedSizeArray32<>' could not be found (are you missing a using directive or an assembly reference?)
UserFileEvent.cs(13, 66): [CS0246] The type or namespace name 'FixedSizeArray12<>' could not be found (are you missing a using directive or an assembly reference?)
ActionManager.cs(153, 25): [CS8795] Partial method 'ActionManager.StartCooldown(ActionType, uint)' must have an implementation part because it has accessibility modifiers.
RaptureAtkModule.cs(124,32): Error CS8377 : The type 'RaptureAtkModule.ItemCache' must be a non-nullable value type, along with all fields at any level of nesting, in order to use it as parameter 'T' in the generic type or method 'InheritsAttribute<T>'
RaptureAtkModule.cs(129,32): Error CS8377 : The type 'RaptureAtkModule.ItemCache' must be a non-nullable value type, along with all fields at any level of nesting, in order to use it as parameter 'T' in the generic type or method 'InheritsAttribute<T>'

I'm using the project solution file and Rider 2024.1.4. Is there something else I should be using instead?

FYI, the unknown long at 0x1A0 in FFXIVClientStructs.FFXIV.Group.Partymember is the players Content ID (errr I think that's what it's called anyway; your eight-byte identifier).

What would a class-based approach look like?

Do we need to need to specify the size of a vtable at all? When I was calculating things, the only xref in a vtable is vf0. The next xref is generally the start of a new vtable. Is this the rule or just common? Do vtables continue past what look like xrefs to data (dwords, bytes, etc)?

@vtbl(0x14169AE50, size=3)  # Cycle through __dict__, basically treating this like an enum
class Component__GUI__AtkResNode(Component__GUI__AtkEventTarget):
    dtor = 1  # also lose the nice layout orientation of all the addresses in a line, unless we turn off formatting, aligning the = signs. 
    ctor = 0x1404CC6B0"

@vtbl(0x14169AEC8)  # More verbose, harder to maintain? 
class Component__Gui__AtkCollisionNode(Component__GUI__AtkResNode):
    @vfunc(1)
    def dtor(): ...
    @func(0x14053F820)
    def ctor(): ...

It looks like a class, but its still basically working the vtables. However the automatic sizing would open up some opportunities I think. Especially if we can identify the first vtbl object regularly. Then they could all be labled something dumb like vtbl_Unknown001.

Trying to find out if I'm inside or not, get a confusing error. Can't find any documentation so any help would be appreciated.

Code:
FFXIVClientStructs.Interop.Resolver rt = FFXIVClientStructs.Interop.Resolver.GetInstance;
rt.SetupSearchSpace();
rt.Resolve();
bool isInside = FFXIVClientStructs.FFXIV.Client.Game.Housing.HousingManager.Instance()->IsInside();

Error:
Function pointer for HousingManager.Instance is null. The resolver was either uninitialized or failed to resolve address with signature E8 ?? ?? ?? ?? 8B 56 7C ?? ?? ?? ?? ?? ?? ?? ??.'

I apologize for opening this but I don't see anything in the repo about the kind of contributions you accept and this has recently come up and I wanted to ask/clarify.

Concerns have been raised with me about some of the things I've recently contributed (and were accepted), in that they are potentially dangerous (or even exploitable).

From my point of view, this project is meant to be an as complete as possible reverse engineering of FFXIV and encapsulates the collective knowledge about the client. There are already things either implemented (or documented in data.yml) which are inherently dangerous and can be used by nefarious developers to create cheats etc (and actively are).

I'm generally of the opinion that knowledge is not harmful and that if people choose to misuse or abuse it, that's on them.

Do you have any official guidance on what is acceptable to document/implement in this project and not?
Should I continue to push things regardless of what they are?

Hello,

I am getting quite a few errors when I run this macro. They are of the sort:

7FF67FB4E810: can't rename byte as 'vtbl_Client::UI::AddonTooltip' because this byte can't have a name (it is a tail byte).
7FF67FB675A8: can't rename byte as 'vtbl_Client::UI::AddonCharaSelectWorldServer' because this byte can't have a name (it is a tail byte).
7FF67FBB5F98: can't rename byte as 'vtbl_Client::UI::AddonWeeklyPuzzle' because this byte can't have a name (it is a tail byte).
7FF67FBBD2A0: can't rename byte as 'vtbl_Client::UI::AddonMJIRecipeNoteBook' because this byte can't have a name (it is a tail byte).

and

Error: Function at 0x7FF67EA5E730 had unexpected name "" during naming of Client::Game::UI::RecipeNote.CancelCrafting
Error: Function at 0x7FF67EA5F690 had unexpected name "" during naming of Client::Game::UI::RecipeNote.IsRecipeUnlocked
Error: Function at 0x7FF67EA5FC00 had unexpected name "" during naming of Client::Game::UI::RecipeNote.FirstRecipeIndex
Error: Function at 0x7FF67EA5FBC0 had unexpected name "" during naming of Client::Game::UI::RecipeNote.GetRecipeByIndex
Error: Function at 0x7FF67EA5FC40 had unexpected name "" during naming of Client::Game::UI::RecipeNote.GetSelectedRecipe
Error: Function at 0x7FF67EA607A0 had unexpected name "" during naming of Client::Game::UI::RecipeNote.Initialize

probably hundreds of them. I did put the ida.cfg file in place. Is there something else I must do to not get these errors? Or is this normal?

Thanks.

Example steps to reproduce:

1. Visit Shirogane Subdivision (Wards 31-60)
2. Check the values of (*AgentMap.Instance()).CurrentOffsetX and (*AgentMap.Instance()).CurrentOffsetY
3. Observe that they are -704 and -704, respectively

These numbers should be 704 and 704. Perhaps something changed in Dawntrail to make them negative?

This is currently breaking Dalamud's GetMapCoordinates() method: goatcorp/Dalamud#1916

imo the source generator is quite hard to debug and use, especially if you'd like to view what their output looks like on GitHub.

I think it would be better to convert it to a standalone application that produces files that can be committed into the repository.

Open to arguments otherwise, though.

Is there any resource that generally describes the FFXI software architecture and how all the various objects interact? I'd be particularly interested in generally understanding how the Raptor GUI toolkit works, as well as the general flow of how addon event handling works.

Using AddonHudLayoutScreen for example.

How are vfuncs accessed while still retaining access to the vtbl? Currently the vtbl is in AtkUnitBase.
AddonHudLayoutScreen should get its own vtbl for its own specific functions. Since we don't necessarily have knowledge of what is overridden like in IDA, it'll likely need to inherit all of AtkUnitBase's names too.
Should vfuncs be differentiated from funcs?
Should we get into the including delegate business?

[VTable(0x1418106A0)].   // This I could like
[Func("AddonOverlayMouseMovedEvent", 0x141023730)]    // not a fan of looking these up by string
public unsafe struct AddonHudLayoutScreen { <stuff> } 

public unsafe struct AddonHudLayoutScreen 
{
    ...
    public static class VTable
    {
        public static readonly IntPtr Address = new IntPtr(0xdeadbeef);
        public static IntPtr this[int index] => Address + 8*index;  // Although you cant have static indexers.
    }
    public static class Functions 
    {
        public static readonly IntPtr AddonOverlayMouseMovedEventAddress = new IntPtr(0x141023730);
    }
    public static class Delegates
    {  // How would these be stored, as plaintext in the data.yml?
        public delegate void AddonOverlayMouseMovedEventDelegate(stuff);
    }
} 

Title says it all. Here is a link to a SS of the issue:

https://i.imgur.com/p60lwvW.png

I have the latest version of FFXIV 6.x installed.

Does anyone know if there exists a data structure for spearfishing?