notadam / lumina

A simple, performant and extensible framework for interacting with FFXIV game data

Lumina

Lumina is a small & high-performance library for interacting with FINAL FANTASY XIV game data in a convenient manner.

Looking for Excel sheets? They've moved here.

Documentation is located here.

Usage

See the Lumina.Example project for a quick start, otherwise for more comprehensive usage examples, check the getting started guide.

lumina's Issues

Item sheet fails to parse, even when generated with updated definitions

System.InvalidCastException
  HResult=0x80004002
  Message=Unable to cast object of type 'System.Int16' to type 'System.Byte'.
  Source=Lumina
  StackTrace:
   at Lumina.Excel.RowParser.ReadField[T](ExcelColumnDataType type) in Lumina\Lumina\Excel\RowParser.cs:line 215
   at Lumina.Excel.RowParser.ReadOffset[T](Int32 offset, Byte bit) in Lumina\Lumina\Excel\RowParser.cs:line 245
   at Lumina.Excel.GeneratedSheets.Item.PopulateData(RowParser parser, Lumina lumina) in Lumina\Lumina\Excel\GeneratedSheets\I\Item.cs:line 356
   at Lumina.Excel.ExcelSheet`1.GetRows() in Lumina\Lumina\Excel\ExcelSheet.cs:line 160

This has happened for a while with the latest lumina version.

I tried updating definitions from a more recent godbert build and manually exporting a new Item sheet via the spaghetti generator, with the same result.

I also tried manually hacking that particular set of bytes for unknown58, but it then failed on the next similar block (unknown60).

Recipe UnkData5 should be "ItemIngredient"

The Recipe type in Lumina.Excel.GeneratedSheets (6.0.2 from nuget) has a property UnkData5 which is an array of RecipeUnkData5Obj - these are ItemIngredients and should be called such.

The SaintCoinach def file appears to be correctly naming these - is there anything I can do to help get the proper name into this library? It appears those classes are generated so it doesn't look like a PR to fix the code directly is the best option.

Feature request: Change FileResource.Data in custom FileResource impl

FileResource.Data has an internal setter, so you can't change it in your own FileResource implementations. This will lead to a potential footgun when using Xande - there is a method to change the data inside of a .sklb file, but it can't update Data, so I had to write my own Write method to write it to a stream.

Should IsValueCreated reference Value instead of _value?

It seems as if when checking if a LazyRow is made via IsValueCreated, since it references _value it's going to always be null until Value is called atleast once. Maybe I'm still thinking in terms of HasValue. Would the appropriate behaviour be to check if Value is null for having a value rather than IsValueCreated?

RowID, HasValue, IsValueCreated, ZoneName over 2 iterations of a sheet.

911 False False
912 False False Gyr Abania
913 False False
914 False False Norvrandt
915 False False
916 False False Norvrandt
917 False False Norvrandt
918 False False Norvrandt
919 False False

911 True True
912 True True Gyr Abania
913 True True
914 True True Norvrandt
915 True True
916 True True Norvrandt
917 True True Norvrandt
918 True True Norvrandt
919 True True

files are unnecessarily copied while being read

• files are copied several times
• inflate isn't in-place and requires a separate buffer
• inflate uses the same pooled mem, but the acquire and release is extra work that likely doesn't need to happen, especially for high throughput
• other things

Issue reading file sqpack file.

Working on updating the model file readers but running into the following issue when reading a mdl file:

File: bg/ffxiv/sea_s1/twn/s1t1/bgplate/0032.mdl

System.IO.InvalidDataException: Found invalid data while decoding.
   at System.IO.Compression.InflaterZlib.Inflate(FlushCode flushCode)
   at System.IO.Compression.InflaterZlib.ReadInflateOutput(Byte[] outputBuffer, Int32 offset, Int32 length, FlushCode flushCode, Int32& bytesRead)
   at System.IO.Compression.InflaterZlib.Inflate(Byte[] bytes, Int32 offset, Int32 length)
   at System.IO.Compression.DeflateStream.Read(Byte[] array, Int32 offset, Int32 count)
   at System.IO.Stream.InternalCopyTo(Stream destination, Int32 bufferSize)
   at System.IO.Stream.CopyTo(Stream destination)
   at Lumina.Data.SqPackStream.ReadFileBlock(Int64 offset, MemoryStream dest, Boolean resetPosition) in \git\Lumina\src\Lumina\Data\SqPackStream.cs:line 304
   at Lumina.Data.SqPackStream.ReadFileBlock(MemoryStream dest, Boolean resetPosition) in \git\Lumina\src\Lumina\Data\SqPackStream.cs:line 282
   at Lumina.Data.SqPackStream.ReadModelFile(FileResource resource, MemoryStream ms) in \git\Lumina\src\Lumina\Data\SqPackStream.cs:line 171
   at Lumina.Data.SqPackStream.ReadFile[T](Int64 offset) in \git\Lumina\src\Lumina\Data\SqPackStream.cs:line 89
   at Lumina.Data.SqPack.ReadFile[T](Int64 offset) in \git\Lumina\src\Lumina\Data\SqPack.cs:line 123
   at Lumina.Data.Category.GetFile[T](ParsedFilePath path) in \git\Lumina\src\Lumina\Data\Category.cs:line 124
   at Lumina.Data.Repository.GetFile[T](Byte cat, ParsedFilePath path) in \git\Lumina\src\Lumina\Data\Repository.cs:line 79
   at Lumina.Data.Repository.GetFile[T](String cat, ParsedFilePath path) in \git\Lumina\src\Lumina\Data\Repository.cs:line 67
   at Lumina.Lumina.GetFile[T](String path) in \git\Lumina\src\Lumina\Lumina.cs:line 148

I am digging into it, but any insight into this issue would be helpful.

Thanks :D

re: Need to develop a Lumina program

Referencing: #28

I just moved to Korea so I do not speak the language. I don't know what your problem is @NotAdam , you do not seem mentally stable but if you cannot do the job then someone else can.

Can't GetFileFromDisk<MdlFile>()

It's because MdlFile tries to read FileInfo.Type which isn't set by GetFileFromDisk, so it NREs.

Repro: var mdlFile = GameData.GetFileFromDisk<MdlFile>("(literally any model file)");

Need to develop a Lumina program

I have the special Korean FFXIV client and the developers intentionally removed English from the sqpack (quest sheets, etc.). I would like you to develop a program with Lumina which can replace these with the matching data from the NA FFXIV client.

I tried just swapping files 0x0a data files, but the Korean client would not load.

Ping @NotAdam or anyone who has the ability to take this job on, name your price.

Some LGB files are parsed incorrectly and fail

There seem to be multiple different issues, but I only have details on one so far.

A number of planner.lgb files have a chunk header that is seemingly different from others, and will parse incorrectly and throw an OOM exception due to an invalid layer count being read from the chunk header.

One example of this is bg/ffxiv/wil_w1/twn/w1t1/level/planner.lgb

In that particular file, it appears that the actual chunk header (the 'LGP1' magic) starts at byte 32, instead of byte 12 as the code assumes. Byte 12 is a 0x1, followed by all 0s until byte 32, so I'm not sure what if anything might ever go there.

Unfortunately, the general structure still seems wrong even accounting for the unexplained 20 bytes (ie, reading the header from where LGP1 is found will still result in incorrect data for at least some fields)

This doesn't occur with every planner file, but it does happen with a number of them.

Some ULD files are parsed incorrectly and fail

Samples:

• ui/uld/macro.uld fails with System.IO.EndOfStreamException: 'Unable to read beyond the end of the stream.'.
• ui/uld/jobhudsam1.uld fails with System.OutOfMemoryException.

Checksum Mistmatch

Trying to get items using the Lumina.GetExcelSheet(); and it crashes with the following error:

Lumina.Excel.Exceptions.ExcelSheetColumnChecksumMismatchException: 'sheet Item column hash mismatch! expected hash: 800968c9, actual hash: 62d7ae33!'

Versions:
Lumina 3.10.0
LuminaExcel (6.3.2)

EDIT:
I downgraded Lumina to 3.8.0 and Excel to 6.1.1 and it works just fine now.
(Had a previous program that was using these versions and everything still works.)

Pet Action Heal/Place?

I can't find Heal/Place action in both generated sheets Action and PetAction. Could you point me to where they are, or resolve them in the sheets?

Broken language request for sub sheets

Requesting a sheet in a specific language will result in the sub sheets not following this request.

Example:
TerritoryType requested as german, sheet is german
TerritoryType.PlaceName sheet is english

Double nested repeat support

https://github.com/ufx/SaintCoinach/blob/master/SaintCoinach/Definitions/Quest.json#L341-L355

more crazy definitions that are missing.