mestrelion / mcworldlib

Yet another library to manipulate Minecraft data, inspired by pymclevel

mcworldlib - Minecraft World Library

Yet another library to manipulate Minecraft data, inspired by the now-defunct pymclevel, building on top of the amazing nbtlib.

Focused on making the bridge between the on-disk save files and directory structure and their NBT content, much like NBTExplorer, presenting all World data in a structured, convenient way so other tools can build on top of it and add more semantics to that data.

---

Features

• Read and write .dat NBT files, both uncompressed and gzip-compressed.
• Read and write .mca/.mcr Anvil region files, lazily loading their contents only when the data is actually requested, also monitoring content changes to efficiently save back to disk only the needed files.
• Read and write .mcc external chunk files, loading from there when indicated by the chunk header in the .mca region file, and automatically selecting the appropriate format on save: external mcc if the chunk data outgrows its previous maximum size (~1 MB), and back to the mca if it shrinks enough to fit there again.

---

Usage

Reading world data

You can open a Minecraft World by several ways:

• Path to a level.dat file, or its open file-like stream object;
• Path to a world directory, containing the level.dat file at its root, as in the example below;
• World name, i.e, the directory basename of a world in the platform-dependent default Minecraft saves/ path. By default, it is the in-game world name.

>>> import mcworldlib as mc
>>> world = mc.load('data/New World')
>>> # Most classes have a pretty print. In many cases, their NBT data.
>>> mc.pretty(world.level)
{
    Data: {
        WanderingTraderSpawnChance: 25,
        BorderCenterZ: 0.0d,
        Difficulty: 2b,
        ...
        SpawnAngle: 0.0f,
        version: 19133,
        BorderSafeZone: 5.0d,
        LastPlayed: 1633981265600L,
        BorderWarningTime: 15.0d,
        ScheduledEvents: [],
        LevelName: "New World",
        BorderSize: 59999968.0d,
        DataVersion: 2730,
        DataPacks: {
            Enabled: ["vanilla"],
            Disabled: ["Fabric Mods"]
        }
    }
}

World.dimensions is a dictionary mapping each dimension to categorized Region files:

>>> mc.pretty(world.dimensions)
{   <Dimension.OVERWORLD: 0>: {   'entities': <Regions(6 regions)>,
                                  'poi': <Regions(0 regions)>,
                                  'region': <Regions(6 regions)>},
    <Dimension.THE_NETHER: -1>: {   'entities': <Regions(0 regions)>,
                                    'poi': <Regions(0 regions)>,
                                    'region': <Regions(0 regions)>},
    <Dimension.THE_END: 1>: {   'entities': <Regions(0 regions)>,
                                'poi': <Regions(0 regions)>,
                                'region': <Regions(0 regions)>}}

And World.regions is handy view of that dictionary containing only the 'region' category, similarly with World.entities and World.poi:

>>> mc.pretty(world.regions)
{   <Dimension.OVERWORLD: 0>: <Regions(6 regions)>,
    <Dimension.THE_NETHER: -1>: <Regions(0 regions)>,
    <Dimension.THE_END: 1>: <Regions(0 regions)>}

>>> regions = world.regions[mc.OVERWORLD]
>>> regions is world.dimensions[mc.OVERWORLD]['region']
True

Regions is a dict-like collection of .mca Anvil region files, grouped in "categories" that match their sub-folder in a given the dimension, such as /entities, /poi, and of course /region.

The dictionary keys are region coordinate tuples, and the values represent Region files. Files are lazily loaded, so initially the values contain only their path:

>>> mc.pretty(regions)
{   ( -2, -1): PosixPath('data/New World/region/r.-2.-1.mca'),
    ( -2,  0): PosixPath('data/New World/region/r.-2.0.mca'),
    ( -1, -1): PosixPath('data/New World/region/r.-1.-1.mca'),
    ( -1,  0): PosixPath('data/New World/region/r.-1.0.mca'),
    (  0, -1): PosixPath('data/New World/region/r.0.-1.mca'),
    (  0,  0): PosixPath('data/New World/region/r.0.0.mca')}

They are automatically loaded when you first access them:

>>> regions[0, 0]
<RegionFile(r.0.0.mca: 167 chunks)>

A RegionFile is a dictionary of chunks, and each Chunk contains its NBT data:

>>> region = regions[-2, 0]
>>> mc.pretty(region)
{
    (  18,   0): <Chunk [18,  0] from Region ( -2,  0) in world at ( -46,   0) saved on 2021-10-11 16:39:17>,
    (  28,   0): <Chunk [28,  0] from Region ( -2,  0) in world at ( -36,   0) saved on 2021-10-11 16:40:50>,
    (  29,   0): <Chunk [29,  0] from Region ( -2,  0) in world at ( -35,   0) saved on 2021-10-11 16:40:50>,
    ...
    (  29,  31): <Chunk [29, 31] from Region ( -2,  0) in world at ( -35,  31) saved on 2021-10-11 16:40:14>,
    (  30,  31): <Chunk [30, 31] from Region ( -2,  0) in world at ( -34,  31) saved on 2021-10-11 16:40:14>,
    (  31,  31): <Chunk [31, 31] from Region ( -2,  0) in world at ( -33,  31) saved on 2021-10-11 16:40:14>
}

>>> chunk = region[30, 31]
>>> mc.pretty(chunk)  # alternatively, print(chunk.pretty())
{
    Level: {
        Status: "structure_starts",
        zPos: 31,
        LastUpdate: 4959L,
        InhabitedTime: 0L,
        xPos: -34,
        Heightmaps: {},
        TileEntities: [],
        Entities: [],
        ...
    },
    DataVersion: 2730
}

You can fetch a chunk by several means, using for example:

• Its key in their region dictionary, using relative coordinates, as the examples above.
• Their absolute (cx, cz) chunk position: world.get_chunk((cx, cz))
• An absolute (x, y, z) world position contained in it: world.get_chunk_at((x, y, z))
• The player current location: world.player.get_chunk()

>>> for chunk in (
...     world.get_chunk((-34, 21)),
...     world.get_chunk_at((100, 60, 100)),
...     world.player.get_chunk(),
... ):
...     print(chunk)
...
<Chunk [30, 21] from Region ( -2,  0) in world at ( -34,  21) saved on 2021-10-11 16:40:50>
<Chunk [ 6,  6] from Region (  0,  0) in world at (   6,   6) saved on 2021-10-11 16:40:50>
<Chunk [18,  0] from Region ( -1,  0) in world at ( -14,   0) saved on 2021-10-11 16:40:48>

Get the block info at any coordinate:

>>> block = world.get_block_at((100, 60, 100))
>>> print(block)
Compound({'Name': String('minecraft:stone')})

Remember the automatic, lazy-loading feature of Regions? In the above examples a few chunks from distinct regions were accessed. So what is the state of the regions dictionary now?

>>> mc.pretty(regions)
  {   ( -2, -1): PosixPath('data/New World/region/r.-2.-1.mca'),
      ( -2,  0): <RegionFile(r.-2.0.mca: 133 chunks)>,
      ( -1, -1): PosixPath('data/New World/region/r.-1.-1.mca'),
      ( -1,  0): <RegionFile(r.-1.0.mca: 736 chunks)>,
      (  0, -1): PosixPath('data/New World/region/r.0.-1.mca'),
      (  0,  0): <RegionFile(r.0.0.mca: 167 chunks)>}

As promised, only the accessed region files were actually loaded, automatically.

Editing world data

Reading and modifying the Player's inventory is quite easy:

>>> inventory = world.player.inventory  # A handy shortcut
>>> inventory is world.level['Data']['Player']['Inventory']
True
>>> # Easily loop each item as if the inventory is a list. In fact, it *is*!
>>> for item in inventory:
...     print(f"Slot {item['Slot']:3}: {item['Count']:2} x {item['id']}")
Slot   0:  1 x minecraft:stone_axe
Slot   1:  1 x minecraft:stone_pickaxe
Slot   2:  1 x minecraft:wooden_axe
Slot   3:  1 x minecraft:stone_shovel
Slot   4:  1 x minecraft:crafting_table
Slot   5: 37 x minecraft:coal
Slot   6:  8 x minecraft:dirt
Slot  11:  2 x minecraft:oak_log
Slot  12:  5 x minecraft:cobblestone
Slot  13:  2 x minecraft:stick
Slot  28:  1 x minecraft:wooden_pickaxe

How about some diamonds? Get 64 blocks of it in each one of your free inventory slots!

>>> backup = mc.List[mc.Compound](inventory[:])  # soon just inventory.copy()
>>> free_slots = set(range(36)) - set(item['Slot'] for item in inventory)
>>> for slot in free_slots:
...     print(f"Adding 64 blocks of Diamond to inventory slot {slot}")
...     item = mc.Compound({
...         'Slot':  mc.Byte(slot),
...         'id':    mc.String('minecraft:diamond_block'),  # Sweet!
...         'Count': mc.Byte(64),  # Enough for you?
...     })
...     inventory.append(item)  # Yup, it's THAT simple!
...
Adding 64 blocks of Diamond to inventory slot 7
Adding 64 blocks of Diamond to inventory slot 8
Adding 64 blocks of Diamond to inventory slot 9
Adding 64 blocks of Diamond to inventory slot 10
Adding 64 blocks of Diamond to inventory slot 14
...
Adding 64 blocks of Diamond to inventory slot 35

>>> # Go on, we both know you want it. I won't judge you.
>>> world.save('data/tests/diamonds')

>>> # Revert it so it doesn't mess with other examples
>>> world.player.inventory = backup

Have fun, you millionaire!

More fun things to do:

>>> chunks = world.entities[mc.OVERWORLD][0, 0]
>>> for chunk in chunks.values():
...     for entity in chunk.entities:
...         print(entity)
...
Chest Minecart at (  81,  18,  21)
Chest Minecart at (  80,  18,  37)
Chest Minecart at (   2,  38, 112)
Sheep at (  36,  70, 116)
Sheep at (  33,  69, 120)
Sheep at (  37,  70, 116)
Item: 3 String at (  14,  25, 152)
Item: 2 String at (  14,  25, 153)
Chicken at (  13,  64, 158)
Chicken at (  12,  64, 156)
Chicken at (   7,  64, 153)
Item: 1 String at (   0,  35, 167)
Cow at (   1,  65, 184)
Cow at (  11,  64, 186)
Chest Minecart at (  17,  32, 187)
Item: 3 String at (  39,  35, 195)
Donkey at (  56,  70, 202)
Donkey at (  57,  71, 203)
Donkey at (  56,  70, 201)
Chicken at (   6,  64, 217)

How about some NBT Explorer nostalgia?

>>> mc.nbt_explorer(world.level)
⊟ Data: 42 entries
├──⊞ CustomBossEvents: 0 entries
├──⊟ DataPacks: 2 entries
│  ├──⊟ Disabled: 1 entry
│  │  ╰─── 0: Fabric Mods
│  ╰──⊟ Enabled: 1 entry
│     ╰─── 0: vanilla
...
├──⊟ Player: 37 entries
│  ├──⊟ abilities: 7 entries
│  │  ├─── flying: Byte(0)
...
│  │  ╰─── walkSpeed: Float(0.10000000149011612)
│  ├──⊟ Brain: 1 entry
│  │  ╰──⊞ memories: 0 entries
...
│  ├──⊟ Inventory: 11 entries
│  │  ├──⊟  0: 4 entries
│  │  │  ├──⊟ tag: 1 entry
│  │  │  │  ╰─── Damage: Int(0)
│  │  │  ├─── Count: Byte(1)
│  │  │  ├─── id: minecraft:stone_axe
│  │  │  ╰─── Slot: Byte(0)
...
│  │  ╰──⊟ 10: 4 entries
│  │     ├──⊟ tag: 1 entry
│  │     │  ╰─── Damage: Int(18)
│  │     ├─── Count: Byte(1)
│  │     ├─── id: minecraft:wooden_pickaxe
│  │     ╰─── Slot: Byte(28)
...
│  ├─── XpTotal: Int(37)
│  ╰──⊕ UUID: 4 entries
├──⊟ Version: 3 entries
│  ├─── Id: Int(2730)
│  ├─── Name: 1.17.1
│  ╰─── Snapshot: Byte(0)
...
├──⊞ ScheduledEvents: 0 entries
├──⊟ ServerBrands: 1 entry
│  ╰─── 0: fabric
├─── allowCommands: Byte(0)
...
├─── WanderingTraderSpawnDelay: Int(19200)
╰─── WasModded: Byte(1)

You want to click that tree, don't you? Sweet Array "icon" for UUID!

Test yourself all the examples in this document:

python3 -m doctest -f -o ELLIPSIS -o NORMALIZE_WHITESPACE README.md
git checkout data/

---

Contributing

Patches are welcome! Fork, hack, request pull! Here is a succinct to-do list:

• Better documentation: Improve this README, document classes, methods and attributes, perhaps adding sphinx-like in-code documentation, possibly hosting at Read the Docs. Add more in-depth usage scenarios.

• Installer: Test and improve current setup.cfg, possibly uploading to Pypi.

• Tests: Expand doctest usage, add at least unittest.

• Semantics: Give semantics to some NBT data, providing methods to manipulate blocks, entities and so on.

• CLI: Add a command-line interface for commonly used operations.

See the To-Do List for more updated technical information and planned features.

If you find a bug or have any enhancement request, please open a new issue

Author

Rodrigo Silva (MestreLion) [email protected]

License and Copyright

Copyright (C) 2019 Rodrigo Silva (MestreLion) <[email protected]>.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.

mcworldlib's Issues

I cant undertand how to check for specific blocks in chunk

Hello.Thanks for lib,by the way i cant undertand how its works,i am new at python,so can you please give example of code for checking for specific block inside chunk,please? My minecraft version is 1.12.2,if its important.

If a region file contains a huge number of entities, it is determined unreadable by mcworldlib

What originally led me to this fantastic library was the fact that I had a world that kept freezing when certain areas loaded, and I needed a CLI/programming tool to try to inspect and fix it, because NBTExplorer was also freezing once I'd found the offending chunk and tried to drill into the chunk properties.

What I was able to see in NBTExplorer was that the offending chunk had 40233 entities listed. As it turns out, making a command block that sprays item entities (which I'd configured to have a lifespan of ~10 ticks) gets leaky in certain circumstances.

When I tried to load the region containing the offending chunk, this is the output I got:

Python 3.8.2 (default, Apr  8 2020, 14:31:25) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mcworldlib as mc
>>> w = mc.load('new red castle broken')
>>> w.regions[-2,-1]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ian/PycharmProjects/mcworldlib/mcworldlib/region.py", line 67, in __getitem__
    self._regions[key] = RegionFile.load(region)
  File "/home/ian/PycharmProjects/mcworldlib/mcworldlib/region.py", line 114, in load
    return cls.parse(open(filename, 'rb'))
  File "/home/ian/PycharmProjects/mcworldlib/mcworldlib/region.py", line 150, in parse
    assert chunk.sector_count <= sector_count <= chunk.sector_count + 1, \
AssertionError: Length mismatch for region (-2, -1) in chunk (7, 15): region header declares 5 4096-byte sectors, but chunk data required 213.

Ultimately, just deleting and recreating the Entities list tag via NBTExplorer fixed the issue, and mcworldlib is now able to read the region, but I still wonder if this is indicative of a bug in this library.

Regions aren't properly loaded when loading Beta 1.7.3 world

When attempting to load a beta 1.7.3 world it is processed correctly without errors but the regions aren't loaded

import mcworldlib as mc

world = mc.load("<ABSOLUTE PATH>\Beta Test World 1.7.3")

mc.pretty(world.level) # Outputs fine

mc.pretty(world.regions) 

# Output:
#{   <Dimension.THE_END: 1>: <Regions(0 regions)>,
#    <Dimension.THE_NETHER: -1>: <Regions(0 regions)>,
#   <Dimension.OVERWORLD: 0>: <Regions(0 regions)>}

Inside the region folder of the world there are 3 mcr region files (r.0.0.mcr, r.0.-1.mcr and r.-1.-1.mcr)

Loading the region directly with r = mc.load_region("./region/r.0.0.mcr") works but attempting to process the chunks of the region results in an error:

r = mc.load_region("./region/r.0.0.mcr")
list(list(r.chunks)[0].get_blocks())

Produces:

Traceback (most recent call last):
  File "C:\Users\<USER>\AppData\Local\Programs\Python\Python311\Lib\site-packages\mcworldlib\chunk.py", line 54, in get_blocks
    for section in self.data_root['Sections']:
                   ~~~~~~~~~~~~~~^^^^^^^^^^^^
  File "C:\Users\<USER>\AppData\Local\Programs\Python\Python311\Lib\site-packages\nbtlib\tag.py", line 1167, in __getitem__
    return super().__getitem__(key)
           ^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'Sections'

That i need to change to make it work with 1.12.2 worlds ?

Hello,thank for lib,but do you know that i need to change to make it work with 1.12.2 worlds?

should COMPRESSION_NONE be 3?

in your code it is 0.
i m not sure but i think that it should be 3.

Static OS

Looks for ~/.minecraft/saves which is default for Linux. Windows is saved in ~/AppData/Roaming/.minecraft/saves

Propose following fix:

REQUEST: provide a bool arg to `get_chunks` that doesn't cache the data

I'l like to iterate over all chunks in a world, but since the Regions hold on to the data, my RAM goes poof. Is there a way to "uncache" the data as I go along?

get_block_at((x, y, z)) returns None

I loaded the region the block is in, yet it returns None
This happens every time I call get_block_at()

Keep NBT-reading functionality when upgrading to newer `nbtlib`

As a consequence of many important re-structuring in nbtlib, specially in the 2.x series, some backward-compatibility was lost and at its current state it can't read some .dat files it used to.

Meanwhile, mcworldlib is still stuck with an older 1.6.3 nbtlib, as it was the last version to support Python 3.6 which is still supported by mcworldlib.

This issue was created to track down this versioning and dependencies issues: bump up the minimum Python requirement to 3.7 (or skip directly to 3.8), so we can bump up the maximum compatible nbtlib as high as possible while still avoiding any NBT-reading breaking changes.

Case in point: mcworldlib must be able to handle this test file without any issues, as requested by @Netherwhal in vberlier/nbtlib#145:

$ git clone https://github.com/MestreLion/mcworldlib
$ cd mcworldib
$ pip3 install -e .
$ python3

>>> import mcworldlib as mc
>>> data = mc.nbt.load_dat('data/6d26bff4.dat')
>>> mc.nbt_explorer(data)
⊟ abilities: 7 entries
├─── flying: 0b
├─── flySpeed: 0.05000000074505806f
├─── instabuild: 0b
├─── invulnerable: 0b
├─── mayBuild: 1b
├─── mayfly: 0b
╰─── walkSpeed: 0.10000000149011612f
⊟ Brain: 1 entry
╰──⊞ memories: 0 entries
⊟ bukkit: 8 entries
├─── expToDrop: 0
...

Block storage format in chunks slightly changed

Technical Changes in 20w17a

Block storage format in chunks slightly changed to speed up various tasks (rendering, pathfinding, world generation, etc)

Memory grows out of hand

Issue

As of now, the region files are lazy loaded, but they never actually get offloaded, which may lead to Out Of Memory exceptions on smaller machines or for larger maps.

Not sure whether this affects many people but I thought it'd be a nice to have.

Expected behaviour

Regions are offloaded after they are queried, or maybe after N regions are loaded, the N-th + 1 pushed the oldest one out of memory.

implementation ideas

Instead of using a dict for storing dimensions' Regions, there could be a class which inherits from Dict but which __getitem__ is overridden and returns an instance of a realized RegionFile without storing it, or something like that, essentially always keeping the LazyLoadFileMap version of the region in the dict rather than the RegionFile itself.

To reproduce

import psutil
import mcworldlib as mc

world = mc.load(level_directory_path)
regions = world.regions[mc.OVERWORLD]

process = psutil.Process()

print(f'start memory: {process.memory_info().rss / 1024 / 1024}MB') 
for region_coords in regions:
     region = regions[region_coords]
     print(process.memory_info().rss / 1024 / 1024, 'MB')

You should see the memory usage grow. like so:

start memory: 70.93359375MB
89.87109375 MB
111.43359375 MB
172.55859375 MB
233.49609375 MB
294.43359375 MB
355.55859375 MB
375.24609375 MB
436.37109375 MB
497.30859375 MB
519.05859375 MB
527.30859375 MB
588.43359375 MB
649.37109375 MB
669.05859375 MB
...