After training using your training code example, I have file .bin.
Can you tell me how to load it to use generate level?
Thank you.

I'm trying to use it, but get the error

File "python3.9/site-packages/mario_gpt/prompter.py", line 113, in output_hidden
self.feature_extraction(prompt, return_tensors="pt")[0]
AttributeError: 'list' object has no attribute 'mean'

code:

from mario_gpt import MarioLM, SampleOutput

# pretrained_model = shyamsn97/Mario-GPT2-700-context-length

mario_lm = MarioLM()

# use cuda to speed stuff up
# import torch
# device = torch.device('cuda')
# mario_lm = mario_lm.to(device)

prompts = ["many pipes, many enemies, some blocks, high elevation"]

# generate level of size 1400, pump temperature up to ~2.4 for more stochastic but playable levels
generated_level = mario_lm.sample(
    prompts=prompts,
    num_steps=1400,
    temperature=2.0,
    use_tqdm=True
)

# show string list
generated_level.level

# show PIL image
generated_level.img

# save image
generated_level.img.save("generated_level.png")

# save text level to file
generated_level.save("generated_level.txt")

# play in interactive
generated_level.play()

# run Astar agent
generated_level.run_astar()

# Continue generation
generated_level_continued = mario_lm.sample(
    seed=generated_level,
    prompts=prompts,
    num_steps=1400,
    temperature=2.0,
    use_tqdm=True
)

# load from text file
loaded_level = SampleOutput.load("generated_level.txt")

# play from loaded (should be the same level that we generated)
loaded_level.play()

Can you check what's wrong with it, or I'm doing something wrong

Hi, very interesting project! I'm trying to reproduce the results training from base and running into a problem. Using the training notebook with default parameters on 20k steps, the model is converging to a loss of ~0.05. I'm getting reasonable looking outputs sampling from this trained model but the characters look wrong:

Any ideas on what's going wrong here?

This is the script:

from mario_gpt import MarioLM

mario_lm = MarioLM()

prompts = ["many pipes, many enemies, some blocks, high elevation"]

generated_level = mario_lm.sample(
prompts=prompts,
num_steps=100,
temperature=2.0,
use_tqdm=True
)

play in interactive

generated_level.play()

run Astar agent

generated_level.run_astar()

And I need more num_steps, but the AI lags when it reach more than 100 steps... its possible to get performance with this python library?

I don't see the Wiki enabled in this repo or I'd suggest putting them there. Maybe a new section at the bottom of the README so people can get a general idea of how long generation takes?

This was with the default example provided in the README

#MarioGPU Benchmarks
## CPU (Intel i7-12700K)
Execution time: 293.0745s

## CPU (AMD Ryzen 9 5950X)
Execution time: 384.3558s

## CPU (Intel Xeon E-2176G (2 cores))
Execution time: 751.6939s

## CUDA (Nvidia Quadro P620)
Execution time: 186.1026s

## CUDA (Nvidia 3070)
Execution time: 21.4382s

## CUDA (Nvidia 3090 Ti)
Execution time: 11.7694s

Thanks for your great work. I tried to implement MarioBERT myself, but the inpainting results were not good. I used the pre-trained model "shyamsn97/MarioBert-448-inpaint-context-length" you provided. I would appreciate it if you can give me some suggestions.

After training the model and generating levels according to the instructions I got a level like this.

I wrote the code on google colab. Please help me to solve this problem, thanks a lot.

Hi,

First of all thanks for your novel implementation! This is very cool to see.

When running the minimal code snippet provided in the readme, I got the following error:

C:\Users\Mossly\Desktop>py mariotest.py
Traceback (most recent call last):
  File "C:\Users\Mossly\Desktop\mariotest.py", line 1, in <module>
    from mario_gpt import MarioLM
ImportError: cannot import name 'MarioLM' from 'mario_gpt' (C:\Users\Mossly\AppData\Local\Programs\Python\Python310\lib\site-packages\mario_gpt\__init__.py)

I resolved it by changing the line:
from mario_gpt import MarioLM
to
from mario_gpt.lm import MarioLM

Perhaps this is just a simple syntactical mistake? Or it could be user error on my part...

Thanks for your attention!

Hello,

I have been using the Training notebook provided in this repository to train my model, and I've encountered an issue where the performance of the trained model is significantly subpar.

Hi!
i love it!
will there be new enemies, levels and maybe midi music for it?

kind regards

Hello,
Your project is awesome, and I'm delighted to have such a fantastic project.
When I cloned your project and trained it myself, I tried to save the tokenizer like save model, but I found the tokenizer_config.json and tokenizer.json not like your config. I don't know how to resize the vocab from 50256 to 256 and set "endoftext" token to id = 0. Could you give me some tips how to fine-tune the tokenizer? I did this because when I ran trained model, I have to use your tokenizer, if I use the tokenizer that I saved decode will go wrong.

This is my tokenizer_config.json file

This is my tokenizer.json file

We are making an open source, tile-based game with fairly similar constraints as super Mario:
https://github.com/fishfolk/jumpy/

We’ve also made progress on procedural level generation using the Wave Function Collapse technique:
fishfolk/jumpy#512

I’d love to chat about ways in which we might be able to collaborate ☺️ happy to talk more here, on Discord or by email.

Hi, i did try to use your exemple but it doesnt works. See the message on the screenshot.

you example script

from mario_gpt.lm import MarioLM
from mario_gpt.utils import view_level, convert_level_to_png

pretrained_model = shyamsn97/Mario-GPT2-700-context-length

mario_lm = MarioLM()

prompts = ["many pipes, many enemies, many blocks, high elevation"]

generate level of size 700, pump temperature up to ~2.4 for more stochastic but playable levels

generated_level = mario_lm.sample(
prompts=prompts,
num_steps=700,
temperature=2.0,
use_tqdm=True
)

show string list

view_level(generated_level, mario_lm.tokenizer)
convert_level_to_png(generated_level, "generated_level.png", mario_lm.tokenizer)

I think I meet one problem about evaluation.I am looking forward to your help.

tracker = self.accelerator.get_tracker("tensorboard")
        tracker.add_image(
            "image", np.array(out.img), i, dataformats="HWC"
        )
except Exception as e:
    print("Failed to evaluate!", e)

Besides,I can always see this output during the trainning process,should I deal with this problem?

REPLACING <PIL.PngImagePlugin.PngImageFile image mode=RGB size=16x16 at 0x16D2A446C10> (96, 13)
REPLACING <PIL.PngImagePlugin.PngImageFile image mode=RGB size=16x16 at 0x16D2A446C10> (97, 13)
REPLACING <PIL.PngImagePlugin.PngImageFile image mode=RGB size=16x16 at 0x16D2A446C10> (98, 13)
REPLACING <PIL.PngImagePlugin.PngImageFile image mode=RGB size=16x16 at 0x16D2A446C10> (99, 13)
REPLACING <PIL.PngImagePlugin.PngImageFile image mode=RGB size=16x16 at 0x16D2A446C10> (100, 13)

Thank you!

This happens randomly when generating a level.

Using the prompts: no blocks, no pipes, many goombas, fireball

shape: torch.Size([1, 678]), torch.Size([1, 1393]) first: 56, last: 51:  99%|██████████████████████████████████████████████████████████████████▌| 1392/1400 [02:58<00:01,  7.82it/s]Traceback (most recent call last):
  File "/home/me/apps/mariogpt/capturePlay.py", line 38, in <module>
    generated_level = mario_lm.sample(                                                                                                                                                File "/home/me/apps/mariogpt/lib/python3.10/site-packages/mario_gpt/lm/gpt.py", line 54, in sample
    return sampler(
  File "/home/me/apps/mariogpt/lib/python3.10/site-packages/mario_gpt/sampler.py", line 248, in __call__
    return self.sample(*args, **kwargs)
  File "/home/me/apps/mariogpt/lib/python3.10/site-packages/mario_gpt/sampler.py", line 223, in sample
    next_tokens, encoder_hidden_states = self.step(
  File "/home/me/apps/mariogpt/lib/python3.10/site-packages/mario_gpt/sampler.py", line 158, in step
    out = self.mario_lm.lm(
  File "/home/me/apps/mariogpt/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/me/apps/mariogpt/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1043, in forward
    transformer_outputs = self.transformer(
  File "/home/me/apps/mariogpt/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/me/apps/mariogpt/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 887, in forward
    outputs = block(
  File "/home/me/apps/mariogpt/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/me/apps/mariogpt/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 388, in forward
    attn_outputs = self.attn(
  File "/home/me/apps/mariogpt/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/me/apps/mariogpt/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 329, in forward
    attn_output, attn_weights = self._attn(query, key, value, attention_mask, head_mask)
  File "/home/me/apps/mariogpt/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 216, in _attn
    attn_output = torch.matmul(attn_weights, value)
RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasSgemmStridedBatched( handle, opa, opb, m, n, k, &alpha, a, lda, stridea, b, ldb, strideb, &beta, c, ldc, stridec, num_batches)`

I'm using:

generated_level = mario_lm.sample(
    prompts=prompts,
    num_steps=1400,
    #num_steps=100,
    temperature=2.0,
    use_tqdm=True
)

This was happening less frequently in 0.1.2 it feels like. I just upgraded to 0.1.3.

It immidiatly closes python when i run setup.py

HI!

I saw your paper; I think it's a fantastic project! I noticed that there isn't a colab notebook available for people to try out the code. So I took the liberty of creating one. Here's the link to the notebook

I am also happy to contribute via PR; just let me know how you would prefer.

May I ask how to use the MarioBert in this repo?
I tried to call MarioLM(mask_model=True), but MarioBert has no sample method:

    generated_level = mario_lm.sample(
AttributeError: 'MarioBert' object has no attribute 'sample'

Hello, I want to evaluate the model but i don't see the code about it, i still a student. Please give me some help

1. TrainingConfig and MarioGPTTrainer cannot be used.
2. mario_lm = MarioLM(lm=BASE, tokenizer=BASE) The parameter here prompts an error. Is there no specific program for this training?

Can you give some specific suggestions？

Ok I sense this will be a very stupid question.

so we generate AI Mario levels. perfect! How do we play them? :p

`ImportError Traceback (most recent call last)
in <cell line: 2>()
1 import torch
----> 2 from mario_gpt import MarioDataset, MarioLM, TrainingConfig, MarioGPTTrainer
3
4 # create basic gpt model
5 BASE = "distilgpt2"

ImportError: cannot import name 'TrainingConfig' from 'mario_gpt' (/usr/local/lib/python3.10/dist-packages/mario_gpt/init.py)`

Hi Shyam, I'm recently following this work and I want to make some extensions to the simulator. I want to know how you get the jar packages here? If the source code is not open-source, is it possible to send me a copy? Thanks in advance!