minimaxir / mtg-gpt-2-cloud-run

Code and UI for running a Magic card text generator API via GPT-2

mtg-gpt-2-cloud-run

Code and UI for running a Magic card text generator API using gpt-2-cloud-run. The card image in the UI is generated by mtg-card-creator-api. You can play with the API here.

It turns out finetuning GPT-2 overfits Magic cards very quickly due to its more-structured format. The workaround is to use the random field encoding option offered by mtgencoding (which generates a random order for each individual card after removing the random seed: see changes in mtg-card-creator-api), generate such encodings many times, and concatenate them together (e.g. via concat.py). As a bonus, this gives the network the ability to condition on any combination of fields as it'll see many different combinations of fields. (see encoding_examples.txt for examples of cards following this encoding)

The network was trained with GPT-2 117M via gpt-2-simple for 6500 steps (about 2 hours on a P100 GPU / $1). Despite that relatively low amount of training, GPT-2 still overfits on names and rules text!

You can see examples of card text generation at various temperatures in the generated_card_dumps folder. Temperatures 0.7 and 1.0 are mostly normal, 1.2 is when the network starts making its own mechanics/cards, 1.5 and 2.0 is when the network becomes MTGCardsmith.

Helpful Notes

• To share the generated card image, you can Save As the generated card locally, and to use it elsewhere, rename it and add a .jpg file extension.
• Since the network overfits, the temperature doesn't have a huge impact; a random temperature between 0.7 and 1.0 is used to make output more random/exciting.
• The network can recite existing card names and rules text of existing cards, but rarely to the same card. The network often makes interesting color shift decisions with changes to CMC/Rarity.
• In terms of color accuracy, color pie is mostly correct. Creature types and mechanics often follow the appropriate color identity. P/T, mana cost, and rarity are balanced.
• Occasionally, it does get card types wrong, e.g. P/T on noncreatures.

See Also

• char-embeddings — My first attempt at generating Magic cards with a neural network (GPT-2 is much better!).
• RoboRosewater — Twitter account which pioneered the use of text generation for Magic cards.
• DroidRosewater — Variant of RoboRosewater which allows user input (although it cheated a bit due to how the RNN generates). Credit to Shoghi for pointing me to the wine solution for generating cards!

Maintainer/Creator

Max Woolf (@minimaxir)

Max's open-source projects are supported by his Patreon. If you found this project helpful, any monetary contributions to the Patreon are appreciated and will be put to good creative use.

License

MIT

Disclaimer

This repo has no affiliation or relationship with OpenAI or Wizards of the Coast.

mtg-gpt-2-cloud-run's Issues

Auto Image Generation for Cards

Hello minimaxir : ),

I have a GAN which was trained on a corpus of magic cards and can generate images. I was wondering if you were interested in adding the ability to make custom machine generated pictures for the cards as well? If so let me know and I'd be happy to work on incorporating that :). Super cool repository thanks for sharing!

No such file or directory: 'checkpoint/run1/hparams.json'

Missing file?

Would be nice to have an install document, btw.

Is the model behind the current site you use still 117M with 6500 steps?

First of all, I love the work you've done. I've done so much with the repos you've created around GPT-2. Last week I generated a cube of GPT-2 generated MtG cards (my friends and I came up with ~900 names to seed the generator), and the draft of it was a total blast.

Getting to the point, I was mainly wondering if the current model you use on your site that generates MtG cards was still what's in the README. I ask because I really like the cards that generates, and I felt my model (345M w/ 6500 steps) wasn't quite where I wanted it to be.

I'm training it on the entirety of MtG (I even have it training on un-sets and celebration for extra interesting stuff), using the same method you did (using mtgencode to generate the encoded strings and randomizing the field order). I'm going to try 117M next to see how that affects the output before trying other things, but I was wondering if you could possibly expand further on how you've trained your model.

Oh, another thing I tweaked: I edited mtgencode to include flavor text in the encoding. HIGHLY recommend it. The result is hysterical.

Thanks again for all your awesome work!

No matter what I do I get the error "Unfortunately, the AI created an invalid card. Please try again!"

Does anybody know how to fix this?