Memory use about tuxemon

[ShadowApex]

Tuxemon memory usage is definitely something I'd like to keep minimal if we can, primarily because of its possible use on the GCW Zero gaming platform. The GCW Zero is a small open source handheld which has only 512MB of RAM. Right now I've observed Tuxemon use as little as ~50MB to as much as ~250MB in my tests:

• loading all images, sounds, etc at startup (as opposed to when needed)

Currently I have made some efforts to load only on-demand, but there could certainly be some improvements to the way things are loaded and released. I do have the methods from the Mek's state framework to load all the assets at the start of the game, but I'm not currently using them. The only pre-loading I'd eventually like to do would be for conjoining maps in order to make transitions from map-to-map relatively seamless.

• making a spider-web of references to game objects...making it impossible to GC things

I do think that Tuxemon may have a similar number of spider-web like references as a result of the evolution of the project over 4-5 years, which may make it a somewhat daunting task. It might still be possible to make some improvements though.

I also helped develop a state 'plugin' system, so states can be added or removed without modifying code. Just drop in the .py file.

I've also done some work on a 'plugin' type system in Tuxemon, mostly for being able to just drop in new event actions and conditions in a new file. Doing something similar for states also sounds beneficial, so long as we can keep it compatible with all the platforms that I'd like to target (Linux, Mac, Windows, Android, & GCW Zero). I know I had to do some extra work to get the plugin system working with all platforms.

I'll definitely check out your changes to the pyroller project. I think Tuxemon could benefit from those kinds of changes.

[bitcraft]

Unless anyone objects, I'd like to make some changes to tuxemon's state system that reduce cyclical dependencies, and make all game states 'plugins'. One nice advantage of this is that states, like world, or combat can be their own modules and code can be split into files with it...instead of spreading it all over the codebase.

What branch should I work from, development or master?

[ShadowApex]

No objections here! I think having a more modular state system would be a huge advantage.

All of the latest development is done on the development branch, so that'd be a good place to work from. I've set up the master branch for "stable" releases of Tuxemon, which I periodically merge from the development branch when I've had a chance to test the new functionality on various platforms.

[bitcraft]

I've encountered a few problems while digging through the code related to this state issue. It seems that many states, especially the combat state, expect the World state to be living while combat is happening. In my perfect world, only one state would be active at a time, but with this situation, its not possible to make it happen easily.

The good news is that the combat state seems to just be setting values related to state transition and what not. That means there just needs to be a standardized API for state transitions and most of these issues will be easy to fix.

So, here's my suggestions, and maybe somebody could pick this up:

• Move all existing state transition stuff, like fades, into its own state.
• Make API for Combat or World that reduces the need for global or semi-global state (like diggin in each other's attributes)

While I realize it probably sucks ("Hey look at the bitcraft guy, coming in here telling us what do do!"), it will make things better in the long run. Generalized state transitions will remove the duplicated code handling it, and a more expressive API for setting combat and world values will make testing easier. When done, memory will be reduced as well. Have a great weekend!

(to clarify, I can do the aforementioned changes, but as a new guy, I don't want to change up too much stuff so that my PR will be a difficult merge...i'm still up for it though.)

[ShadowApex]

I agree with you on all points. I also think having a separate state for various transitions is a great idea.

Creating that standardized API seems to be an interesting challenge. As you said, there will need to be a system in place to pass the attributes needed to set up each individual state (e.g. the player and opponent objects passed to the Combat state, etc.), which could be done in a variety of different ways. I'm not sure which approach would be best.

Here's sort of what I was thinking:

The core.tools.Control object, which manages the states, could have an extra attribute to hold the parameters needed to initialize the next state, and then pass those parameters to the state on initialization during the flip_state method. Then we could do something like this when we want to switch states, like for example, when we start a battle, instead of:

Current Method

# 'game' object is an instance of core.tools.Control
game.next = "COMBAT"
game.done = True

We would do something like this instead:

New Method

game.next = "COMBAT"
game.next_params = {"player": player1, "opponent": player2, "combat_type": "boss"}
game.done = True

Then in the flip_state method, I can pass these parameters to the next state, creating a new instance of the state if we modify the state_dict to have references to the state classes instead of an instance of a state:

core/main.py

state_dict = {"START": start.StartScreen,
              "WORLD": world.World,
              "COMBAT": combat.Combat,
              "PC": pc.PC,
              "TRANSITION": transition.Transition}

core/tools.py

def flip_state(self):
    previous, self.state_name = self.state_name, self.state.next
    persist = self.state.cleanup()
    self.state = self.state_dict[self.state_name](self, self.next_params)
    self.state.startup(self.current_time, persist)
    self.state.previous = previous

This would make it so states only communicate through the core.tools.Control object, thus freeing up the references when a state is not in use.

[bitcraft]

This sounds like a good plan. I've already made that change to the
state_dict, and state flipping is working on my dev. branch, so we have the
same idea there. The next thing to do would be handling transitions. I'll
compile a list of things blocking me a little later today.

On Sat, Nov 28, 2015 at 2:18 AM, William Edwards [email protected]
wrote:

I agree with you on all points. I also think having a separate state for
various transitions is a great idea.

Creating that standardized API seems to be an interesting challenge. As
you said, there will need to be a system in place to pass the attributes
needed to set up each individual state (e.g. the player and opponent
objects passed to the Combat state, etc.), which could be done in a variety
of different ways. I'm not sure which approach would be best.

Here's sort of what I was thinking:

The core.tools.Control object, which manages the states, could have an
extra attribute to hold the parameters needed to initialize the next state,
and then pass those parameters to the state on initialization during the
flip_state method. Then we could do something like this when we want to
switch states, like for example, when we start a battle, instead of:
Current Method

'game' object is an instance of core.tools.Control

game.next = "COMBAT"
game.done = True

We would do something like this instead:
New Method

game.next = "COMBAT"
game.next_params = {"player": player1, "opponent": player2, "combat_type": "boss"}
game.done = True

Then in the flip_state method, I can pass these parameters to the next
state, creating a new instance of the state if we modify the state_dict
to have references to the state classes instead of an instance of a state:

core/main.py

state_dict = {"START": start.StartScreen,
"WORLD": world.World,
"COMBAT": combat.Combat,
"PC": pc.PC,
"TRANSITION": transition.Transition}

core/tools.py

def flip_state(self):
previous, self.state_name = self.state_name, self.state.next
persist = self.state.cleanup()
self.state = self.state_dict[self.state_name](self, self.next_params)
self.state.startup(self.current_time, persist)
self.state.previous = previous

This would make it so states only communicate through the
core.tools.Control object, thus freeing up the references when a state is
not in use.

—
Reply to this email directly or view it on GitHub
#58 (comment).

[bitcraft]

For deeply nested menus, or complex state transitions, it makes sense to organize them in a stack. I'm going down this route now, but it would involve new API for the Control class. Currently, state are changed by setting State.next and State.done. This means states are either 'done' or 'not done', and there isn't an opportunity to pause and resume states.

Imagine being in the world state. You don't want to destroy the state just to pull up a menu. You'd like to pause the world, open the menu, then let the menu go back to the world state.

Currently, this is accomplished by "self.next = self.previous", but I need to remove that, as it isn't clean design...States shouldn't holds refs to other states, and they shouldn't know the order they are created in...that is for the Control class to determine...with a stack.

So, here is my proposal:

class Control:
    def push_state(self, state_name, params=None):
        """ Push new state onto state stack.  State will change next loop.
            The previous state will be paused.

        :param state_name: name of next state
        :param params: dict of config to pass to state
        :return: None
        """

    def pop_state(self):
        """ Pop the currently running state.  The previously running state will resume.

        :return: 
        """

class State:
    def pause(self):
        """ Called when state is pushed back in the stack, allowed to pause

        :return: 
        """

    def reusme(self):
        """ Called after state was paused, but is now at top of stack again

        :return: 
        """

Example:

# all States will have a reference to the "game" or "Control"
State.control

# within States....
# open a menu, pause the current state
self.control.push_state("menu", params)

# resume previous state
self.control.pop_state()

# open a new state, replace the current state
self.control.pop_state()
self.control.push_state("new state", params)

This method removes the use of State.next and State.done, and makes cleaner design.

Thoughts?

[ShadowApex]

I think this is pretty good. It provides a good way to push state parameters and creates an easy-to-follow stack of states. With the current system, the flow of states is a harder to follow since you have to dig into each state's next and previous attributes to determine the order that states were changed. I can't think of anything I would change with this proposed system.

[bitcraft]

New state system is in place, nice work. I think that the next goal to reduce memory use would be to audit references to pygame surfaces and make sure they are GC'd when not needed. This is something I can look into.

[ShadowApex]

Sounds good!