davidtjones / melee-env

A gym-esque environment for Super Smash Bros. Melee.

melee-env's Issues

Agents can potentially produce undesirable behavior during menu operations

This seems to happen primarily with fast forward turned on. I'm unsure what causes this, but it seems possible that the agent's final input is not properly flushed before getting back into the stage selection. This can cause problems when the menu selection agent tries to select the next stage. I believe this is fixed by 072f3da in the dev channel.

Allow for circumventing the action space so that "dumb" bots will work with melee-env

Problem

Currently, all agents are required to obtain controller inputs from the action space, which are then executed via ControllerState instances. While this behavior is desirable for learning agents, non-learning agents are unable to be implemented on melee-env. See this section of code.

Possible fix

Separate AI-agents into two groups, where learning agents (LA) utilize the action space and non-learning agents (NLA) are allowed to execute on their own controllers. This may be relatively simple.

Some thoughts

This would be a good point to reconsider whether the observation returned by MeleeEnv will be a numpy array or just the whole gamestate from libmelee, which may just make a lot more sense and allow LA's observations (probably heavily filtered/modified versions of libmelee's gamestate) to not affect other agents (currently all agents are forced to accept the same observation space).

One big issue here is that the current implementation can keep track of when players are defeated and provide the correct information to fill in to the observation. gamestate just drops the player entirely, which is problematic when the agent expects an observation of a given size. I think libmelee should really handle this better, but it may be a slippi thing. Idea: Agents can keep track of their last observation (last_observation), and perhaps there can be some indication to agents when other agents have been defeated so that rows from last_observation can be inserted into current_observation or something.

Separate the environment and AI work

The environment functions fairly well and should really be separate from the planned AI work. I'll be stripping out the unrelated work and then publishing this on pip.

Add support for stages other than FD

See title. Should be fairly simple to pass a stage parameter to "setup" to choose a specific stage for every episode, if desired.

2-player game won't start

Problem: When playing a game with two players, the game hangs at startup and doesn't continue.

Expected behavior: game should proceed to a two player game and proceed as normal

Related info:
Traceback

  File "agents_example.py", line 28, in <module>
    observation, reward, done, info = env.setup(enums.Stage.BATTLEFIELD)
  File "/home/david/bin/python/melee-env/melee_env/env.py", line 87, in setup
    self.gamestate = self.console.step()
  File "/home/david/.conda/envs/melee/lib/python3.8/site-packages/melee/console.py", line 360, in step
    message = self._slippstream.dispatch(self._polling_mode)
  File "/home/david/.conda/envs/melee/lib/python3.8/site-packages/melee/slippstream.py", line 65, in dispatch
    event = self._host.service(wait_time)
KeyboardInterrupt

Add windows support

Windows support seems pretty straightforward and there is already some structure in place. The windows unique steps will be

1. Download the .zip archive from project-slippi/Ishiiruka
2. Extract the contents to home/melee-env/Slippi/
3. Run the executable once to generate config files

Then the existing code should be able to handle the remaining configuration and gecko codes.

Add support for characters other than Fox

Currently every agent will be forced to choose Fox. It should be simple to designate which character the agent should choose by passing a parameter to the agent at initialization, e.g.,

agent = Random(enums.Characters.KIRBY, 4)