zuoxingdong / mazelab

A customizable framework to create maze and gridworld environments

mazelab: A customizable framework to create maze and gridworld environments.

This repository contains a customizable framework to create maze and gridworld environments with gym-like API. It has modular designs and it allows large flexibility for the users to easily define their own environments such as changing rendering colors, adding more objects, define available actions etc. The motivation of this repository is, as maze or gridworld are used very often in the reinforcement learning community, however, it is still lack of a standardized framework.

The repo will be actively maintained, any comments, feedbacks or improvements are highly welcomed.

Installation

Install dependencies

Run the following command to install all required dependencies:

pip install -r requirements.txt

Note that it is highly recommanded to use an Miniconda environment:

conda create -n mazelab python=3.7

Install mazelab

Run the following commands to install mazelab from source:

git clone https://github.com/zuoxingdong/mazelab.git
cd mazelab
pip install -e .

Installing from source allows to flexibly modify and adapt the code as you pleased, this is very convenient for research purpose which often needs fast prototyping.

Getting started

Detailed tutorials is coming soon. For now, it is recommended to have a look in examples/ or the source code.

Examples

We have provided a Jupyter Notebook for each example to illustrate how to make various of maze environments, and generate animation of the agent's trajectory following the optimal actions solved by our build-in Dijkstra optimal planner.

Simple empty maze

Random shape maze

Random maze

U-maze

Double T-maze

Morris water maze

How to create your own maze/gridworld environment

• Define Generator: You can define your own maze generator, simply generate a two dimensional numpy array consisting of objects labeled by integers.

• Subclass BaseMaze: Define your own maze by creating all Object and their properties

    class Maze(BaseMaze):
        @property
        def size(self):
            return 10, 10

        def make_objects(self):
            free = Object('free', 0, color.free, False, np.stack(np.where(x == 0), axis=1))
            obstacle = Object('obstacle', 1, color.obstacle, True, np.stack(np.where(x == 1), axis=1))
            agent = Object('agent', 2, color.agent, False, [])
            goal = Object('goal', 3, color.goal, False, [])
            return free, obstacle, agent, goal

• Define Motion: Define your own motion or simply use the one we provided: VonNeumannMotion, MooreMotion

• Define gym environment: Subclass the BaseEnv to define the gym environment.

• Register environment: It is recommended to register your own environment to allow easy-to-use gym.make

    gym.envs.register(id=some_name, entry_point=your_file_or_class, max_episode_steps=some_steps)

• Build-in Dijkstra solver: For simple goal-reaching environment, one could use the build-in Dijkstra solver to compute the optimal action sequences given the current agent position and goal position.

    actions = dijkstra_solver(impassable_array, motions, start, goal)

• Record video of executing optimal actions: Wrap the environment with gym's Monitor to make a video animation.
  • Practical tip: One can use imageio to convert mp4 video to GIF. Refer to the examples for more details.

What's new

• 2019-04-05 (v0.2.0)

  • Entirely re-written for much simpler API
  • Subclassing gym.Env to allow all related compatibilities

• 2019-01-01 (v0.1.0)

  • Bugfixes

• 2018-11-18 (v0.0.1)

  • Initial release

Roadmap

• More extensive documentations
• More different kinds of mazes
• More color patterns

Reference

Please use this bibtex if you want to cite this repository in your publications:

@misc{mazelab,
      author = {Zuo, Xingdong},
      title = {mazelab: A customizable framework to create maze and gridworld environments.},
      year = {2018},
      publisher = {GitHub},
      journal = {GitHub repository},
      howpublished = {\url{https://github.com/zuoxingdong/mazelab}},
    }

mazelab's Issues

env.reset() doesn't work

when running

from gym_maze.envs import MazeEnv
from gym_maze.envs.generators import *
maze = RandomBlockMazeGenerator(maze_size=4, obstacle_ratio=0.0)
env = MazeEnv(maze)
env.reset()

I get the following error:

Traceback (most recent call last):
WARN: gym.spaces.Box autodetected dtype as <class 'numpy.float32'>. Please provide explicit dtype.  File "C:/Users/Yitzhak/Projects/gym-maze/example_1.py", line 5, in <module>
    env.reset()
  File "C:\Users\Yitzhak\Anaconda3\lib\site-packages\gym\core.py", line 71, in reset
    raise NotImplementedError
NotImplementedError

Bugs in passable check in _is_valid() method of Maze object

It seems that negative or out of edge position will result index error in _is_valid() method, because of the passable check.
I think it should be changed to
def _is_valid(self, position):
nonnegative = position[0] >= 0 and position[1] >= 0
within_edge = position[0] < self.maze.size[0] and position[1] < self.maze.size[1]
if (nonnegative and within_edge):
passable = not self.maze.to_impassable()[position[0]][position[1]]
else:
passable = False
return nonnegative and within_edge and passable

---

IndexError Traceback (most recent call last)
in
35 action = agent.act(state, epsilon)
36
---> 37 _, reward, done, _ = env.step(action)
38 next_state = env.unwrapped.maze.objects.agent.positions[0]
39

//anaconda3/lib/python3.7/site-packages/gym/wrappers/time_limit.py in step(self, action)
14 def step(self, action):
15 assert self._elapsed_steps is not None, "Cannot call env.step() before calling reset()"
---> 16 observation, reward, done, info = self.env.step(action)
17 self._elapsed_steps += 1
18 if self._elapsed_steps >= self._max_episode_steps:

in step(self, action)
21 current_position = self.maze.objects.agent.positions[0]
22 new_position = [current_position[0] + motion[0], current_position[1] + motion[1]]
---> 23 valid = self._is_valid(new_position)
24 if valid:
25 self.maze.objects.agent.positions = [new_position]

in _is_valid(self, position)
44 nonnegative = position[0] >= 0 and position[1] >= 0
45 within_edge = position[0] < self.maze.size[0] and position[1] < self.maze.size[1]
---> 46 passable = not self.maze.to_impassable()[position[0]][position[1]]
47 return nonnegative and within_edge and passable
48

IndexError: index 6 is out of bounds for axis 0 with size 6

[Question] Multi-Agent Support

Hi,

May I check if it is possible to create an environment supporting multi-agent exploration? Thank you!

TypeError: namedtuple() got an unexpected keyword argument 'defaults'

Following the jupyter notebook, I'm getting the above error when running the line:
from mazelab import BaseMaze

TypeError Traceback (most recent call last)
in
1 import numpy as np
----> 2 from mazelab import BaseMaze
3 from mazelab import Object
4 from mazelab import DeepMindColor as color
5

~/anaconda3/envs/slime/mazelab/mazelab/init.py in
3 from .object import Object
4
----> 5 from .motion import VonNeumannMotion
6 from .motion import MooreMotion
7

~/anaconda3/envs/slime/mazelab/mazelab/motion.py in
4 VonNeumannMotion = namedtuple('VonNeumannMotion',
5 ['north', 'south', 'west', 'east'],
----> 6 defaults=[[-1, 0], [1, 0], [0, -1], [0, 1]])
7
8

TypeError: namedtuple() got an unexpected keyword argument 'defaults'

Track of updates for new release

• Use ImageViewer from lagom to display observation, do not use saving list of arrays, it is too slow.
• Synchronize some tricks from lagom
• Redesign classes

Add Sokoban environment

[Bug] Blurry Rendering of Environment

Hi,

I was running through the examples you have provided and attempted to render the the empty maze environment. However, the resulting image is oddly blurry as attached. Would you know the reason as to why and if there are any fixes?

Thank you!

Refactor Env: use gym.Env

Support metadata: reward_range, spec etc

Could you add License?

Thanks for releasing this project. Could you add License?