simondlevy / neat-gym Goto Github PK

View Code? Open in Web Editor NEW

51.0 51.0 16.0 776 KB

Neuro-evolution for OpenAI Gym environments

Python 98.41% Makefile 1.59%

neat-gym's People

Contributors

Stargazers

Watchers

Forkers

coletta1 mehrdadzakershahrak arielbwong gittestusername sikora507 miouzikal mihirk284 jiaoalvin and3rsonls soniccan yamatakeru kaiserho standardgalactic kwon-jh iq-scm alyonav

neat-gym's Issues

Add --res option for food-gathering demo script

Move --seed, --ngen to config file

Make sure checkpoint is working from config file

Use different extension for saved Hyper nets vs ordinary NEAT

Replace step_novelty with info dictionary

Normalize time axis for copter-plot.py

Fix column error in neat-plot.py (treats time as mean fitness)

Pickle environment name, not environment object

Don't require .cfg file for testing

neat-evolve should save average, best fitnesses (and novelties) for plotting

Allow environment to specify substrate for HyperNEAT

Move .parse() out of Novelty class

Derivce input/output architecture from environment, so doesn't have to go in .cfg

Add --save option to neat-test.py

Fix reporting for novelty search

--hyper option should use .py file to generate hidden-unit layout

Merge .subs, .names files into .cfg/.cppn

Issue with discrete action spaces

Hi again Simon!

I am trying to use neat-gym with some of the Gym toy text examples that are based on discrete action spaces, but I'm having some issues. From my limited understanding of neat-gym it inherits the observation and action space from the registered Gym environment, but I am getting the following errors whenever I attempt to use neat-gym with any of the discrete valued Gym environments:

$pablo python3 neat-evolve.py --eshyper config/frozenlake

 ****** Running generation 0 ****** 

/usr/lib/python3/dist-packages/requests/__init__.py:89: RequestsDependencyWarning: urllib3 (1.26.6) or chardet (4.0.0) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/pablo/Documents/development/frozenlake/neat-gym/neat-evolve.py", line 438, in eval_genome
    return config.eval_net_mean(net, genome)
  File "/home/pablo/Documents/development/frozenlake/neat-gym/neat-evolve.py", line 182, in eval_net_mean
    else self.eval_net_mean_reward(net, genome))
  File "/home/pablo/Documents/development/frozenlake/neat-gym/neat-evolve.py", line 191, in eval_net_mean_reward
    reward, steps = eval_net(net,
  File "/home/pablo/Documents/development/frozenlake/neat-gym/neat_gym/__init__.py", line 123, in eval_net
    action = net.activate(state)
  File "/home/pablo/.local/lib/python3.8/site-packages/neat/nn/recurrent.py", line 27, in activate
    if len(self.input_nodes) != len(inputs):
TypeError: object of type 'int' has no len()
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/pablo/Documents/development/frozenlake/neat-gym/neat-evolve.py", line 766, in <module>
    main()
  File "/home/pablo/Documents/development/frozenlake/neat-gym/neat-evolve.py", line 759, in main
    winner = pop.run(pe.evaluate, config.ngen, args.maxtime)
  File "/home/pablo/Documents/development/frozenlake/neat-gym/neat-evolve.py", line 471, in run
    fitness_function(list(self.population.items()), self.config)
  File "/home/pablo/.local/lib/python3.8/site-packages/neat/parallel.py", line 30, in evaluate
    genome.fitness = job.get(timeout=self.timeout)
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
TypeError: object of type 'int' has no len()

$pablo python3 neat-evolve.py config/frozenlake

Traceback (most recent call last):
  File "/home/pablo/Documents/development/frozenlake/neat-gym/neat-evolve.py", line 766, in <module>
    main()
  File "/home/pablo/Documents/development/frozenlake/neat-gym/neat-evolve.py", line 736, in main
    config = _GymNeatConfig(args)
  File "/home/pablo/Documents/development/frozenlake/neat-gym/neat-evolve.py", line 109, in __init__
    num_inputs = env.observation_space.shape[0]
IndexError: tuple index out of range

I've tried a couple of the other envs that utilize spaces.Discrete, all with the same type of error. Any ideas?

Make environments for HyperNEAT tasks in Stanley et al. 2009

Eliminate command-line options (just use config file / commenting-out)

Add --time option for neat-evolve.py

evolve --hyper NHID or NHID1,NHID2,...

Return to single neat-evolve script with --hyper, --eshyper

group = parser.add_mutually_exclusive_group()
group.add_argument('--arg1', action='store_true')
group.add_argument('--arg2', action='store_false')

xor-evolve --novelty k threshold limit

neat-plot should show stdev as dashed lines

Support reps in novelty search

Use OpenCV to animate NEAT test

Compare NEAT XOR to PUREPLES

--cfgdir => --config [default config/<env_name>.cfg]

Save visuals of best at each generation

Saves speciation diagram

evolve.py visual uses nodenames for Hyper but not plain NEAT

Change -hyper.cfg file and draw_net to reflect CPPN 5->1 restriction

Fix bug in test.py with hyper-derived nets

Add --time option for neat-plot.py

Generate substrate procedurally

Replace saveArchive() with pickling

Factor .cfg files

Cannot import Augmented Genome

I installed neat-gym acccording to the specifications in the ReadMe.

In neat-gym/neat_gym/novelty/tests/, the xor_evolve.py throws an error because AugmentedGenome and the NeatConfig modules cannot be imported.

Question about dynamic action space and ideal architecture

Hi Simon, thanks for releasing this project!

I am interested in implementing NEAT for function approximation, by sampling data points from two "competing" IoT sensors and then using a simple scoring mechanism to determine which sensor outputs the most correct data points. Problem is, the sensors can output two or three values depending on when they are sampled. Dataset looks like this:

[40, 9] [8, 4]
[42, 1, 3] [9, 22]
[8, 5, 7] [2, 9, 80]
[30, 20] [5, 90]
[1, 20, 90] [90, 7]
[70, 5, 90] [90, 9, 20]
[5, 8, 4] [1, 20, 5]

So should I be zero padding everything so that each sample is exactly three elements, or is it possible to use neat-gym in conjunction with a dynamic Gym observation space?

Would ES-HyperNEAT be the best option for this? Any pointers on implementing neat-gym with strictly integer-valued data would be greatly appreciated!