Getting error from the example given in the Geting Started page about eureka

[JasonMa2016]

Thank you for your interest in our work! What you are observing is the case when none of the Eureka generations is executable in the first iteration. In our paper, we use 16 samples per iteration and we find that produces at least one executable reward program in all our tasks.

We are happy to accept a PR that implements a feature where the sampling continues until at least one executable reward program is generated.

[jgforbes]

I am getting similar messages from the gtp-3.5-turbo-16k-0613 model.
Code Run 12 execution error!
Code Run 13 cannot parse function signature!
I increased runs to 24 and iterations to 10.
No successful executions of the returned code.

When installing the Eureka code this command gave some errors.
Is rl_games required for the demonstrations?
cd ../rl_games; pip install -e .

It would be nice to reproduce your work.........

[JasonMa2016]

Here is my running the code. Eureka with gpt-4-0314 can generate more than half executable rewards in the first iteration to seed the evolution.

[2023-10-24 00:33:13,735][root][INFO] - Workspace: /home/exx/Projects/eureka-private/eureka/outputs/eureka/2023-10-24_00-33-13
[2023-10-24 00:33:13,735][root][INFO] - Project Root: /home/exx/Projects/eureka-private/eureka
[2023-10-24 00:33:13,735][root][INFO] - Using LLM: gpt-4-0314
[2023-10-24 00:33:13,735][root][INFO] - Task: ShadowHand
[2023-10-24 00:33:13,735][root][INFO] - Task description: to make the shadow hand spin the object to a target orientation
[2023-10-24 00:33:13,758][root][INFO] - Iteration 0: Generating 16 samples with gpt-4-0314
[2023-10-24 00:35:31,723][root][INFO] - Iteration 0: Prompt Tokens: 1735, Completion Tokens: 5685, Total Tokens: 12625
[2023-10-24 00:35:31,724][root][INFO] - Iteration 0: Processing Code Run 0
[2023-10-24 00:35:51,353][root][INFO] - Iteration 0: Code Run 0 execution error!
[2023-10-24 00:35:51,354][root][INFO] - Iteration 0: Processing Code Run 1
[2023-10-24 00:36:01,260][root][INFO] - Iteration 0: Code Run 1 successfully training!
[2023-10-24 00:36:01,260][root][INFO] - Iteration 0: Processing Code Run 2
[2023-10-24 00:36:11,567][root][INFO] - Iteration 0: Code Run 2 execution error!
[2023-10-24 00:36:11,568][root][INFO] - Iteration 0: Processing Code Run 3
[2023-10-24 00:36:21,733][root][INFO] - Iteration 0: Code Run 3 execution error!
[2023-10-24 00:36:21,733][root][INFO] - Iteration 0: Processing Code Run 4
[2023-10-24 00:36:33,710][root][INFO] - Iteration 0: Code Run 4 successfully training!
[2023-10-24 00:36:33,710][root][INFO] - Iteration 0: Processing Code Run 5
[2023-10-24 00:36:45,957][root][INFO] - Iteration 0: Code Run 5 successfully training!
[2023-10-24 00:36:45,957][root][INFO] - Iteration 0: Processing Code Run 6
[2023-10-24 00:36:57,649][root][INFO] - Iteration 0: Code Run 6 execution error!
[2023-10-24 00:36:57,649][root][INFO] - Iteration 0: Processing Code Run 7
[2023-10-24 00:37:10,399][root][INFO] - Iteration 0: Code Run 7 successfully training!
[2023-10-24 00:37:10,399][root][INFO] - Iteration 0: Processing Code Run 8
[2023-10-24 00:37:24,650][root][INFO] - Iteration 0: Code Run 8 successfully training!
[2023-10-24 00:37:24,650][root][INFO] - Iteration 0: Processing Code Run 9
[2023-10-24 00:37:37,578][root][INFO] - Iteration 0: Code Run 9 execution error!
[2023-10-24 00:37:37,578][root][INFO] - Iteration 0: Processing Code Run 10
[2023-10-24 00:37:49,788][root][INFO] - Iteration 0: Code Run 10 execution error!
[2023-10-24 00:37:49,788][root][INFO] - Iteration 0: Processing Code Run 11
[2023-10-24 00:38:03,219][root][INFO] - Iteration 0: Code Run 11 execution error!
[2023-10-24 00:38:03,219][root][INFO] - Iteration 0: Processing Code Run 12
[2023-10-24 00:38:18,007][root][INFO] - Iteration 0: Code Run 12 successfully training!
[2023-10-24 00:38:18,007][root][INFO] - Iteration 0: Processing Code Run 13
[2023-10-24 00:38:34,541][root][INFO] - Iteration 0: Code Run 13 successfully training!
[2023-10-24 00:38:34,541][root][INFO] - Iteration 0: Processing Code Run 14
[2023-10-24 00:38:52,689][root][INFO] - Iteration 0: Code Run 14 successfully training!
[2023-10-24 00:38:52,689][root][INFO] - Iteration 0: Processing Code Run 15
[2023-10-24 00:39:08,733][root][INFO] - Iteration 0: Code Run 15 successfully training!

Now, this is Eureka using gpt-3.5-turbo-16k-0613:

[2023-10-24 00:40:14,778][root][INFO] - Workspace: /home/exx/Projects/eureka-private/eureka/outputs/eureka/2023-10-24_00-40-14
[2023-10-24 00:40:14,778][root][INFO] - Project Root: /home/exx/Projects/eureka-private/eureka
[2023-10-24 00:40:14,778][root][INFO] - Using LLM: gpt-3.5-turbo-16k-0613
[2023-10-24 00:40:14,778][root][INFO] - Task: ShadowHand
[2023-10-24 00:40:14,778][root][INFO] - Task description: to make the shadow hand spin the object to a target orientation
[2023-10-24 00:40:14,800][root][INFO] - Iteration 0: Generating 16 samples with gpt-3.5-turbo-16k-0613
[2023-10-24 00:40:50,280][root][INFO] - Iteration 0: Prompt Tokens: 1735, Completion Tokens: 4594, Total Tokens: 6329
[2023-10-24 00:40:50,280][root][INFO] - Iteration 0: Processing Code Run 0
[2023-10-24 00:40:52,922][root][INFO] - Iteration 0: Code Run 0 execution error!
[2023-10-24 00:40:52,922][root][INFO] - Iteration 0: Processing Code Run 1
[2023-10-24 00:41:03,572][root][INFO] - Iteration 0: Code Run 1 successfully training!
[2023-10-24 00:41:03,572][root][INFO] - Iteration 0: Processing Code Run 2
[2023-10-24 00:41:15,421][root][INFO] - Iteration 0: Code Run 2 successfully training!
[2023-10-24 00:41:15,421][root][INFO] - Iteration 0: Processing Code Run 3
[2023-10-24 00:41:15,421][root][INFO] - Iteration 0: Code Run 3 cannot parse function signature!
[2023-10-24 00:41:15,421][root][INFO] - Iteration 0: Processing Code Run 4
[2023-10-24 00:41:26,643][root][INFO] - Iteration 0: Code Run 4 execution error!
[2023-10-24 00:41:26,643][root][INFO] - Iteration 0: Processing Code Run 5
[2023-10-24 00:41:37,264][root][INFO] - Iteration 0: Code Run 5 execution error!
[2023-10-24 00:41:37,264][root][INFO] - Iteration 0: Processing Code Run 6
[2023-10-24 00:41:50,919][root][INFO] - Iteration 0: Code Run 6 successfully training!
[2023-10-24 00:41:50,919][root][INFO] - Iteration 0: Processing Code Run 7
[2023-10-24 00:42:01,638][root][INFO] - Iteration 0: Code Run 7 execution error!
[2023-10-24 00:42:01,638][root][INFO] - Iteration 0: Processing Code Run 8
[2023-10-24 00:42:17,147][root][INFO] - Iteration 0: Code Run 8 successfully training!
[2023-10-24 00:42:17,147][root][INFO] - Iteration 0: Processing Code Run 9
[2023-10-24 00:42:29,767][root][INFO] - Iteration 0: Code Run 9 successfully training!
[2023-10-24 00:42:29,767][root][INFO] - Iteration 0: Processing Code Run 10
[2023-10-24 00:42:32,455][root][INFO] - Iteration 0: Code Run 10 execution error!
[2023-10-24 00:42:32,455][root][INFO] - Iteration 0: Processing Code Run 11
[2023-10-24 00:42:46,817][root][INFO] - Iteration 0: Code Run 11 successfully training!
[2023-10-24 00:42:46,817][root][INFO] - Iteration 0: Processing Code Run 12
[2023-10-24 00:43:04,635][root][INFO] - Iteration 0: Code Run 12 successfully training!
[2023-10-24 00:43:04,635][root][INFO] - Iteration 0: Processing Code Run 13
[2023-10-24 00:43:21,069][root][INFO] - Iteration 0: Code Run 13 successfully training!
[2023-10-24 00:43:21,069][root][INFO] - Iteration 0: Processing Code Run 14
[2023-10-24 00:43:41,452][root][INFO] - Iteration 0: Code Run 14 successfully training!
[2023-10-24 00:43:41,453][root][INFO] - Iteration 0: Processing Code Run 15
[2023-10-24 00:43:51,951][root][INFO] - Iteration 0: Code Run 15 execution error!

Given that API calls are stochastic, it is possible that the entire iteration does not have a single executable reward. In this case, I suggest just re-running the code again. A more long term solution would be to keep sampling until an executable reward is returned; we welcome a PR that implements this feature!

[jgforbes]

OK, I am missing the rl_games.common module.

[jgforbes]

This error when running: pip install -e .
in rl_gamesRequirement already satisfied: distlib<1,>=0.3.7 in /home/gymuser/.local/lib/python3.8/site-packages (from virtualenv->ray<2.0.0,>=1.11.0->rl_games==1.6.1) (0.3.7)
Installing collected packages: rl-games
Running setup.py develop for rl-games
ERROR: Command errored out with exit status 1:
command: /opt/conda/bin/python3.8 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/gymuser/Eureka/rl_games/setup.py'"'"'; file='"'"'/home/gymuser/Eureka/rl_games/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps --user --prefix=
cwd: /home/gymuser/Eureka/rl_games/
Complete output (3 lines):
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'setuptools'
----------------------------------------
ERROR: Command errored out with exit status 1: /opt/conda/bin/python3.8 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/gymuser/Eureka/rl_games/setup.py'"'"'; file='"'"'/home/gymuser/Eureka/rl_games/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps --user --prefix= Check the logs for full command output.

[jgforbes]

need setuptools in pyproject.toml
https://github.com/Denys88/rl_games/pull/190/files#diff-50c86b7ed8ac2cf95bd48334961bf0530cdc77b5a56f852c5c61b89d735fd711