Comments (6)
Thank you for your interest in our work! What you are observing is the case when none of the Eureka generations is executable in the first iteration. In our paper, we use 16 samples per iteration and we find that produces at least one executable reward program in all our tasks.
We are happy to accept a PR that implements a feature where the sampling continues until at least one executable reward program is generated.
from eureka.
I am getting similar messages from the gtp-3.5-turbo-16k-0613 model.
Code Run 12 execution error!
Code Run 13 cannot parse function signature!
I increased runs to 24 and iterations to 10.
No successful executions of the returned code.
When installing the Eureka code this command gave some errors.
Is rl_games required for the demonstrations?
cd ../rl_games; pip install -e .
It would be nice to reproduce your work.........
from eureka.
Here is my running the code. Eureka with gpt-4-0314
can generate more than half executable rewards in the first iteration to seed the evolution.
[2023-10-24 00:33:13,735][root][INFO] - Workspace: /home/exx/Projects/eureka-private/eureka/outputs/eureka/2023-10-24_00-33-13
[2023-10-24 00:33:13,735][root][INFO] - Project Root: /home/exx/Projects/eureka-private/eureka
[2023-10-24 00:33:13,735][root][INFO] - Using LLM: gpt-4-0314
[2023-10-24 00:33:13,735][root][INFO] - Task: ShadowHand
[2023-10-24 00:33:13,735][root][INFO] - Task description: to make the shadow hand spin the object to a target orientation
[2023-10-24 00:33:13,758][root][INFO] - Iteration 0: Generating 16 samples with gpt-4-0314
[2023-10-24 00:35:31,723][root][INFO] - Iteration 0: Prompt Tokens: 1735, Completion Tokens: 5685, Total Tokens: 12625
[2023-10-24 00:35:31,724][root][INFO] - Iteration 0: Processing Code Run 0
[2023-10-24 00:35:51,353][root][INFO] - Iteration 0: Code Run 0 execution error!
[2023-10-24 00:35:51,354][root][INFO] - Iteration 0: Processing Code Run 1
[2023-10-24 00:36:01,260][root][INFO] - Iteration 0: Code Run 1 successfully training!
[2023-10-24 00:36:01,260][root][INFO] - Iteration 0: Processing Code Run 2
[2023-10-24 00:36:11,567][root][INFO] - Iteration 0: Code Run 2 execution error!
[2023-10-24 00:36:11,568][root][INFO] - Iteration 0: Processing Code Run 3
[2023-10-24 00:36:21,733][root][INFO] - Iteration 0: Code Run 3 execution error!
[2023-10-24 00:36:21,733][root][INFO] - Iteration 0: Processing Code Run 4
[2023-10-24 00:36:33,710][root][INFO] - Iteration 0: Code Run 4 successfully training!
[2023-10-24 00:36:33,710][root][INFO] - Iteration 0: Processing Code Run 5
[2023-10-24 00:36:45,957][root][INFO] - Iteration 0: Code Run 5 successfully training!
[2023-10-24 00:36:45,957][root][INFO] - Iteration 0: Processing Code Run 6
[2023-10-24 00:36:57,649][root][INFO] - Iteration 0: Code Run 6 execution error!
[2023-10-24 00:36:57,649][root][INFO] - Iteration 0: Processing Code Run 7
[2023-10-24 00:37:10,399][root][INFO] - Iteration 0: Code Run 7 successfully training!
[2023-10-24 00:37:10,399][root][INFO] - Iteration 0: Processing Code Run 8
[2023-10-24 00:37:24,650][root][INFO] - Iteration 0: Code Run 8 successfully training!
[2023-10-24 00:37:24,650][root][INFO] - Iteration 0: Processing Code Run 9
[2023-10-24 00:37:37,578][root][INFO] - Iteration 0: Code Run 9 execution error!
[2023-10-24 00:37:37,578][root][INFO] - Iteration 0: Processing Code Run 10
[2023-10-24 00:37:49,788][root][INFO] - Iteration 0: Code Run 10 execution error!
[2023-10-24 00:37:49,788][root][INFO] - Iteration 0: Processing Code Run 11
[2023-10-24 00:38:03,219][root][INFO] - Iteration 0: Code Run 11 execution error!
[2023-10-24 00:38:03,219][root][INFO] - Iteration 0: Processing Code Run 12
[2023-10-24 00:38:18,007][root][INFO] - Iteration 0: Code Run 12 successfully training!
[2023-10-24 00:38:18,007][root][INFO] - Iteration 0: Processing Code Run 13
[2023-10-24 00:38:34,541][root][INFO] - Iteration 0: Code Run 13 successfully training!
[2023-10-24 00:38:34,541][root][INFO] - Iteration 0: Processing Code Run 14
[2023-10-24 00:38:52,689][root][INFO] - Iteration 0: Code Run 14 successfully training!
[2023-10-24 00:38:52,689][root][INFO] - Iteration 0: Processing Code Run 15
[2023-10-24 00:39:08,733][root][INFO] - Iteration 0: Code Run 15 successfully training!
Now, this is Eureka using gpt-3.5-turbo-16k-0613
:
[2023-10-24 00:40:14,778][root][INFO] - Workspace: /home/exx/Projects/eureka-private/eureka/outputs/eureka/2023-10-24_00-40-14
[2023-10-24 00:40:14,778][root][INFO] - Project Root: /home/exx/Projects/eureka-private/eureka
[2023-10-24 00:40:14,778][root][INFO] - Using LLM: gpt-3.5-turbo-16k-0613
[2023-10-24 00:40:14,778][root][INFO] - Task: ShadowHand
[2023-10-24 00:40:14,778][root][INFO] - Task description: to make the shadow hand spin the object to a target orientation
[2023-10-24 00:40:14,800][root][INFO] - Iteration 0: Generating 16 samples with gpt-3.5-turbo-16k-0613
[2023-10-24 00:40:50,280][root][INFO] - Iteration 0: Prompt Tokens: 1735, Completion Tokens: 4594, Total Tokens: 6329
[2023-10-24 00:40:50,280][root][INFO] - Iteration 0: Processing Code Run 0
[2023-10-24 00:40:52,922][root][INFO] - Iteration 0: Code Run 0 execution error!
[2023-10-24 00:40:52,922][root][INFO] - Iteration 0: Processing Code Run 1
[2023-10-24 00:41:03,572][root][INFO] - Iteration 0: Code Run 1 successfully training!
[2023-10-24 00:41:03,572][root][INFO] - Iteration 0: Processing Code Run 2
[2023-10-24 00:41:15,421][root][INFO] - Iteration 0: Code Run 2 successfully training!
[2023-10-24 00:41:15,421][root][INFO] - Iteration 0: Processing Code Run 3
[2023-10-24 00:41:15,421][root][INFO] - Iteration 0: Code Run 3 cannot parse function signature!
[2023-10-24 00:41:15,421][root][INFO] - Iteration 0: Processing Code Run 4
[2023-10-24 00:41:26,643][root][INFO] - Iteration 0: Code Run 4 execution error!
[2023-10-24 00:41:26,643][root][INFO] - Iteration 0: Processing Code Run 5
[2023-10-24 00:41:37,264][root][INFO] - Iteration 0: Code Run 5 execution error!
[2023-10-24 00:41:37,264][root][INFO] - Iteration 0: Processing Code Run 6
[2023-10-24 00:41:50,919][root][INFO] - Iteration 0: Code Run 6 successfully training!
[2023-10-24 00:41:50,919][root][INFO] - Iteration 0: Processing Code Run 7
[2023-10-24 00:42:01,638][root][INFO] - Iteration 0: Code Run 7 execution error!
[2023-10-24 00:42:01,638][root][INFO] - Iteration 0: Processing Code Run 8
[2023-10-24 00:42:17,147][root][INFO] - Iteration 0: Code Run 8 successfully training!
[2023-10-24 00:42:17,147][root][INFO] - Iteration 0: Processing Code Run 9
[2023-10-24 00:42:29,767][root][INFO] - Iteration 0: Code Run 9 successfully training!
[2023-10-24 00:42:29,767][root][INFO] - Iteration 0: Processing Code Run 10
[2023-10-24 00:42:32,455][root][INFO] - Iteration 0: Code Run 10 execution error!
[2023-10-24 00:42:32,455][root][INFO] - Iteration 0: Processing Code Run 11
[2023-10-24 00:42:46,817][root][INFO] - Iteration 0: Code Run 11 successfully training!
[2023-10-24 00:42:46,817][root][INFO] - Iteration 0: Processing Code Run 12
[2023-10-24 00:43:04,635][root][INFO] - Iteration 0: Code Run 12 successfully training!
[2023-10-24 00:43:04,635][root][INFO] - Iteration 0: Processing Code Run 13
[2023-10-24 00:43:21,069][root][INFO] - Iteration 0: Code Run 13 successfully training!
[2023-10-24 00:43:21,069][root][INFO] - Iteration 0: Processing Code Run 14
[2023-10-24 00:43:41,452][root][INFO] - Iteration 0: Code Run 14 successfully training!
[2023-10-24 00:43:41,453][root][INFO] - Iteration 0: Processing Code Run 15
[2023-10-24 00:43:51,951][root][INFO] - Iteration 0: Code Run 15 execution error!
Given that API calls are stochastic, it is possible that the entire iteration does not have a single executable reward. In this case, I suggest just re-running the code again. A more long term solution would be to keep sampling until an executable reward is returned; we welcome a PR that implements this feature!
from eureka.
OK, I am missing the rl_games.common module.
from eureka.
This error when running: pip install -e .
in rl_gamesRequirement already satisfied: distlib<1,>=0.3.7 in /home/gymuser/.local/lib/python3.8/site-packages (from virtualenv->ray<2.0.0,>=1.11.0->rl_games==1.6.1) (0.3.7)
Installing collected packages: rl-games
Running setup.py develop for rl-games
ERROR: Command errored out with exit status 1:
command: /opt/conda/bin/python3.8 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/gymuser/Eureka/rl_games/setup.py'"'"'; file='"'"'/home/gymuser/Eureka/rl_games/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps --user --prefix=
cwd: /home/gymuser/Eureka/rl_games/
Complete output (3 lines):
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'setuptools'
----------------------------------------
ERROR: Command errored out with exit status 1: /opt/conda/bin/python3.8 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/gymuser/Eureka/rl_games/setup.py'"'"'; file='"'"'/home/gymuser/Eureka/rl_games/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps --user --prefix= Check the logs for full command output.
from eureka.
need setuptools in pyproject.toml
https://github.com/Denys88/rl_games/pull/190/files#diff-50c86b7ed8ac2cf95bd48334961bf0530cdc77b5a56f852c5c61b89d735fd711
from eureka.
Related Issues (20)
- subprocess.POPEN waits forever
- Question about "Reward Generation Problem". Having env ground truth for RL seems weird. HOT 2
- An environmental issue HOT 3
- Can this play pokemon on gameboy?
- Error with Eureka Pen Spinning Demo HOT 4
- find the Human Normalised Score metric in the log HOT 2
- Unfair comparison! Human designed reward in the source code
- visual perception (from the robot camera)
- Error with example code
- HumanoidGPT environment error HOT 1
- No module named fbx HOT 3
- IndexError: list index out of range HOT 2
- Request for Information on Prompts Used in Curriculum Learning for Pen Spinning Task
- - HOT 3
- thoughts and follow-up questions after paper read
- Error with building wheels for numpy ? HOT 1
- How to train on multiple GPUs HOT 4
- The entire procedure of the experiment HOT 1
- questions regarding use of Eureka for a real Robot HOT 1
- No well-defined code fencing in README.md
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from eureka.