Coder Social home page Coder Social logo

actor-critic-continuous-keras's People

Contributors

hyeokreal avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar

Forkers

wwx13 zeroweight matteobrentegani

actor-critic-continuous-keras's Issues

a2c_continuous.py total reward stays < 0

Hello,

I am not sure where I may be going wrong. I just copy pasted the a2c_continuous.py file and even after 3000 episodes the 10 episode average reward has converged from -133 to -2 or something. It doesnt even cross 0 , can you please let me know how did you manage to converge this to +100 in the same number of episodes ?

When I run it, after some time it keeps bouncing between -10 and -2.

I also tried the a3c_continuous.py and same happens there as well.

Thanks

tensorflow.python.framework.errors_impl.NotFoundError

@Hyeokreal which Tensorflow and Keras version do yo use?

I tried running a3c_continuous.py, but I get these errors:

2018-02-16 11:55:39.292348: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2 AVX
[2018-02-16 11:55:39,904] Making new env: Pendulum-v0
[2018-02-16 11:55:39,911] Making new env: Pendulum-v0
[2018-02-16 11:55:39,923] Making new env: Pendulum-v0
[2018-02-16 11:55:39,963] Making new env: Pendulum-v0
[2018-02-16 11:55:39,979] Making new env: Pendulum-v0
[2018-02-16 11:55:39,996] Making new env: Pendulum-v0
[2018-02-16 11:55:40,024] Making new env: Pendulum-v0
[2018-02-16 11:55:40,088] Making new env: Pendulum-v0
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _do_call
    return fn(*args)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1329, in _run_fn
    status, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_1/IsVariableInitialized_2/IsVariableInitialized_46:0: not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/anaconda/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "a3c_continuous.py", line 180, in run
    action = self.get_action(state)
  File "a3c_continuous.py", line 244, in get_action
    mu, sigma_sq = self.actor.predict(np.reshape(state, [1, self.state_size]))
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1842, in predict
    verbose=verbose, steps=steps)
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1337, in _predict_loop
    batch_outs = f(ins_batch)
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2473, in __call__
    session = get_session()
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 189, in get_session
    [tf.is_variable_initialized(v) for v in candidate_vars])
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
    feed_dict_tensor, options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1344, in _do_run
    options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1363, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_1/IsVariableInitialized_2/IsVariableInitialized_46:0: not found

Exception in thread Thread-7:
Traceback (most recent call last):
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _do_call
    return fn(*args)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1329, in _run_fn
    status, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_8/IsVariableInitialized_3/IsVariableInitialized_2/IsVariableInitialized_43:0: not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/anaconda/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "a3c_continuous.py", line 180, in run
    action = self.get_action(state)
  File "a3c_continuous.py", line 244, in get_action
    mu, sigma_sq = self.actor.predict(np.reshape(state, [1, self.state_size]))
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1842, in predict
    verbose=verbose, steps=steps)
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1337, in _predict_loop
    batch_outs = f(ins_batch)
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2473, in __call__
    session = get_session()
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 189, in get_session
    [tf.is_variable_initialized(v) for v in candidate_vars])
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
    feed_dict_tensor, options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1344, in _do_run
    options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1363, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_8/IsVariableInitialized_3/IsVariableInitialized_2/IsVariableInitialized_43:0: not found

Exception in thread Thread-5:
Traceback (most recent call last):
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _do_call
    return fn(*args)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1329, in _run_fn
    status, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_8/IsVariableInitialized_6/IsVariableInitialized_51:0: not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/anaconda/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "a3c_continuous.py", line 180, in run
    action = self.get_action(state)
  File "a3c_continuous.py", line 244, in get_action
    mu, sigma_sq = self.actor.predict(np.reshape(state, [1, self.state_size]))
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1842, in predict
    verbose=verbose, steps=steps)
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1337, in _predict_loop
    batch_outs = f(ins_batch)
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2473, in __call__
    session = get_session()
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 189, in get_session
    [tf.is_variable_initialized(v) for v in candidate_vars])
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
    feed_dict_tensor, options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1344, in _do_run
    options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1363, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_8/IsVariableInitialized_6/IsVariableInitialized_51:0: not found

Exception in thread Thread-4:
Traceback (most recent call last):
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _do_call
    return fn(*args)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1329, in _run_fn
    status, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_36/IsVariableInitialized_11/IsVariableInitialized_6:0: not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/anaconda/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "a3c_continuous.py", line 180, in run
    action = self.get_action(state)
  File "a3c_continuous.py", line 244, in get_action
    mu, sigma_sq = self.actor.predict(np.reshape(state, [1, self.state_size]))
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1842, in predict
    verbose=verbose, steps=steps)
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1337, in _predict_loop
    batch_outs = f(ins_batch)
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2473, in __call__
    session = get_session()
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 189, in get_session
    [tf.is_variable_initialized(v) for v in candidate_vars])
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
    feed_dict_tensor, options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1344, in _do_run
    options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1363, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_36/IsVariableInitialized_11/IsVariableInitialized_6:0: not found

episode:  1 / score :  [-168.84377024] / step :  200
episode:  2 / score :  [-161.87116017] / step :  200
episode:  3 / score :  [-159.24898023] / step :  200
episode:  4 / score :  [-187.5850562] / step :  200
episode:  5 / score :  [-167.4228075] / step :  200
episode:  6 / score :  [-185.07506142] / step :  200
episode:  7 / score :  [-169.44206692] / step :  200
episode:  8 / score :  [-183.16870669] / step :  200
episode:  9 / score :  [-137.96890288] / step :  200
episode:  10 / score :  [-171.25066072] / step :  200
episode:  11 / score :  [-151.99627662] / step :  200
episode:  12 / score :  [-121.34443564] / step :  200
^CTraceback (most recent call last):
  File "a3c_continuous.py", line 267, in <module>
    global_agent.train()
  File "a3c_continuous.py", line 137, in train
    time.sleep(20)
KeyboardInterrupt
episode:  13 / score :  [-177.44202396] / step :  200
episode:  14 / score :  [-177.48592821] / step :  200
episode:  15 / score :  [-167.54128117] / step :  200
episode:  16 / score :  [-187.03103497] / step :  200
episode:  17 / score :  [-175.32132064] / step :  200
episode:  18 / score :  [-124.67301435] / step :  200
episode:  19 / score :  [-177.01564837] / step :  200
episode:  20 / score :  [-142.00295605] / step :  200
^CException ignored in: <module 'threading' from '/anaconda/lib/python3.6/threading.py'>
Traceback (most recent call last):
  File "/anaconda/lib/python3.6/threading.py", line 1294, in _shutdown
    t.join()
  File "/anaconda/lib/python3.6/threading.py", line 1056, in join
    self._wait_for_tstate_lock()
  File "/anaconda/lib/python3.6/threading.py", line 1072, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt
^C

This is my environment:

$ pip show keras
Name: Keras
Version: 2.1.4
Summary: Deep Learning for humans
Home-page: https://github.com/keras-team/keras
Author: Francois Chollet
Author-email: [email protected]
License: MIT
Location: /anaconda/lib/python3.6/site-packages
Requires: scipy, pyyaml, six, numpy

$ pip show tensorflow
Name: tensorflow
Version: 1.5.0
Summary: TensorFlow helps the tensors flow
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: [email protected]
License: Apache 2.0
Location: /anaconda/lib/python3.6/site-packages
Requires: tensorflow-tensorboard, protobuf, numpy, wheel, six, absl-py

How to make a3c_continuous works other environments?

Hi,
I use a3c_continuous for another env in which inputs are an image of env and robot's sensors datas.
For me the main problem is in line 239 where feeding datas in optimizors dont work and I get this error : 'list' object has not attribute 'dtype'.
By searching I found out that the problem is in how I feed the datas in model, but I really cant fix it.
I want ask you, is there any extension of your code for multi input and multi output environment?

Best Regards.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.