aws / sagemaker-rl-container Goto Github PK

Add https://github.com/DLR-RM/stable-baselines3 and https://github.com/DLR-RM/rl-baselines3-zoo
container for sagemaker, so training rl models with stable baselines 3 will be possible.

Is it possible to generate a release on github for this version, as set in the setup.py? Thanks!

I was trying to use
462105765813.dkr.ecr.us-east-1.amazonaws.com/sagemaker-rl-coach-container:coach-1.0.0-tf-gpu-py3
both locally and in Sagemaker Studio, and got the following error:

framework error: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
  File "/usr/lib/python3.6/imp.py", line 243, in load_module
    return load_dynamic(name, filename, file)
  File "/usr/lib/python3.6/imp.py", line 343, in load_dynamic
    return _load(spec)
ImportError: libcuda.so.1: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/sagemaker_containers/_trainer.py", line 73, in train
    framework = importlib.import_module(framework_name)
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.6/dist-packages/sagemaker_tensorflow_container/training.py", line 23, in <module>
    import tensorflow as tf
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/__init__.py", line 24, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 74, in <module>
    raise ImportError(msg)
ImportError: Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
  File "/usr/lib/python3.6/imp.py", line 243, in load_module
    return load_dynamic(name, filename, file)
  File "/usr/lib/python3.6/imp.py", line 343, in load_dynamic
    return _load(spec)
ImportError: libcuda.so.1: cannot open shared object file: No such file or directory


Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/errors

Locally, I was able to work around it by

I was trying to follow the walkthrough bandits_movielens_testbed walkthrough because I want to build and vowpal wabbit adf model and it seems that the sagemaker-rl-vw-container:adf image has been removed and theres no documentation on how to use it.

Hi.

Good day.

Is it possible to not run redis on the GPU? At the moment I am getting the following error using it training deep racer:

subscribe scheduled to be closed ASAP for overcoming of output buffer limits

It seems as though the process wants to allocate gigs of memory into redis but the GPU only has about 7GB, whereas the system has free memory to use.

I'm just not sure how to get it to use the CPU. I tried creating an image myself and making the following change to start.sh:

CUDA_VISIBLE_DEVICES=-1 redis-server --bind 0.0.0.0 &

But when I run the image it doesn't use GPU at all.

Any ideas how to have redis use the systems memory and not the GPU memory? Thanks.

Regards.

Container: sagemaker-rl-ray-container:ray-0.8.2-tf-*-py36

A bug was introduced in this container after an update in the latest version of pyglet. This update breaks the API contract and causes some errors when visualization is enabled (stack trace).

Solution: Downgrade pyglet to version 1.3.2 --> pyglet==1.3.2

Could you change that in the Dockerfile and also update the built images available in SageMaker, please?

Source: tensorflow/agents#163

ray.exceptions.RayTaskError(AttributeError): #33[36mray::RolloutWorker.sample()#33[39m (pid=119, ip=10.2.216.148)
File "python/ray/_raylet.pyx", line 452, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 430, in ray._raylet.execute_task.function_executor
File "/usr/local/lib/python3.6/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 488, in sample
batches = [self.input_reader.next()]
File "/usr/local/lib/python3.6/dist-packages/ray/rllib/evaluation/sampler.py", line 52, in next
batches = [self.get_data()]
File "/usr/local/lib/python3.6/dist-packages/ray/rllib/evaluation/sampler.py", line 95, in get_data
item = next(self.rollout_provider)
File "/usr/local/lib/python3.6/dist-packages/ray/rllib/evaluation/sampler.py", line 301, in _env_runner
base_env.poll()
File "/usr/local/lib/python3.6/dist-packages/ray/rllib/env/base_env.py", line 308, in poll
self.new_obs = self.vector_env.vector_reset()
File "/usr/local/lib/python3.6/dist-packages/ray/rllib/env/vector_env.py", line 96, in vector_reset
return [e.reset() for e in self.envs]
File "/usr/local/lib/python3.6/dist-packages/ray/rllib/env/vector_env.py", line 96, in
return [e.reset() for e in self.envs]
File "/usr/local/lib/python3.6/dist-packages/gym/wrappers/monitor.py", line 39, in reset
self._after_reset(observation)
File "/usr/local/lib/python3.6/dist-packages/gym/wrappers/monitor.py", line 188, in _after_reset
self.reset_video_recorder()
File "/usr/local/lib/python3.6/dist-packages/gym/wrappers/monitor.py", line 209, in reset_video_recorder
self.video_recorder.capture_frame()
File "/usr/local/lib/python3.6/dist-packages/gym/wrappers/monitoring/video_recorder.py", line 101, in capture_frame
frame = self.env.render(mode=render_mode)
File "/usr/local/lib/python3.6/dist-packages/gym/core.py", line 249, in render
return self.env.render(mode, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/gym/envs/classic_control/continuous_mountain_car.py", line 143, in render
return self.viewer.render(return_rgb_array = mode=='rgb_array')
File "/usr/local/lib/python3.6/dist-packages/gym/envs/classic_control/rendering.py", line 105, in render
arr = np.frombuffer(image_data.data, dtype=np.uint8)