B.A. in Linguistics
B.S. in Computer Science and Engineeringmiroblog / deep_rl_trader Goto Github PK
View Code? Open in Web Editor NEWTrading Environment(OpenAI Gym) + DDQN (Keras-RL)
Trading Environment(OpenAI Gym) + DDQN (Keras-RL)
B.A. in Linguistics
B.S. in Computer Science and EngineeringHi, thank you for this implementation of reinforcement learning.
I created a google colab file. I succeded to run most of the code but I got an error at the very last part.
This is the error im getting :
`Training for 5500 steps ...
start episode ... XBTUSD_5m_70000_train.csv at 0
ValueError Traceback (most recent call last)
in ()
1 while True:
2 # train
----> 3 dqn.fit(env, nb_steps=5500, nb_max_episode_steps=10000, visualize=False, verbose=2)
4 try:
5 # validate
/usr/local/lib/python3.6/dist-packages/rl/core.py in fit(self, env, nb_steps, action_repetition, callbacks, verbose, visualize, nb_max_start_steps, start_step_policy, log_interval, nb_max_episode_steps)
180 observation, r, done, info = self.processor.process_step(observation, r, done, info)
181 for key, value in info.items():
--> 182 if not np.isreal(value):
183 continue
184 if key not in accumulated_info:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()`
How can I fix this?
I shared the colab file so it can mabe help other people to.
https://colab.research.google.com/drive/1DyURfsL9091Hx8IsEKwPGFxVl_aUEspp
Thank you for your help,
greg
I noticed in the /data folder, the training data in /train includes all data for validation data in /test. There's no validation split in the model so I assume validation datapoints also have a chance to be trained by the model.
Doesn't that lead to overfit and exaggerated model performance?
Could not find a version that satisfies the requirement anaconda-client==1.6.0
Hi there, nice work.
However I think there is a look-ahead bias.
Every timestep, you get state and this state includes the current closeprice.
Then with step method you calculate profit as:
self.exit_price = self.closingPrice
self.reward += ((self.entry_price - self.exit_price)/self.exit_price + 1)*(1-self.fee)**2 - 1 # calculate reward
In this case you are using the same information that you already used to predict the next action.
What do you think about it?
Making all the changes to the PIP code to run, this is not the first time this error has occurred:
Traceback (most recent call last):
File "y:/python_udemy/deep_rl_trader-master/deep_rl_trader/ddqn_rl_trader.py", line 80, in
main()
File "y:/python_udemy/deep_rl_trader-master/deep_rl_trader/ddqn_rl_trader.py", line 59, in main
processor=NormalizerProcessor())
File "C:\Users\danilo.martins\Anaconda3\lib\site-packages\rl\agents\dqn.py", line 111, in init
raise ValueError('Model output "{}" has invalid shape. DQN expects a model that has one dimension for each action, in this case {}.'.format(model.output, self.nb_actions))
ValueError: Model output "Tensor("dense_2/BiasAdd:0", shape=(?, 3), dtype=float32)" has invalid shape. DQN expects a model that has one dimension for each action, in this case 3.
What am I doing wrong, you know?
That's what I just tried to give the RUN to see how it would work here.
The project implements train and test function. If it is used in actual trading envirionment, how to use this model to predict action when every K bar is close? The DQNAgent has not predict function, use model.predict() or some function else?
Using TensorFlow backend.
2020-04-26 16:18:38.050381: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
2020-04-26 16:18:38.050737: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
ImportError: numpy.core.multiarray failed to import
ImportError: numpy.core._multiarray_umath failed to import
ImportError: numpy.core.umath failed to import
2020-04-26 16:18:38.671400: F tensorflow/python/lib/core/bfloat16.cc:675] Check failed: PyBfloat16_Type.tp_base != nullptr
1.Is this because I dont have a GPU?
2.My IDE is Pycharm, should use Anaconda to match the environment? (Cause the pycharm keeps telling me some requirements couldnt be installed, including "Anaconda client ==1.6.0 , bitarray == 0.8.1 ......... "
Thank you so much for this great project.
When i try to run ddqn_rl_trader.py on windows (my computer has no GPU, so i use LSTM instead of CuDNNLSTM), i get the following errors:
2019-01-17 17:06:16.101245: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary
start episode ... XBTUSD_5m_70000_train.csv at 0
Traceback (most recent call last):
File "ddqn_rl_trader.py", line 81, in
main()
File "ddqn_rl_trader.py", line 65, in main
dqn.fit(env, nb_steps=5500, nb_max_episode_steps=10000, visualize=False, verbose=0)
File "C:\Python36\lib\site-packages\rl\core.py", line 182, in fit
if not np.isreal(value):
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
May i ask what change i can make to this problem?
Thanks a lot
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.