miroblog / tf_deep_rl_trader

Trading Environment(OpenAI Gym) + PPO(TensorForce)

Skills

         
   

Education📚

- Seoul National Univ

B.A. in Linguistics

B.S. in Computer Science and Engineering

Career💼

RippleAI

Research Engineer | 2019.3 ~ 2021.11

- RippleHouse(A.I. Powered Live Chat Simulator)

- Clipper(Video Analysis Software)

- AfreecaTV Esports Analyzer(ESports Analysis Software)

- Video Chatter (Multi-Modal Chatbot)

Awards 🏆

- VQA for the visually impaired @ MSIT & NIPA 2021 Online A.I. Challenge, 2nd place 🥈

- Sarcasm Detection Challenge @ ACL 2020, Figurative Language Processing, 1st place 🥇

- Multi Label Action Detection Challenge @ ICCV 2019, Multimodal Video Analysis, 4th place

tf_deep_rl_trader's Issues

not able to test the whole thing

Traceback (most recent call last):
File "ppo_trader.py", line 134, in
main()
File "ppo_trader.py", line 131, in main
test_runner.run(num_episodes=1, deterministic=True, testing=True, episode_finished=print_simple_log)
File "/home/kedarnath/anaconda3/envs/DeepRLTrader/lib/python3.6/site-packages/tensorforce/execution/runner.py", line 104, in run
state, terminal, step_reward = self.environment.execute(action=action)
File "/home/kedarnath/Desktop/tf_deep_rl_trader-master/tf_deep_rl_trader-master/env/gymWrapper.py", line 69, in execute
state, reward, terminal, _ = self.gym.step(action)
File "/home/kedarnath/Desktop/tf_deep_rl_trader-master/tf_deep_rl_trader-master/env/TFTraderEnv.py", line 82, in step
s, r, d, i = self._step(action)
File "/home/kedarnath/Desktop/tf_deep_rl_trader-master/tf_deep_rl_trader-master/env/TFTraderEnv.py", line 156, in _step
self.n_short))
FileNotFoundError: [Errno 2] No such file or directory: './info/ppo_1579793.4444944328_LS_0_0.info'

Not an issue: How to adapt for multiple datasets?

I'm loving working with your project.

I was wondering if you might be able to help me with something. I'm wanting to adapt your model to be able to run over multiple datasets. I have lots of different .CSV files containing the 5 minute historical data of many different stocks, however each dataset file is not very big. Each one only contains one trading day's worth of historical data. I'm basically trying to implement your reinforcement learning model on a daytrading type basis.

So, how might one modify your model to be able to train on multiple different datasets at once, when each dataset is fairly small, depicting past stocks that were day traded on in the past? Could it somehow loop through every file in the train and test folders, or would it be best to compile it into one dataset, and somehow have the model recognize a different ticket symbol so that it doesn't try to learn a pattern of pricing if the prices for the different tickers are all over the place. Would love your feedback! Thanks!

it shows Segmentation fault (core dumped) after 10 iterations

2019-08-02 02:18:35.623360: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open s
hared object file: No such file or directory
2019-08-02 02:18:35.623434: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open s
hared object file: No such file or directory
2019-08-02 02:18:35.623505: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open sha
red object file: No such file or directory
2019-08-02 02:18:35.623560: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open s
hared object file: No such file or directory
2019-08-02 02:18:35.623624: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot op
en shared object file: No such file or directory
2019-08-02 02:18:35.623686: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot op
en shared object file: No such file or directory
2019-08-02 02:18:35.623750: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared ob
ject file: No such file or directory
2019-08-02 02:18:35.623768: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1663] Cannot dlopen some GPU libraries. Skipping registering GPU devices...
2019-08-02 02:18:35.624180: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-08-02 02:18:35.723627: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least o
ne NUMA node, so returning NUMA node zero
2019-08-02 02:18:35.724424: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x564b519a8e50 executing computations on platform CUDA. Devices:
2019-08-02 02:18:35.724468: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): Tesla V100-SXM2-16GB, Compute Capability 7.0
2019-08-02 02:18:35.727145: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2200000000 Hz
2019-08-02 02:18:35.727827: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x564b514d92b0 executing computations on platform Host. Devices:
2019-08-02 02:18:35.727870: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): ,
2019-08-02 02:18:35.727997: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-08-02 02:18:35.728024: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187]
2019-08-02 02:18:36.257449: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu
_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilat
ion_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
start episode ... XBTUSD_5m_16000_train.csv at 7963
Tick: 8000/ Portfolio (krw-won): 984243.3935784912
Long: 3/ Short: 7
Tick: 8100/ Portfolio (krw-won): 936529.4092928643
Long: 12/ Short: 21
Tick: 8200/ Portfolio (krw-won): 915573.1424759856
Long: 22/ Short: 35
Tick: 8300/ Portfolio (krw-won): 862624.4792269642
Long: 35/ Short: 45
Tick: 8400/ Portfolio (krw-won): 878349.9144196685
Long: 49/ Short: 52
Tick: 8500/ Portfolio (krw-won): 863488.3085565092
Long: 60/ Short: 61
Tick: 8600/ Portfolio (krw-won): 776054.9662000438
Long: 71/ Short: 72
Tick: 8700/ Portfolio (krw-won): 763382.4955556361
Long: 82/ Short: 83
Tick: 8800/ Portfolio (krw-won): 734898.3695485827
Long: 90/ Short: 92
Tick: 8900/ Portfolio (krw-won): 685093.6483718295
Long: 103/ Short: 98
Segmentation fault (core dumped)

Any idea?

Test and train datasets are the same

Your test and train datasets are the same

render() got an unexpected keyword argument 'close'

first I want to thank you for your great share. It very rare to find trading reinforcement learning system with ppo.
I have an error when I run this code.
SInce i dont have talib installed i replaced this line :
import talib
by this
import ta as talib

I have then an error when i launch the code :
render() got an unexpected keyword argument 'close'
from core.py line 164 and 203
and closer line 67

how can i fix it?

Thank you,
Greg

Changes in pandas

The latest version of pandas 1.0.3 does not contain the pandas.stats section. Will there be any improvements in github?

How can I improve it ?

I can duplicate your result of episode 100 (1000000 -> 1586872.177567203 ), but I try to episode 200,300,500... , the result became worse , it make a loss .
My question is
1.How improve it , more samples , feature , episode , window_size or other?
2.How can I check the algorithmic is correct and bug free ?

how to print the number of long and short trade when an episode is finished?

Hi, thank you again for your work.
I have tried to put the number of trades inside the episode_finished function, but i cant find a good solution to do it. This is what I tried :

def episode_finished(r):
    reward = "%.6f" % (r.episode_rewards[-1])
    print("Finished episode {ep} of {totalep} after {ts} timesteps (reward: {reward})".format(ep=r.episode, ts=r.episode_timestep,
                                                                                 reward=reward, totalep=episodes_number))
    #print("with {l} long position and {s} short".format(l=long, s=short))
    #print("with {l} long position and short".format(l=n_long))
    #print("info:", info)
    
    if np.mean(r.episode_rewards[-1]) > 0 :
        r.agent.save_model(SAVE_DIR, append_timestep=False)
    return True

why do you add 4 states in the environement?

I went deeper in the code but their is something i cant understand.
In the file TFTraderEnv.py
I found this line inside the OhlcvEnv() class:
self.shape = (self.window_size, self.n_features+4)

I cant find those 4 features you add just before the training.
Where are they in the code?
Thank you for your awesome work.
greg