entrpn / fingym

A tool for developing reinforcement learning algorithms focused in stock prediction

FinGym

Fingym is a toolkit for developing reinforcement learning algorithms tailored specifically for stock market trading. This is the fingym open-source library, which gives you access to a standardized set of environments.

See What's New section below

fingym makes no assumptions about the structure of your agent, and is compatible with any numerical computation library, such as TensorFlow or Theano. You can use it from Python code.

If you're not sure where to start, we recommend beginning with the docs on our site.

Contents of this document

• Basics
  • Supported systems
• Installation
• Environments
• Observations
• Actions
• Examples
• Testing
• What's new

Basics

There are two basic concepts in reinforcement learning: the environment (namely, the outside world) and the agent (namely, the algorithm you are writing). The agent sends actions to the environment, and the environment replies with observations and rewards (that is, a score).

The core fingym interface is Env, which is the unified environment interface. There is no interface for agents; that part is left to you. The following are the Env methods you should know:

• reset(self): Reset the environment's state. Returns observation.
• step(self, action): Step the environment by one timestep. Returns observation, reward, done, info.

Supported systems

We currently support Python 3.5 -- 3.7.

Installation

You can perform a minimal install of fingym with:

git clone git clone https://github.com/entrpn/fingym
cd fingym
pipenv shell
pipenv install -e .

If you prefer, you can do a minimal install of the packaged version directly from PyPI:

pip install fingym

Environments

See the fingym site.

Observations

See the fingym site.

Actions

See the fingym site.

Examples

See the examples directory.

• Run examples/agents/buy_and_hold_agent.py to run a simple buy and hold agent.
• Run examples/agents/random_agent.py to run a simple random agent.
• Run examples/agents/dqn_agent.py to run a dqn agent.
• Run examples/agents/evolutionary_agent.py to run a generic algorithm.
• Run examples/agents/evolutionary_agent_w_crossover.py to run a generic algorithm using crossover.

Testing

If you cloned this repo, add fingym to python path:

>> export PYTHONPATH=$PYTHONPATH:/path/to/fingym/fingym

We are using pytest for tests. You can run them via:

pytest

What's new

• 2020-02-05: First release. 3 year spy intraday minute steps. 10 year daily steps.
• 2020-02-26: More environments from different symbols.
• 2020-04-14: Renamed package from gym to fingym
• 2020-05-01: Added random walk environments and alphavantage environment.
• 2020-05-07: Added IEX environment with caching to reduce consumption of message quotes.

fingym's Issues

404 error - twin-delayed-ddpg

FYI, this link in your documentation gives a 404 error:
https://entrpn.github.io/fingym/agents/twin-delayed-ddpg

Deep Evolution

Dear Fingym Team!

I am delighted by the Deep Evolution Agent.
Is there a chance that a version for Forex gets published?
It should be able to do short trading, and the Environment should limit the agent to only buy, sell or hold one unit at each step. Like this it will be easier to adopt it for Live Trading.

Best regards and keep up the great work!

Matthias

error running evolutionary_agent_w_crossover

Hi, nice work, thanks for sharing.
I am getting the following error.
I think output_probabilities needs to be renormalized.

***** agent score *****
score:  99302.44000000003
score:  189596.57999999993
score:  279890.71999999986
***** agent score *****
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/data/projects/fingym/gym/examples/agents/evolutionary_agent_w_crossover.py", line 278, in <module>
    rewards = run_agents_n_times(env,agents,3) # average of x times
  File "/home/data/projects/fingym/gym/examples/agents/evolutionary_agent_w_crossover.py", line 148, in run_agents_n_times
    avg_score.append(return_average_score(env, agent, runs))
  File "/home/data/projects/fingym/gym/examples/agents/evolutionary_agent_w_crossover.py", line 141, in return_average_score
    score += run_agent(env, agent)
  File "/home/data/projects/fingym/gym/examples/agents/evolutionary_agent_w_crossover.py", line 130, in run_agent
    action = agent.act(state)
  File "/home/data/projects/fingym/gym/examples/agents/evolutionary_agent_w_crossover.py", line 74, in act
    action = np.random.choice(range(self.action_size),1,p=output_probabilities).item()
  File "mtrand.pyx", line 926, in numpy.random.mtrand.RandomState.choice
ValueError: probabilities do not sum to 1