Coder Social home page Coder Social logo

scorpjd / collaboration-competition-maddpg Goto Github PK

View Code? Open in Web Editor NEW

This project forked from eyarhouma/collaboration-competition-maddpg

0.0 1.0 0.0 606 KB

My solution to Collaboration and Competition using MADDPG algorithm, Udacity 3rd project of Deep RL Nanodegree from the paper "Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments"

Home Page: https://arxiv.org/pdf/1706.02275.pdf

Python 21.23% Jupyter Notebook 78.77%

collaboration-competition-maddpg's Introduction

Deep Reinforcement Learning : Collaboration and Competition using MADDPG

In this repository i used MADDPG algorithm https://arxiv.org/pdf/1706.02275.pdf to solve Collaboration and Competition , Udacity's 3rd project for Deep RL Nanodegree from the paper "Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments"

Project's goal

Trained Agent

In this project, two agents control rackets to bounce a ball over a net. If an agent hits the ball over the net, it receives a reward of +0.1. If an agent lets a ball hit the ground or hits the ball out of bounds, it receives a reward of -0.01. Thus, the goal of each agent is to keep the ball in play.

The task is episodic, and in order to solve the environment, the agents must get an average score of +0.5 (over 100 consecutive episodes, after taking the maximum over both agents). Specifically,

  • After each episode, we add up the rewards that each agent received (without discounting), to get a score for each agent. This yields 2 (potentially different) scores. We then take the maximum of these 2 scores.
  • This yields a single score for each episode.

The environment is considered solved, when the average (over 100 episodes) of those scores is at least +0.5.

Solving the environment

The task is episodic, and in order to solve the environment, the agents must get an average score of +0.5 (over 100 consecutive episodes, after taking the maximum over both agents). Specifically,

  • After each episode, we add up the rewards that each agent received (without discounting), to get a score for each agent. This yields 2 (potentially different) scores. We then take the maximum of these 2 scores.
  • This yields a single score for each episode.

The environment is considered solved, when the average (over 100 consecutive episodes) of those scores is at least +0.5.

Setting up the environment

  1. The environment can be downloaded from one of the links below for all operating systems:

  2. Place the downloaded file in the same directory as this GitHub repository and unzip the file.

  3. Use the requirements.txt file to set up a python environment with all necessary packages installed.

Instructions

See the main file Tennis.ipynb to get an introduction to the environment and follow the steps to solving the environment. The main classes are defined in the file MADDPG_agent.py.

Approach and solution

The reinforcement learning approach we use in this project is called Multi Agent Deep Deterministic Policy Gradients (MADDPG). see this paper. In this model every agent itself is modeled as a Deep Deterministic Policy Gradient (DDPG) agent (see this paper) where, however, some information is shared between the agents.

In particular, each of the agents in this model has its own actor and critic model. The actors each receive as input the individual state (observations) of the agent and output a (two-dimensional) action. The crit`ic model of each actor, however, receives the states and actions of all actors concatenated.

Throughout training the agents all use a common experience replay buffer (a set of stored previous 1-step experiences) and draw independent samples.

Details of the implementation including the neural nets to model actor and critic models can be found in the modules MADDPG_agent.py and models.py as well as the report (report.pdf`). With the current set of models and hyperparameters the environment can be solved in around 3200 steps.

collaboration-competition-maddpg's People

Contributors

eyarhouma avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.