mnoukhov / emergent-compete Goto Github PK
View Code? Open in Web Editor NEWCode for Emergent Communication under Competition (AAMAS 2021)
License: GNU General Public License v3.0
Code for Emergent Communication under Competition (AAMAS 2021)
License: GNU General Public License v3.0
To test the usefulness of the state "context", we have a receiver agent (e.g. deterministic) paired with a sender that simply sends it's input as a message. Therefore the receiver is just trying to guess the bias based off of repeated interactions and with gradient-based methods should get 100% accuracy on the later rounds.
current situation with deterministic, bias [0,10] shows that it is not near 100% accuracy
but that the current state is indeed helpful to some extent (rewards per round shown here)
need to change:
look into:
implement MADDPG (https://arxiv.org/abs/1706.02275) to train the agents
Following #1, no try to achieve 100% accuracy without being given the extra state variable prev_diff
(previous signed distance to target). Instead use previous reward states to infer the same information.
Since only one previous reward state does not give you this information because it is unsigned (e.g. guess = 7
, distance = 1
means the true reward was either 6
or 8
) use two previous states which should allow to disambiguate
Does a protocol always emerge in the biased game when we expect it to?
We expect communication to emerge when the reward with communication > reward without communication. In our specific game this translates to bias < 2 pi
aka not fully competitive
Show that changing from cooperative to non-cooperative causes language drift
the current format of actions and loss is:
the issue is that the output of the network can be arbitrarily large or small, and the loss for some prediction x
is the same as the loss for x + k*2pi
for any integer k
. This means that the gradient can go towards one minima for some input x
and towards a different minima for another similar input x+eps
.
one way to fix this has been to initialize the network to strictly positive values that guarantee all outputs are in the range [0, circumference]
but this will be a lot harder if the model is larger, so perhaps it makes sense to change the loss function or output to be resilient to this
ideas:
k
from being output infinitymulti-agent deterministic - deterministic with non-zero bias and previous two states, following up on #6
now that #3 is solved for a deterministic policy learned with backprop, try to do the same thing learning with RL
current A2C method only matches one-shot baseline
but results across rounds look promising, test some hyperparameter tuning and see if we can't improve performance, otherwise maybe switch to DDPG?
It seems that the right way of logging with tensorboard could be
/agent_name/
)currently not updating live for unknown reasons
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.