tobiasnorlund / deepsim Goto Github PK

Simple market economy

Goal of experiment is to get started and set up environment for social simulation using reinforcement learning, through a very simple experiment with an open market economy. Idea is as follows.

There are two agent types, buyers and sellers that co-exist in the simulation environment. Buyers (representing regular people) have a hunger need (for food and water etc.) that need to be satisfied for the individual to stay healthy. To satisfy the need, a buyer can buy food from a seller to a negotiable price. The goal of each buyer is to minimize its percieved hunger need. Sellers (representing food providing corporations) want to maximize their profit by selling food to buyers as expensive as possible. If a buyer does not get enough food, he eventually dies and is removed from the environment.

The environment is initialised with a set of buyer and seller agents. The simulation is then carried out in discrete time steps. At each time step, each buyer is matched with a seller. The two agents individually place a bid on how much they are willing to pay/sell one pice of food for. If the buyer's bid is larger than the seller's, a transaction takes place. For the buyers to be able to afford food, they are provided with a small paycheck at each timestep.

The experiment will constitute two parts, the simulation environment and the decision making process.

Simulation environment (simulator)

The simulation environment is responsible for matching the buyers and sellers, collect bids and execute transactions. It also controls the buyers needs and whether they die as a consequence of not buying enough food.

Decision making process

The decision making process is responsible for deciding which actions each agent should take at each time step. In this case, the actions are what bid to place. Again, buyers want to minimize their hunger, and sellers to maximize their profit. The idea here is to let the decision making process be a reinforcement learning model that learns the optimal actions to take in a given environment.

Interesting observations:

• What incentives are created for sellers to lower their bids? Competition from other sellers? Make sure buyers survive?

Potential extensions:

• Sellers could be modelled to also have costs associated with their food production, and could go bankrupt if they don't sell anything.