jungangge / drl_for_ddbc

Simulation codes for the manuscript "Deep Reinforcement Learning for Distributed Dynamic MISO Downlink-Beamforming Coordination" submitted to IEEE Transactions on Communications

Simulation codes for the manuscript "Deep Reinforcement Learning for Distributed Dynamic MISO Downlink-Beamforming Coordination", which has been accepted for publication in IEEE Transactions on Communications.

---

Requirements to run the simulation programs and plot the figures

The simulation programs require Python3 with installed packages such as Keras, Tensorflow, Numpy, Scipy and etc. The figures are plot by MATLAB.

Structure of the demo simulation program

./DRL_for_DDBC/codebook/codebook.mat the file for saving the codebook matrix used in current simulation program.

./DRL_for_DDBC/data the folder to save the simulation results of the four schemes.

./DRL_for_DDBC/rates the folder to save the achievable rate of each cell within the simulation process

./DRL_for_DDBC/base_station.py the simulator of the base station

./DRL_for_DDBC/cellular_network.py the simulator of the cellular network

./DRL_for_DDBC/channel.py the simulator of the channels

./DRL_for_DDBC/config.py the file to save the configuration of the current simulation program

./DRL_for_DDBC/data_process.py the python script to plot the simulation results of the current simulation program

./DRL_for_DDBC/dqn_for_singleagent.py the DQN agent at each BS

./DRL_for_DDBC/drl.py the DTDE DRL-based scheme

./DRL_for_DDBC/export_locations.py export the locations of BSs and UEs as .mat files

./DRL_for_DDBC/fp_algorithm.py the ideal FP approach

./DRL_for_DDBC/functions.py some extra functions requied in the simulation

./DRL_for_DDBC/greedy.py the greedy scheme

./DRL_for_DDBC/neural_network.py the file to save the configurations of the neural network in DQN

./DRL_for_DDBC/random_choose.py the random scheme

./DRL_for_DDBC/user_equipment.py the simulator of the user equipments

In each file above, there are detailed annotations to help you understand the simulation program.

drl_for_ddbc's Issues

Could you specify LICENSE for your codes?

I am trying to do related research.
Could you please specify the license to reuse the code?
If I reuse the code, I will provide the appropriate citation.

A few questions for the codes in channel.py

Hello Jungang,
Appreciate for sharing your codes and your impressive work.
I am a beginner in wireless communication and AI. Aftering reading your codes, I have a few confusions in your calculation of channel vector h. In your paper, the expression of h is :

Here are my questions:

1. In channel.py line 46, why using self.h instead of self.H for calculating channel gain? Should the path loss be ignored in the drl method? And why the path loss wasn't being divided by np.sqrt(self.multi_paths)?
2. In channel.py line 38 the calculation of steering vector, why it is divided by np.sqrt(self.bs.n_antennas) ?

For the error in channel.py

Dear Jungang:

In your init part,
self.g = (np.random.randn(1, self.multi_paths) + np.random.randn(1, self.multi_paths) * 1j) / np.sqrt(2 * self.multi_paths)

and in the ir_change part, it is
e = (np.random.randn(1, self.multi_paths) + np.random.randn(1, self.multi_paths) * 1j)
* np.sqrt(1 - np.square(self.rho)) / np.sqrt(2)
it seems that you miss the self.multi_paths in the ir_change part, this problem greatly increase the strength of the channel you use, it should be
e = (np.random.randn(1, self.multi_paths) + np.random.randn(1, self.multi_paths) * 1j)
* np.sqrt(1 - np.square(self.rho)) / np.sqrt(2 *self.multi_paths )
I am not sure whether this will influence all the results in your paper.(maybe the results should be lower in your paper)

you can check the norm value of self.g by using np.linalg.norm(self.g) in cascaded time slots and you can find that the strength increases a lot from slot 1 to slot 2.

Bests,

Kris

Hello,sorry to bother you.

Hello, sorry to bother you. I have been reading your paper and code "Deep Reinforcement Learning for Distributed Dynamic MISO Downlink-Beamforming Coordination" recently. I still don't understand the way of generating codebook files mentioned in the article. Could you please upload the source code of generating codebook files?

DRL_for_DDBC variable: "ir_change" issue

Hi Jungang

I am a beginner of AI

1. I don't understand what is "ir_change" with \DRL_for_DDBC-master\code\train_manner\DTDE\DRL_for_DDBC\drl.py .
   cn.update(ir_change=False, actions=actions)
cn.update(ir_change=True))

why use the ir_change= False & True to update cn?

2. variable of channel.py have a 11 or 01 ?
   self.r_power11
self.r_power01

What do those two numbers mean?