ashwinrj / federated-learning-pytorch Goto Github PK

Hi, I want to try the model on the new dataset, which py files will i need to change? (utils.py and ?)

I think in this function we should divide the loss at the end by len(testloader), or am I missing something?

There is a writing mistake in /src/utils.py.

In function get_dataset(args) (line24),

        train_dataset = datasets.MNIST(data_dir, train=True, download=True,
                                       transform=apply_transform)

        test_dataset = datasets.MNIST(data_dir, train=False, download=True,
                                      transform=apply_transform)

should be

        train_dataset = datasets.CIFAR10(data_dir, train=True, download=True,
                                       transform=apply_transform)

        test_dataset = datasets.CIFAR10(data_dir, train=False, download=True,
                                      transform=apply_transform)

Hi thanks for providing this wonderful repository, but I'm wondering if there will be support for parallelization of client training in each round

specifically, making the local update in federated_main.py to be executed by parallel processes

for idx in idxs_users:
            local_model = LocalUpdate(args=args, dataset=train_dataset,
                                      idxs=user_groups[idx], logger=logger)
            w, loss = local_model.update_weights(
                model=copy.deepcopy(global_model), global_round=epoch)
            local_weights.append(copy.deepcopy(w))
            local_losses.append(copy.deepcopy(loss))

or

are there suggestions for start working on this approach?

the code should be the change to the place marked in the red box

While running the code, the following attribute errors were coming. Can anyone tell the reasons for such errors??
For equal case:

Traceback (most recent call last):
  File "src/federated_main.py", line 36, in <module>
    train_dataset, test_dataset, user_groups = get_dataset(args)
  File "C:\Users\sharm\Downloads\Federated-Learning-PyTorch-master\src\utils.py", line 41, in get_dataset
    user_groups = cifar_noniid(train_dataset, args.num_users)
  File "C:\Users\sharm\Downloads\Federated-Learning-PyTorch-master\src\sampling.py", line 173, in cifar_noniid
    labels = np.array(dataset.train_labels)
  File "C:\Users\sharm\.conda\envs\newEnv\lib\site-packages\torch\utils\data\dataset.py", line 83, in __getattr__
    raise AttributeError
AttributeError

For Unequal case:

Traceback (most recent call last):
  File "src/federated_main.py", line 36, in <module>
    train_dataset, test_dataset, user_groups = get_dataset(args)
  File "C:\Users\sharm\Downloads\Federated-Learning-PyTorch-master\src\utils.py", line 38, in get_dataset
    raise NotImplementedError()
NotImplementedError

In this for loop, are we looping over all users? The current code seems to be fixed for all users.

In federated_main.py file , once you average out the weights, you calculate accuracy on TEST set of each client. So why is that you're calling it train accuracy/loss?

Anything I'm missing here?

Is this really training accuracy? Because in the update.py file the accuracy is computed as the test accuracy for the local test data.

Hi I tried to run "python src/federated_main.py --model=cnn --dataset=cifar --gpu=0 --iid=1 --epochs=10"
but is not working. (in any option w federate_main.py including dataset, model, so)

I found several issues from your git and modified those parts, but it seems like there r additional problem w loop of 'federated_main.py'.

Is there anyone else who r suffering from same issue or have fixed them?

The result of paper is so good ,but i can't get similar result by this code

In the MLP model, I think in the last layer it should be F.log_softmax instead of softmax.
Otherwise, the NLL loss would return negative values.

Traceback (most recent call last):
File "federated_main.py", line 33, in
torch.cuda.set_device(args.gpu)
File "D:\Anaconda3\lib\site-packages\torch\cuda_init_.py", line 243, in set_device
device = _get_device_index(device)
File "D:\Anaconda3\lib\site-packages\torch\cuda_utils.py", line 20, in _get_device_index
device = torch.device(device)
RuntimeError: Expected one of cpu, cuda, mkldnn, opengl, opencl, ideep, hip, msnpu device type at start of device string: 0

Hi, thanks for the code.

According to the lines in update.py:

if self.args.optimizer == 'sgd':
    optimizer = torch.optim.SGD(model.parameters(), lr=self.args.lr,
                                momentum=0.5)
elif self.args.optimizer == 'adam':
    optimizer = torch.optim.Adam(model.parameters(), lr=self.args.lr,
                                 weight_decay=1e-4)

The optimizer is created for every epoch, is that correct?

It seems to me that the following line shouldn't be inside the loop and it should be moved outside the loop.

The accuracy is about 40% under the following condition. Is there an improving way?
Local epoch:10
batch size: 10
global epoch: 50
learning rate: 0.5 ~ 0.001

Anybody any idea ?

Line 105, there should be
idxs=user_groups[c]
because we need to traverse all users

Here, I think we have to replace the idx with c because we want to calculate the Training error of the global_model on all training data (after the averaging).

In the original paper, it uses a weighted average here. However, the implementation in average_weights is the simple average. Is there a bug or do I misunderstand something? Thanks!

While executing federated learning code and MLP code i am getting this error
raceback (most recent call last):
File "src/federated_main.py", line 129, in
with open(file_name, 'wb') as f:
FileNotFoundError: [Errno 2] No such file or directory: '../save/objects/cifar_cnn_5_C[0.1]_iid[1]_E[10]_B[10].pkl'
170500096it [09:57, 285128.32it/s]
whether i have to create some files

Hello, i am just curious to know, what does line 72 in update.py do? Does it forward the image to the model?
log_probs = model(images)

raise ValueError("batch_size should be a positive integer value, "
ValueError: batch_size should be a positive integer value, but got batch_size=0

First thanks for the amazing work, but when I want to run CNN on CIFAR10 dataset there is some issue it got runtime error I wonder how to solve it. And the link is the error message.

Is in update.py line 64

Thanks for you work again.

Hello: I ran 'python src/federated_main.py --model=cnn --dataset=mnist --iid=0 --epochs=10 --gpu=1'
But keep receiving error message:
Traceback (most recent call last):
File "src/federated_main.py", line 34, in
if args.gpu_id:
AttributeError: 'Namespace' object has no attribute 'gpu_id'

I am using Windows 10 and make sure I have GPU and GPU 1 in my task manager. Thanks

Hello, your project has helped me a lot. Thank you very much. But I have a question: why do I need copy.deepcopy(model) when I am trying to implement a federated learning model, it seems that without copy.deepcopy all models will have the same weight. It's only when you use it that the model is different. So why is that?

1. I wonder why can I run "federated_main.py" on only one GPU (stand alone deployment). Because I got the acc.png and loss.png, so I believed that I do run this .py successfully, is that right? Does the codes and experiments involve communication period? Can this be called federated learning?
2. If so, which sentences of the codes realize the communication?
3. How to get the information( specific figuresf) of its communication time and the volume of communication data?

Looking forward to somebody's reply. Millions of thanks!

1.为什么我能在单机上跑通 "federated_main.py"文件？因为我在单台服务器上运行依旧得到了loss.png和acc.png，所以我认为我应该是跑通了。但这其中有没有通信？能算真正的联邦吗？
2.如果可以的话，到底是哪行代码实现的通信呢？
3.怎么能够获得通信时间和通信数据量这些信息？
期待热心网友的解答 谢谢！

Whether support multi - machine training,could you support an example?