chriswaites / pyvacy Goto Github PK

I tried to train text classifier using LSTM and the accuracy is always 0.51 and below

I am recently finetuning a gpt-like model on some toy-dataset with fake personal data. I am trying to test differential privacy on this LLMs. I wonder if pyvacy will work for this task, or is there any other better options?

nn.Flatten was not implemented in the version of PyTorch specified in environment.yml, version pytorch=1.0.1

The recommended way to have this functionality for use in Sequential blocks before it was added to PyTorch was to define a class for it.

why microbatch can not equal to minibatch ?
when i set microbatch equal to minibatch , there are error as follow:
AttributeError: 'NoneType' object has no attribute 'data'

The example implementation given here shows that data is broken down into minibatches and then into microbatches of 1 or few samples each. I was wondering instead of calculating loss like: -

 for X_microbatch, y_microbatch in microbatch_loader(TensorDataset(X_minibatch, y_minibatch)):
        optimizer.zero_microbatch_grad()
        loss = loss_function(model(X_microbatch), y_microbatch)
        loss.backward()
        optimizer.microbatch_step()
    optimizer.step()

what if I did: -

 for x_minibatch, y_minibatch in minibatch_loader(train_dataset):
        loss = [] #List of losses from each microbatch
        for microbatch in minibatch
               loss.update(loss_function(model(X_microbatch), y_microbatch))
        
        for loss_mini in loss:        
              optimizer.zero_microbatch_grad()
              loss_mini.backward(retain_graph = True)
              optimizer.microbatch_step()
    optimizer.step()

Would this implementation give different results?

Hi,

Is it possible to apply DP with different values of epsilon?

Thanks,

Hi Chris,
Thank you for your codes in privacy algorithm implemented in Pytorch. I'm a new learner in Privacy field and I'm studying attribute inference / membership inference. TensorFlow Privacy repository mainly includes privacy algorithms in DP. However, it includes few codes about attribute inference privacy.

Could you upload some codes about attribute / membership inference in the future?

Thanks in advance!

Hi, thanks for your codes. I just ran the minist.py in the tutorial. But it said that
ImportError: cannot import name 'sampling' from 'pyvacy'

I want to know how to do with this.

Thanks!