docomoinnovations / aws-serverless-vertical-federated-learning Goto Github PK

Is your feature request related to a problem? Please describe.
VFL client is able to access the intermediate files stored in the S3 bucket such as embeddings and gradients of other VFL clients.

Describe the solution you'd like
Introduce S3 prefix for intermediate files and corresponding IAM policy for each client to restrict the access to the intermediate files from other VFL clients

Describe alternatives you've considered
N/A

Additional context
N/A

Is your feature request related to a problem? Please describe.
Avoid using torch.load to follow the best practice.

Describe the solution you'd like
Use other input/output method instead of torch.load

Describe alternatives you've considered

• CSV input/output based on Numpy

Additional context
N/A

Is your feature request related to a problem? Please describe.
No, it's just for refactoring the program.

Describe the solution you'd like
Introducing DI (Dependency Injection) for ClientTrainer.

Describe alternatives you've considered
N/A

Additional context
N/A

Is your feature request related to a problem? Please describe.
SQS queues are provided per VFL client. In the current implementation, the VFL client can access the queue intended for the other clients. It should only be restricted by the appropriate VFL client.

Describe the solution you'd like

• Introduce SQS resource policy to restrict access from the other VFL client
• Provide a script to create IAM users for VFL clients as for convenience because SQS resource policy needs specific IAM user names

Describe alternatives you've considered
N/A

Additional context
N/A

Is your feature request related to a problem? Please describe.
No, it's just for management stuff on the GitHub repository.

Describe the solution you'd like
Modify the .gitignore file on top of the repository to ignore tempfiles of Python

Describe alternatives you've considered
N/A

Additional context
N/A

Is your feature request related to a problem? Please describe.
No, it would be just for keeping the code quality.

Describe the solution you'd like
Add unit test for client training.

Describe alternatives you've considered
N/A

Additional context
N/A

Is your feature request related to a problem? Please describe.
No, it's just for refactoring VFL client training program.

Describe the solution you'd like
Introduce DI (Dependency Injection) for ClientTrainer

Describe alternatives you've considered
N/A

Additional context
N/A

Is your feature request related to a problem? Please describe.
No, it would be just for keeping the code quality.

Describe the solution you'd like

• Add unit test code for server model

Describe alternatives you've considered
N/A

Additional context
N/A

Is your feature request related to a problem? Please describe.
The OS version to execute CLI is important information to trace error/exception when it occurs.

Describe the solution you'd like
Add the OS version to README.

Describe alternatives you've considered
N/A

Additional context
N/A

Is your feature request related to a problem? Please describe.
The container images of the AWS Lambda function seem to include unnecessary setup tools.

Describe the solution you'd like
Remove unnecessary Python setup tool from the container images

Describe alternatives you've considered
N/A

Additional context
The images for setting up the VFL server and VFL server training are the scope of this issue.

Is your feature request related to a problem? Please describe.
ClientTrainer class takes too many tasks. This may cause the difficulty to change the code in the future.

Describe the solution you'd like
Remove the tasks which are not related the client training directly. Manage the tasks not related to client training outside of ClientTrainer class.

Describe alternatives you've considered
N/A

Additional context
N/A

Is your feature request related to a problem? Please describe.
The reference of the data set which used for the experiment of VFL is not clear.

Describe the solution you'd like
Add clear reference of the dataset.

Describe alternatives you've considered
N/A

Additional context
N/A

Describe the bug
When running VFL training, the VFL client fails with the error below:

Traceback (most recent call last):
  File "local_training.py", line 404, in <module>
    client_trainer.start()
  File "local_training.py", line 234, in start
    self.embed_file = self.__forward(self.session)
  File "local_training.py", line 277, in __forward
    return self.__save_embed(s3_bucket, key)
  File "local_training.py", line 251, in __save_embed
    self.s3.meta.client.upload_file(file_name, bucket, key)
AttributeError: 'ClientTrainer' object has no attribute 's3'

To Reproduce
Steps to reproduce the behavior:

1. Start server training
2. Start client training
3. See error in the first step

Expected behavior
VFL client doesn't have error

Screenshots
N/A

Desktop (please complete the following information):

• OS: M1 Mac (Client)

Smartphone (please complete the following information):
N/A

Additional context
N/A