docomoinnovations / aws-serverless-vertical-federated-learning Goto Github PK
View Code? Open in Web Editor NEWCloud-Native Architecture for Vertical Federated Learning
License: MIT License
Cloud-Native Architecture for Vertical Federated Learning
License: MIT License
Is your feature request related to a problem? Please describe.
VFL client is able to access the intermediate files stored in the S3 bucket such as embeddings and gradients of other VFL clients.
Describe the solution you'd like
Introduce S3 prefix for intermediate files and corresponding IAM policy for each client to restrict the access to the intermediate files from other VFL clients
Describe alternatives you've considered
N/A
Additional context
N/A
Is your feature request related to a problem? Please describe.
Avoid using torch.load
to follow the best practice.
Describe the solution you'd like
Use other input/output method instead of torch.load
Describe alternatives you've considered
Additional context
N/A
Is your feature request related to a problem? Please describe.
No, it's just for refactoring the program.
Describe the solution you'd like
Introducing DI (Dependency Injection) for ClientTrainer.
Describe alternatives you've considered
N/A
Additional context
N/A
Is your feature request related to a problem? Please describe.
SQS queues are provided per VFL client. In the current implementation, the VFL client can access the queue intended for the other clients. It should only be restricted by the appropriate VFL client.
Describe the solution you'd like
Describe alternatives you've considered
N/A
Additional context
N/A
Is your feature request related to a problem? Please describe.
No, it's just for management stuff on the GitHub repository.
Describe the solution you'd like
Modify the .gitignore file on top of the repository to ignore tempfiles of Python
Describe alternatives you've considered
N/A
Additional context
N/A
Is your feature request related to a problem? Please describe.
No, it would be just for keeping the code quality.
Describe the solution you'd like
Add unit test for client training.
Describe alternatives you've considered
N/A
Additional context
N/A
Is your feature request related to a problem? Please describe.
No, it's just for refactoring VFL client training program.
Describe the solution you'd like
Introduce DI (Dependency Injection) for ClientTrainer
Describe alternatives you've considered
N/A
Additional context
N/A
Is your feature request related to a problem? Please describe.
No, it would be just for keeping the code quality.
Describe the solution you'd like
Describe alternatives you've considered
N/A
Additional context
N/A
Is your feature request related to a problem? Please describe.
The OS version to execute CLI is important information to trace error/exception when it occurs.
Describe the solution you'd like
Add the OS version to README.
Describe alternatives you've considered
N/A
Additional context
N/A
Is your feature request related to a problem? Please describe.
The container images of the AWS Lambda function seem to include unnecessary setup tools.
Describe the solution you'd like
Remove unnecessary Python setup tool from the container images
Describe alternatives you've considered
N/A
Additional context
The images for setting up the VFL server and VFL server training are the scope of this issue.
Is your feature request related to a problem? Please describe.
ClientTrainer class takes too many tasks. This may cause the difficulty to change the code in the future.
Describe the solution you'd like
Remove the tasks which are not related the client training directly. Manage the tasks not related to client training outside of ClientTrainer class.
Describe alternatives you've considered
N/A
Additional context
N/A
Is your feature request related to a problem? Please describe.
The reference of the data set which used for the experiment of VFL is not clear.
Describe the solution you'd like
Add clear reference of the dataset.
Describe alternatives you've considered
N/A
Additional context
N/A
Describe the bug
When running VFL training, the VFL client fails with the error below:
Traceback (most recent call last):
File "local_training.py", line 404, in <module>
client_trainer.start()
File "local_training.py", line 234, in start
self.embed_file = self.__forward(self.session)
File "local_training.py", line 277, in __forward
return self.__save_embed(s3_bucket, key)
File "local_training.py", line 251, in __save_embed
self.s3.meta.client.upload_file(file_name, bucket, key)
AttributeError: 'ClientTrainer' object has no attribute 's3'
To Reproduce
Steps to reproduce the behavior:
Expected behavior
VFL client doesn't have error
Screenshots
N/A
Desktop (please complete the following information):
Smartphone (please complete the following information):
N/A
Additional context
N/A
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.