edabati / fsdl-2022-weak-supervision-project

Hi there 👋

I'm currently working as an Applied ML Scientist at Thomson Reuters Labs.

• I've contributed to: scikit-learn, skops, pandas, setfit
• Personal projects and demos: Weak Supervision and Deep Learning with text data

📫 How to reach me: LinkedIn

fsdl-2022-weak-supervision-project's Issues

Create model UI for users

Effort: 1 person (max 2)

We should create a simple UI for users to query and interact with the model.
The UI should require an input text from the user, request the model prediction using the model API endpoint in services/api-serveless and show the prediction in an informative way.

Ideally it should be setup to run in a docker container (or AWS EC2 like the example course labs?).

Options for implementation:

• Gradio
• Streamlit

Tidying up the repo and documentation

Effort: 3-4 people

• Document everything
• Make sure that the project is reproducible
• Remove unused code and clean up folder structure
• etc

Export model to Torchscript or ONNX

Effort: 1 person

Once we have a model created in issue #7, we should export it to a format that is more production friendly.
The task is to create a script that takes a trained model, export it to a format and save it somewhere.

Format options:

• ONNX: https://huggingface.co/docs/transformers/main/en/serialization
• Torchscript: https://huggingface.co/docs/transformers/main/en/torchscript

Storage options: W&B or HugginfFace Hub (?)

Train 1st model using Weak Labels

Effort: 1 person (max 2)

We created a labeled dataset created with Weak Supervision available here: https://huggingface.co/datasets/bergr7/weakly_supervised_ag_news

We should now train the first model on that and save it to a place accessible for the rest of the team.

Suggested steps:

• fine-tune distilbert
• train using huggingface Transformers API
• track experiment in W&B: docs
• store the Model in the W&B model registry and/or hugging face hub

We should have a team space in W&B if I am not wrong.

Finish Active Learning script + documentation/readme

Effort: 2 people (max 3)

First attempt of an active learning loop with connection to Rubrix for labelling was made in branch active-learning-rubrix. See this notebook

Some parts are missing:

• use unlabelled data resulted from weak labelling
• decide on loop initialisation (probably using prediction of model created in #7)
• decide sampling strategy
• decide whether small-text should be used or we should code something from scratch
• decide if we can switch away from the @listener API to something more straightforward like this example

Also we should create a documentation explaining the design decisions, describing how an active learning loop could be used and deployed, etc.

FYI @bergr7