jaketae / pygrad

Pure Python autograd library based on NumPy

Hi there 👋 I'm Jake.

I'm passionate about generative modeling, text-to-speech, NLP, and recommendation systems.

I recently graduated from Yale University with a BSc in CS and Math. Previously, I was a

• Machine Learning & Software Engineer Intern at Facebook
• Machine Learning Engineer Intern at Hugging Face
• Machine Learning Research Intern at Neosapience

pygrad's Issues

Add type annotations

Is your feature request related to a problem? Please describe.
Type annotations are self-documenting, provide users and developers with better code completion ,and is now a norm in many open source libraries.

Describe the solution you'd like
Use the built-in typing to annotate function signatures, i.e. arguments and return types.

Describe alternatives you've considered
Docstrings might help, but type annotations are more concise and self-explanatory.

Setup CI/CD

Is your feature request related to a problem? Please describe.
Setup CI/CD for automatic unit testing and coverage reporting.

Describe the solution you'd like
On push and PR/merge, a pipeline has to kick into make sure that

• all unit tests pass
• all type annotations match
• appropriate listing/styling have been applied

Optimizer error when param.grad is None

Describe the bug
Optimizers may attempt to update parameters whose gradients are None. Since NoneType cannot be broadcasted into float, this returns an error.

To Reproduce
Steps to reproduce the behavior:

1. Create a neural network (nn.Module instance)
2. Create an optimizer (nn.optim.Adam)
3. Attempt model weights update via optimizer.step()
4. Function call returns AttributeError: 'NoneType' object has no attribute 'data'

Expected behavior
Optimizer should only seek to update parameters whose gradient values are not None.

Add contributing guidelines

Is your feature request related to a problem? Please describe.
Delineate contributing guidelines for other contributors.

Describe the solution you'd like
Add CONTRIBUTING.md.

Add unit tests

Is your feature request related to a problem? Please describe.
Currently, there is one one very basic unit test case. Write more tests before expanding or modifying any API.

Describe the solution you'd like

• Tests for each function in pygrad.functions
• Test for pygrad.nn module components
• Test for pygrad.data components