aayushsameershah / neural-net-zero-to-hero-with-andrej

This repository contains the collection of explorative notebooks pure in python and in the language that we, humans can read. Have tried to compile all lectures from the Andrej Karpathy's 💎 playlist on Neural Networks - which we will end up with building GPT.

Heiye!

I am Aayush ∞ Shah and I am a Data Science passioniate from Bhuj and trying to move on to leave comfortzone. I love to learn things but hate when I don't note them down make it accessible to everyone. Have made many projects and content in the form of books available on my GitHub.

Passoniate in:
—

🤓  Learning & Noting stuff — Beautifully.
👨‍💻  Creating things from scratch.
✨  Pretty write styling.
🤩  Presenting beautiful vizzes.
🎨  UI Designer by Hobby.

Proffessional in:
—

💡  Grabbing insights from handsome data.
📈  Analyzing trends and presentation in simple words.
🤖  Training and Predicting using ML models.
🎯  Evaluation of models and improving accuracy.
#️⃣  Statistics and data metrices.

Currently Learning:
—

📊 50 Essential concepts in Statistics: This is A-Z in statistics! A fun to learn book.
🧠 Spelled out GPT: Making a notebook course to create GPT from scratch! Amazed to learn with Andrej Karpathy 🙌

Handy code snippents:
—

📜 My Gists: Here, I have stored many useful code snippets which become handy across the projects.

My Special Friends:
—

  •   •  • 

neural-net-zero-to-hero-with-andrej's Issues

Micrograd subtraction gradient is wrong

I'm not very experienced with calculus, but my intuition along with ChatGPT tells me that this is wrong:

def __sub__(self, other):
    other = Value(other) if not isinstance(other, Value) else other
    out = Value(self.data - other.data, (self, other), '-')
    def _backward():
        self.grad += 1.0 * out.grad
        other.grad += 1.0 * out.grad
    out._backward = _backward
    return out

Shouldn't the gradient of other be other.grad += -1.0 * out.grad?