duplys / perceptron

Jupyter notebooks illustrating some basics of machine learning

Summary 🤖

This repository contains Jupyter notebooks illustrating few basics of machine learning, primarily the basics of artificial neural networks. I created it to explain working principles behind machine learning to technical leadership at the company I work for.

The learning journey starts with a simple perceptron introduced in 1957, progresses through the development of multi-layer perceptrons and recurrent neural networks (RNNs) for sequence-to-sequence tasks, and finishes with modern-day transformer architecture and large language models (LLMs). For explanations of sequence-to-sequence models, RNNs, and transformers, I primarily reference the excellent NLP course by Lena Voita.

The following table of contents is helpful if you want to navigate the material in logical order.

Table of contents

• Perceptron
• Multi-layer Perceptron (MLP)
  • Backpropagation
  • Training an MLP
    • Gradient
  • MNIST example
• Sequence-to-sequence tasks
• Encoder-decoder framework
• Recurring Neural Networks (RNNs)
  • RNN example
• Attention
• Self-Attention
• 🚧 Positional Encoding
• 🚧 Normalization
• Transformer architecture
  • Foundation Models
  • 🚧 Generative Pre-trained Transformer
• Use cases for generative AI
  • Retrieval-Augmented Generation (RAG)
  • Image segmentation
• Hands-on
  • 🚧 Hands-on: GitHub Copilot
  • 🚧 Hands-on: Using LLMs
• Business impact
  • HAI AI Index Report

Running the Jupyter notebooks

The requirements.txt file in the root of this repository lists all Python packages and their corresponding versions installed in my Python virtual environment (the file was obtained by pip freeze > requirements.txt).

perceptron's Issues

Graphviz

dot = graphviz.Digraph(comment='The Round Table')

dot.node('A', 'King Arthur')
dot.node('B', 'Sir Bedevere the Wise')
dot.node('L', 'Sir Lancelot the Brave')

dot.edges(['AB', 'AL'])
dot.edge('B', 'L', constraint='false')

dot