List of references and online resources related to data science, machine learning and deep learning.
Recent Deep Learning Links (https://deep-learning-links.carrd.co/)
- fast.ai (https://www.fast.ai/)
- Walk with fastai (https://walkwithfastai.com/)
- Practical Deep Learning for Coders (https://course.fast.ai/)
- Hugging Face Course (https://huggingface.co/course/chapter1)
- MIT 6.S191: Introduction to Deep Learning (http://introtodeeplearning.com/)
- Stanford University CS231n: Convolutional Neural Networks for Visual Recognition (http://cs231n.stanford.edu/)
- Stanford University CS193p: Developing Applications for iOS using SwiftUI (https://cs193p.sites.stanford.edu/)
- Stanford University CS224N: Natural Language Processing with Deep Learning | Winter 2021 (https://www.youtube.com/playlist?list=PLoROMvodv4rOSH4v6133s9LFPRHjEmbmJ)
- Stanford University CS25: Transformers United (https://web.stanford.edu/class/cs25/index.html)
- Stanford University CS230: Deep Learning (https://cs230.stanford.edu/)
- Yann LeCun’s Deep Learning Course at CDS (https://cds.nyu.edu/deep-learning)
- UC Berkeley - Full Stack Deep Learning (https://fullstackdeeplearning.com/)
- New York University - PyTorch Deep Learning (https://atcold.github.io/pytorch-Deep-Learning/)
- University of Amsterdam - UvA Deep Learning Tutorials! (https://uvadlc-notebooks.readthedocs.io/en/latest/) (https://www.youtube.com/playlist?list=PLdlPlO1QhMiAkedeu0aJixfkknLRxk1nA)
- AI Hub (https://aihub.cloud.google.com/)
- DEEP LEARNING WITH PYTORCH: A 60 MINUTE BLITZ (https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html)
- Lightning Flash (https://lightning-flash.readthedocs.io/en/latest/index.html)
- TensorFlow Tutorials (https://www.tensorflow.org/tutorials/)
- Keras Guide (https://www.tensorflow.org/guide/keras/sequential_model)
- Machine Learning Crash Course with TensorFlow APIs (https://developers.google.com/machine-learning/crash-course/ml-intro)
- Crash Course - MXNet, Gluon (https://mxnet.apache.org/versions/1.8.0/api/python/docs/tutorials/getting-started/crash-course/index.html)
- GluonCV: a Deep Learning Toolkit for Computer Vision (https://cv.gluon.ai/contents.html)
- AutoGluon: AutoML for Text, Image, and Tabular Data (https://auto.gluon.ai/stable/index.html)
- 10 minutes to pandas (https://pandas.pydata.org/pandas-docs/stable/user_guide/)
- XGBoost (https://xgboost.readthedocs.io/en/latest/index.html)
- XGBoost - Notes on Parameter Tuning (https://xgboost.readthedocs.io/en/latest/tutorials/param_tuning.html)
- Kaggle (https://www.kaggle.com/learn)
- The Super Duper NLP Repo (https://notebooks.quantumstat.com/)
- Hugging Face Transformers Notebooks (https://huggingface.co/transformers/master/notebooks.html)
- Hugging FaceCommunity Notebooks (https://huggingface.co/transformers/master/community.html)
- Machine Learning for Beginners (https://github.com/microsoft/ML-For-Beginners)
- BigQuery cookbook (https://support.google.com/analytics/answer/4419694#zippy=%2Cin-this-article)
- PythonAlgos (https://pythonalgos.com/resources/)
- Captum - an open source, extensible library for model interpretability built on PyTorch (https://captum.ai/docs/introduction)
- Pinecone - A managed, cloud-native vector database with a simple API (https://www.pinecone.io/learn/)
- ML YouTube Courses (https://github.com/dair-ai/ML-YouTube-Courses)
- Towhee Codelabs (https://codelabs.towhee.io/)
- Emma Ding - Data Science Resources (https://www.emmading.com/free-data-science-interview-resources)
- NLPlanet - Practical NLP with Python (https://www.nlplanet.org/course-practical-nlp/index.html)
- Haystack (https://haystack.deepset.ai/tutorials)
- Zero to GPT (https://github.com/VikParuchuri/zero_to_gpt)
- Interactive Coding Challenges (https://github.com/donnemartin/interactive-coding-challenges)
- Machine Learning and AI Books (https://mltechniques.com/shop/)
- Google Machine Learning Education (https://developers.google.com/machine-learning)
-
https://developers.google.com/machine-learning/guides/deep-learning-tuning-playbook/faq
- Jerry Hargrove - AWS Cloud Diagrams & Notes (https://www.awsgeek.com/)
- Cheat Sheets for Machine Learning and Data Science - by Aqeel Anwar (https://sites.google.com/view/datascience-cheat-sheets)
- Cheat Sheets for Machine Learning Interview Topics - by Aqeel Anwar (https://medium.com/swlh/cheat-sheets-for-machine-learning-interview-topics-51c2bc2bab4f)
- ML Cheatsheet (https://ml-cheatsheet.readthedocs.io/en/latest/index.html)
- Convolutional Neural Networks cheatsheet (https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-convolutional-neural-networks)
- Deep Learning cheatsheet (https://stanford.edu/~shervine/teaching/cs-229/cheatsheet-deep-learning)
- Probability Cheatsheet (http://www.wzchen.com/probability-cheatsheet/)(https://static1.squarespace.com/static/54bf3241e4b0f0d81bf7ff36/t/55e9494fe4b011aed10e48e5/1441352015658/probability_cheatsheet.pdf)
- Bayesian (https://blogs.kent.ac.uk/jonw/files/2015/04/Puza2005.pdf)
- https://github.com/kailashahirwar/cheatsheets-ai
- pip command options (https://manpages.debian.org/stretch/python-pip/pip.1)
- RAG cheatsheet (https://miro.com/app/board/uXjVNvklNmc=/)
- 28 Jupyter Notebook Tips, Tricks, and Shortcuts (https://www.dataquest.io/blog/jupyter-notebook-tips-tricks-shortcuts/)
- How do I terminate active resources that I no longer need on my AWS account? (https://aws.amazon.com/premiumsupport/knowledge-center/terminate-resources-account-closure/)
- AWS Resource Hub (https://resources.awscloud.com/)
- AWS Machine Learning University (https://aws.amazon.com/machine-learning/mlu/)
- Boto3 documentation (https://boto3.amazonaws.com/v1/documentation/api/latest/index.html#)
- Amazon SageMaker Python SDK (https://sagemaker.readthedocs.io/en/stable/index.html)
- Learn Python On AWS Workshop (https://learn-to-code.workshop.aws/)
- Sagemaker Immersion Day - Self-Paced Lab (https://sagemaker-immersionday.workshop.aws/en/prerequisites/option2.html)
- Data Engineering Immersion Day (https://catalog.us-east-1.prod.workshops.aws/workshops/976050cc-0606-4b23-b49f-ca7b8ac4b153/en-US)
- QuickSight Workshops (https://catalog.workshops.aws/quicksight/en-US)
- Starter kit for the AWS Deepracer Challenge (https://gitlab.aicrowd.com/deepracer/neurips-2021-aws-deepracer-starter-kit)
- Amazon SageMaker Examples (https://github.com/aws/amazon-sagemaker-examples)
- Amazon SageMaker Examples Notebooks (https://sagemaker-examples.readthedocs.io/en/latest/index.html)
- Amazon SageMaker Course by Chandra Lingam (https://github.com/ChandraLingam/AmazonSageMakerCourse)
- MLOps Workshop with Amazon SageMaker (https://github.com/aws-samples/amazon-sagemaker-mlops-workshop)
- Managed Spot Training and Checkpointing for built-in XGBoost (https://github.com/aws-samples/amazon-sagemaker-managed-spot-training/blob/main/xgboost_built_in_managed_spot_training_checkpointing/xgboost_built_in_managed_spot_training_checkpointing.ipynb)
- The Open Guide to Amazon Web Services (https://github.com/open-guides/og-aws)
- [Video] Amazon SageMaker Technical Deep Dive Series (https://www.youtube.com/watch?v=uQc8Itd4UTs&list=PLhr1KZpdzukcOr_6j_zmSrvYnLUtgqsZz&index=2)
- [Video] Improve Data Science Team Productivity Using Amazon SageMaker Studio - AWS Online Tech Talks (https://www.youtube.com/watch?v=-WzkbdioMJE)
- Optimizing costs for machine learning with Amazon SageMaker (https://aws.amazon.com/blogs/machine-learning/optimizing-costs-for-machine-learning-with-amazon-sagemaker/)
- Choosing the right GPU for deep learning on AWS (https://towardsdatascience.com/choosing-the-right-gpu-for-deep-learning-on-aws-d69c157d8c86)
- Select right ML instances for training and inference jobs (https://pages.awscloud.com/rs/112-TZM-766/images/AL-ML%20for%20Startups%20-%20Select%20the%20Right%20ML%20Instance.pdf)
- Deploy fast and scalable AI with NVIDIA Triton Inference Server in Amazon SageMaker (https://aws.amazon.com/blogs/machine-learning/deploy-fast-and-scalable-ai-with-nvidia-triton-inference-server-in-amazon-sagemaker/)
- Accelerate BERT inference with Hugging Face Transformers and AWS Inferentia (https://huggingface.co/blog/bert-inferentia-sagemaker)
- Static Quantization with Hugging Face
optimum
for ~3x latency improvements (https://www.philschmid.de/static-quantization-optimum)(https://github.com/philschmid/optimum-static-quantization) - HuggingFace SageMaker Forum (https://discuss.huggingface.co/c/sagemaker/17)
- Using NGC with AWS Setup Guide (https://docs.nvidia.com/ngc/ngc-aws-setup-guide/)
- kubernetes-sagemaker-demos (https://github.com/shashankprasanna/kubernetes-sagemaker-demos)
- A quick guide to distributed training with TensorFlow and Horovod on Amazon SageMaker (https://towardsdatascience.com/a-quick-guide-to-distributed-training-with-tensorflow-and-horovod-on-amazon-sagemaker-dae18371ef6e)
- AWS EC2 User Guide - Connect to your Linux instance (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstances.html)
-
How to set up a GPU instance for machine learning on AWS (https://kstathou.medium.com/how-to-set-up-a-gpu-instance-for-machine-learning-on-aws-b4fb8ba51a7c)
-
Running Jupyter notebooks with AWS (http://bebi103.caltech.edu.s3-website-us-east-1.amazonaws.com/2020b/content/lessons/lesson_04/aws_usage.html)
-
Access remote code in a breeze with JupyterLab via SSH (https://towardsdatascience.com/access-remote-code-in-a-breeze-with-jupyterlab-via-ssh-8c6a9ffaaa8c)
-
Setting up Jupyter on the Cloud (https://kiwidamien.github.io/setting-up-jupyter-on-the-cloud.html)
-
How to Connect AWS EC2 Instance using Session Manager (https://www.kodyaz.com/aws/connect-aws-ec2-instance-using-session-manager.aspx)
-
NVTabular Cloud Integration (https://nvidia-merlin.github.io/NVTabular/main/resources/cloud_integration.html)
- Create real-time clickstream sessions and run analytics with Amazon Kinesis Data Analytics, AWS Glue, and Amazon Athena (https://aws.amazon.com/blogs/big-data/create-real-time-clickstream-sessions-and-run-analytics-with-amazon-kinesis-data-analytics-aws-glue-and-amazon-athena/)
- Identifying and working with sensitive healthcare data with Amazon Comprehend Medical (https://lifesciences-resources.awscloud.com/healthcare-life-sciences-aws-for-industries/identifying-and-working-with-sensitive-healthcare-data-with-amazon-comprehend-medical)
- Amazon Comprehend Medical – Natural Language Processing for Healthcare Customers (https://aws.amazon.com/blogs/aws/amazon-comprehend-medical-natural-language-processing-for-healthcare-customers/)
- Extract and visualize clinical entities using Amazon Comprehend Medical (https://aws.amazon.com/blogs/machine-learning/extract-and-visualize-clinical-entities-using-amazon-comprehend-medical/)
- AWS Simple Workflow vs AWS Step Functions vs Apache Airflow (https://digitalcloud.training/aws-simple-workflow-vs-aws-step-functions-vs-apache-airflow/)
- Is it the end for Apache Airflow? (https://uncledata.medium.com/is-it-the-end-for-apache-airflow-81ef027becf4)
- Scalable data preparation & ML using Apache Spark on AWS (https://github.com/debnsuma/sagemaker-studio-emr-spark)
- Interactively fine-tune Falcon-40B and other LLMs on Amazon SageMaker Studio notebooks using QLoRA (https://aws.amazon.com/blogs/machine-learning/interactively-fine-tune-falcon-40b-and-other-llms-on-amazon-sagemaker-studio-notebooks-using-qlora/)
- Boost your Resume with these Five AWS Projects: Easy, Intermediate, and Expert Levels with Repository Links (https://towardsaws.com/boost-your-resume-with-this-five-aws-projects-easy-intermediate-and-expert-levels-with-6224eef9e2ae)
- Building End-to-End Machine Learning Pipelines with Amazon SageMaker: A Step-by-Step Guide (https://medium.com/anolytics/building-end-to-end-machine-learning-pipelines-with-amazon-sagemaker-a-step-by-step-guide-8531f73b38cd)
- How to optimize AWS Lambda & Kinesis to process 5 million records per minute (https://towardsaws.com/how-to-optimize-aws-lambda-kinesis-to-process-5-million-messages-c3ed5a143c2d)
- Deploying a Trained CTGAN Model on an EC2 Instance: A Step-by-Step Guide (https://tutorialsdojo.com/deploying-a-trained-ctgan-model-on-an-ec2-instance-a-step-by-step-guide/)
-
Serverless Model Deployment in AWS: Streamlining with Lambda, Docker, and S3 (https://tutorialsdojo.com/serverless-model-deployment-in-aws-streamlining-with-lambda-docker-and-s3/)
- StatQuest (https://statquest.org/video-index/)
- Making Friends with Machine Learning (https://decision.substack.com/p/making-friends-with-machine-learning)
- Artificial Intelligence - All in One (https://www.youtube.com/c/ArtificialIntelligenceAllinOne)
- DeepLearningAI (https://www.youtube.com/c/Deeplearningai/playlists)
- Deep Learning for Computer Vision (Andrej Karpathy, OpenAI) (https://www.youtube.com/watch?v=u6aEYuemt0M)
- Deep Visualization Toolbox (https://www.youtube.com/watch?v=AgkfIQ4IGaM)
- Nuts and Bolts of Applying Deep Learning (Andrew Ng) (https://www.youtube.com/watch?v=F1ka6a13S9I)
- NIPS 2016 tutorial: "Nuts and bolts of building AI applications using Deep Learning" by Andrew Ng (https://www.youtube.com/watch?v=wjqaz6m42wU)
- 3Blue1Brown (https://www.3blue1brown.com/)
- Intellipaat - Data Science Online Course (https://www.youtube.com/watch?v=82pV44hr7kQ)
- Jay Alammar (https://www.youtube.com/channel/UCmOwsoHty5PrmE-3QhUBfPQ/videos)
- Abhishek Thakur (https://www.youtube.com/c/AbhishekThakurAbhi/playlists)
- PyTorch Performance Tuning Guide - Szymon Migacz, NVIDIA (https://www.youtube.com/watch?v=9mS1fIYj1So)
- NVIDIA Grandmaster Series (https://www.youtube.com/watch?v=bHuww-l_Sq0&list=PL5B692fm6--uXbxtmPJz5nu3Xmc1JUm3F)
- Data Professor (https://www.youtube.com/c/DataProfessor/playlists)
- Rasa Algorithm Whiteboard - Transformers & Attention 1: Self Attention (https://www.youtube.com/watch?v=yGTUuEx3GkA)
- Smart Home (https://www.youtube.com/c/AlexTeo/featured)
- Elliot Waite - Machine Learning, Coding, Math Animations (https://www.youtube.com/c/elliotwaite/videos)
- James Briggs - NLP semantic search, vector similarity search (https://www.youtube.com/c/JamesBriggs/playlists)
- Dataquest (https://www.youtube.com/channel/UC_lePY0Lm0E2-_IkYUWpI5A)
- ByteByteGo (https://www.youtube.com/c/ByteByteGo/playlists)
- Nicholas Renotte - Learn Machine Learning (https://www.youtube.com/@NicholasRenotte/playlists)
- Andrej Karpathy - Neural Networks: Zero to Hero (https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ) (https://github.com/0ssamaak0/Karpathy-Neural-Networks-Zero-to-Hero)
- NLP Summit (https://www.nlpsummit.org/)
- The Hundred-Page Machine Learning Book (http://themlbook.com/wiki/doku.php)
- Machine Learning Engineering (http://www.mlebook.com/wiki/doku.php)
- Approaching (Almost) Any Machine Learning Problem (https://github.com/abhishekkrthakur/approachingalmost/blob/master/AAAMLP.pdf)
- The fastai book (https://github.com/fastai/fastbook)
- https://books.google.com.sg/books?id=yATuDwAAQBAJ&pg=PA470&lpg=PA470&dq=AdaptiveConcatPool2d+vs+AdaptiveAvgPool2d&source=bl&ots=NKltks4CYL&sig=ACfU3U2xJo3iFtgSSLpQoUGEFYzrouhYzQ&hl=en&sa=X&ved=2ahUKEwiy29KGzNLwAhUUVH0KHTqSC3YQ6AEwCXoECAYQAw#v=onepage&q&f=false
- Introduction to Probability for Data Science (https://probability4datascience.com/index.html)
- Probabilistic Machine Learning: An Introduction (https://probml.github.io/pml-book/book1.html)
- Dive into Deep Learning (https://d2l.ai/)
- Personalized Machine Learning by Julian McAuley (https://cseweb.ucsd.edu/~jmcauley/pml/pml_book.pdf)
- Machine Learning for Credit Card Fraud detection - Practical handbook (https://fraud-detection-handbook.github.io/fraud-detection-handbook/Foreword.html)
- Deep Learning (https://www.deeplearningbook.org/)
- Efficient Python Tricks and Tools for Data Scientists (https://khuyentran1401.github.io/Efficient_Python_tricks_and_tools_for_data_scientists/README.html)
- Deep Learning Interviews: Hundreds of fully solved job interview questions from a wide range of key topics in AI (https://github.com/BoltzmannEntropy/interviews.ai)
- Data Distribution Shifts and Monitoring (https://huyenchip.com/2022/02/07/data-distribution-shifts-and-monitoring.html)
- Competitive Programmer’s Handbook (https://cses.fi/book/book.pdf)
- Free and/or open source books on machine learning, statistics, data mining, etc (https://github.com/josephmisiti/awesome-machine-learning/blob/master/books.md)
- Lucene in Action - Second Edition (https://livebook.manning.com/book/lucene-in-action-second-edition/appendix-b/)
- Build a Large Language Model (From Scratch) by Sebastian Raschka (https://github.com/rasbt/LLMs-from-scratch)
- Machine Learning Q and AI book by Sebastian Raschka (https://github.com/rasbt/MachineLearning-QandAI-book)
- 2015 Cyclical Learning Rates for Training Neural Networks (https://arxiv.org/abs/1506.01186)
- 2017 Decoupled Weight Decay Regularization (https://arxiv.org/abs/1711.05101)
- 2018 Mixed Precision Training (https://arxiv.org/pdf/1710.03740.pdf)
- 2020 ReadNet: A Hierarchical Transformer Framework for Web Article Readability Analysis (https://link.springer.com/chapter/10.1007/978-3-030-45439-5_3)
- 2021 A Survey of Transformers (https://arxiv.org/pdf/2106.04554.pdf)
- 2022 Formal Algorithms for Transformers (https://arxiv.org/pdf/2207.09238.pdf)
- 2023 Transformer models: an introduction and catalog (https://arxiv.org/pdf/2302.07730.pdf) (http://bit.ly/3YFqRn9)
- Applied Deep Learning - Part 4: Convolutional Neural Networks (https://towardsdatascience.com/applied-deep-learning-part-4-convolutional-neural-networks-584bc134c1e2)
- CNNs from different viewpoints (https://medium.com/impactai/cnns-from-different-viewpoints-fab7f52d159c)
- Image Kernels - Explained Visually (https://setosa.io/ev/image-kernels/)
- Increase the Accuracy of Your CNN by Following These 5 Tips I Learned From the Kaggle Community (https://towardsdatascience.com/increase-the-accuracy-of-your-cnn-by-following-these-5-tips-i-learned-from-the-kaggle-community-27227ad39554)
- A Beginner's Guide To Understanding Convolutional Neural Networks (https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/)
- A Beginner's Guide To Understanding Convolutional Neural Networks Part 2 (https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/)
- The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3) (https://adeshpande3.github.io/The-9-Deep-Learning-Papers-You-Need-To-Know-About.html)
- CNN Explainer (https://poloclub.github.io/cnn-explainer/)
- Understanding LSTM Networks (http://colah.github.io/posts/2015-08-Understanding-LSTMs/)
- A Visual Guide to Recurrent Layers in Keras (https://amitness.com/2020/04/recurrent-layers-keras/)
- A Visual Guide to FastText Word Embeddings (https://amitness.com/2020/06/fasttext-embeddings/)
- The Illustrated Word2vec (https://jalammar.github.io/illustrated-word2vec/)
- An Intuitive Understanding of Word Embeddings: From Count Vectors to Word2Vec (https://www.analyticsvidhya.com/blog/2017/06/word-embeddings-count-word2veec/)
- Intuitive Understanding of Seq2seq model & Attention Mechanism in Deep Learning (https://medium.com/analytics-vidhya/intuitive-understanding-of-seq2seq-model-attention-mechanism-in-deep-learning-1c1c24aace1e)
- How to Develop Word Embeddings in Python with Gensim (https://machinelearningmastery.com/develop-word-embeddings-python-gensim/)
- Interactive Analysis of Sentence Embeddings (https://amitness.com/interactive-sentence-embeddings/)
- Cosine Similarity for Vector Space Models (Part III) (https://blog.christianperone.com/2013/09/machine-learning-cosine-similarity-for-vector-space-models-part-iii/)
- Recommender Systems competitions solutions (https://github.com/NVIDIA-Merlin/competitions)
- NVIDIA at RecSys 2022 (https://www.nvidia.com/en-us/events/recsys/)
-
Tutorials (https://recsys.acm.org/recsys22/tutorials/)
-
Tutorial Hands on Explainable Recommender Systems with Knowledge Graphs (https://explainablerecsys.github.io/recsys2022/)
-
Neural Re-ranking Tutorial (RecSys 22) (https://librerank-community.github.io/)
-
PIRS - Psychology-informed Recommender Systems (https://socialcomplab.github.io/pirs-psychology-informed-recsys/)
- NVIDIA Merlin (https://medium.com/nvidia-merlin)
-
Merlin Jupyter Notebook Examples (https://catalog.ngc.nvidia.com/orgs/nvidia/resources/merlin_notebooks)
-
Merlin Models Example Notebooks (https://github.com/NVIDIA-Merlin/models/tree/main/examples)
-
NVIDIA Deep Learning Examples for Tensor Cores (https://github.com/NVIDIA/DeepLearningExamples)
-
Merlin Systems Example Notebook (https://github.com/NVIDIA-Merlin/systems/tree/main/examples)
-
Building a Four-Stage Recommender Pipeline (https://github.com/NVIDIA-Merlin/systems#building-a-four-stage-recommender-pipeline)
-
Exploring Production Ready Recommender Systems with Merlin (https://medium.com/nvidia-merlin/exploring-production-ready-recommender-systems-with-merlin-66bba65d18f2)
-
Recommender Models: Reducing Friction with Merlin Models (https://medium.com/nvidia-merlin/recommender-models-reducing-friction-with-merlin-models-4ea799fc3d89)
-
Scale faster with less code using Two Tower with Merlin (https://medium.com/nvidia-merlin/scale-faster-with-less-code-using-two-tower-with-merlin-c16f32aafa9f)
-
Transformers4Rec: Building Session-Based Recommendations with an NVIDIA Merlin Library (https://developer.nvidia.com/blog/transformers4rec-building-session-based-recommendations-with-an-nvidia-merlin-library/)
-
HugeCTR, a GPU-accelerated recommender framework (https://github.com/NVIDIA-Merlin/HugeCTR)
-
Recommender Systems at NVIDIA on Demand (https://www.nvidia.com/en-us/on-demand/search/?facet.mimetype[]=event%20session&layout=list&ncid=so-medi-419714&page=1&q=recommender%20systems&sort=date)
-
Recommender Systems Best Practices (https://resources.nvidia.com/en-us-recsys-white-paper/merlin-technical-ove)
-
Training a Recommender System on DGX A100 with 100B+ Parameters in TensorFlow 2 (https://developer.nvidia.com/blog/training-a-recommender-system-on-dgx-a100-with-100b-parameters-in-tensorflow-2/)
- Recommender Systems, Not Just Recommender Models (2022-04-15)(https://medium.com/nvidia-merlin/recommender-systems-not-just-recommender-models-485c161c755e)
- How NVIDIA Supports Recommender Systems feat. Even Oldridge | Stanford MLSys Seminar Episode 27 (https://www.youtube.com/watch?v=wPso35VkuCs)
- [Video] Building and Deploying a Multi-Stage Recommender System with NVIDIA Merlin (https://www.youtube.com/watch?v=BQC-SGdIdD8)
- Building and Deploying a Multi-Stage Recommender System with Merlin (https://resources.nvidia.com/en-us-merlin/bad-a-multi-stage-recommender?)
- How to Build a Winning Deep Learning Powered Recommender System-Part 3 (https://developer.nvidia.com/blog/how-to-build-a-winning-deep-learning-powered-recommender-system-part-3/) (https://github.com/NVIDIA-Merlin/competitions/tree/main/WSDM_WebTour2021_Challenge)
- [Video] Mastering Multilingual Recommender Systems | Grandmaster Series E9 | Winning Amazon’s 2023 KDD Cup (https://www.youtube.com/watch?v=IECznzY_Ko4)
- Google Courses - Recommendation Systems (https://developers.google.com/machine-learning/recommendation)
- Dive Into Deep Learning - Chapter on Recommender Systems (http://d2l.ai/chapter_recommender-systems/index.html)
- Reference End to End Architectures:
-
Alibaba Cloud - Recommender System: Ranking Algorithms and Training Architectures (https://www.alibabacloud.com/blog/recommender-system-ranking-algorithms-and-training-architectures_596643)
- Basics of Recommender Systems (https://towardsdatascience.com/basics-of-recommender-systems-6f0fba58d8a)
- Understanding Matrix Factorization for recommender systems (https://towardsdatascience.com/understanding-matrix-factorization-for-recommender-systems-4d3c5e67f2c9)
- Building a Music Recommendation Engine with Probabilistic Matrix Factorization in PyTorch (https://towardsdatascience.com/building-a-music-recommendation-engine-with-probabilistic-matrix-factorization-in-pytorch-7d2934067d4a)
- Customer Segmentation in Online Retail (https://towardsdatascience.com/customer-segmentation-in-online-retail-1fc707a6f9e6)
- Sparse Matrices (https://www.youtube.com/watch?v=Lhef_jxzqCg)
- scipy.sparse.csr_matrix example (https://stackoverflow.com/questions/53254104/cant-understand-scipy-sparse-csr-matrix-example)
- Understanding min_df and max_df in scikit CountVectorizer (https://stackoverflow.com/questions/27697766/understanding-min-df-and-max-df-in-scikit-countvectorizer)
- Build a Recommendation Engine With Collaborative Filtering (https://realpython.com/build-recommendation-engine-collaborative-filtering/)
- Collaborative Filtering Recommendation with Co-Occurrence Algorithm (https://songxia-sophia.medium.com/collaborative-filtering-recommendation-with-co-occurrence-algorithm-dea583e12e2a)
- Recommendation System Series (https://towardsdatascience.com/recommendation-system-series-part-1-an-executive-guide-to-building-recommendation-system-608f83e2630a)
- Wayfair Tech Blog (https://www.aboutwayfair.com/careers/tech-blog?q=&s=0&f0=0000017b-63b5-d47e-adff-f7bda4220000)
- TensorFlow Recommenders Tutorial (https://www.tensorflow.org/recommenders/examples/basic_retrieval)
- Eugene Yan
-
System Design for Recommendations and Search (2021-06-27)(https://eugeneyan.com/writing/system-design-for-discovery/)
-
Patterns for Personalization in Recommendations and Search (2021-06-13)(https://eugeneyan.com/writing/patterns-for-personalization/)
-
Real-time Machine Learning For Recommendations (2021-01-10)(https://eugeneyan.com/writing/real-time-recommendations/)
-
Beating the Baseline Recommender with Graph & NLP in Pytorch (2020-01-13)(https://eugeneyan.com/writing/recommender-systems-graph-and-nlp-pytorch/#natural-language-processing-nlp-and-graphs)
- Search, Rank, and Recommendations (https://medium.com/mlearning-ai/search-rank-and-recommendations-35cc717772cb)(https://www.kaggle.com/code/sbrvrm/search-ranking/notebook)
- Vector representation of products Prod2Vec: How to get rid of a lot of embeddings (https://towardsdatascience.com/vector-representation-of-products-prod2vec-how-to-get-rid-of-a-lot-of-embeddings-26265361457c)
- Deep Recommender Systems at Facebook feat. Carole-Jean Wu | Stanford MLSys Seminar Episode 24 (https://www.youtube.com/watch?v=5xcd0V9m6Xs)
- Twitter's Recommendation Algorithm (https://blog.twitter.com/engineering/en_us/topics/open-source/2023/twitter-recommendation-algorithm) (https://github.com/twitter/the-algorithm) (https://github.com/twitter/the-algorithm-ml)
- Personalized recommendations articles by Gaurav Chakravorty (https://www.linkedin.com/today/author/gauravchak?trk=article-ssr-frontend-pulse_more-articles)
- Accelerating AI: Implementing Multi-GPU Distributed Training for Personalized Recommendations (https://multithreaded.stitchfix.com/blog/2023/06/08/distributed-model-training/)
- How Instacart Uses Machine Learning-Driven Autocomplete to Help People Fill Their Carts (https://tech.instacart.com/how-instacart-uses-machine-learning-driven-autocomplete-to-help-people-fill-their-carts-9bc56d22bafb)
- Getting started with Vector DBs in Python (https://code.dblock.org/2023/06/16/getting-started-with-vector-dbs-in-python.html) (https://github.com/dblock/vectordb-hello-world/)
- Pinecone - A managed, cloud-native vector database with a simple API (https://www.pinecone.io/learn/) (https://docs.pinecone.io/docs/examples)
- Weaviate (https://weaviate.io/blog.html)
- Billion-scale Approximate Nearest Neighbor Search (https://matsui528.github.io/cvpr2020_tutorial_retrieval/)
- PQk-means is a Python library for efficient clustering of large-scale data (https://github.com/DwangoMediaVillage/pqkmeans)
- Nanopq (https://nanopq.readthedocs.io/en/latest/source/tutorial.html)
-
-
-
PQTable (http://yusukematsui.me/project/pqtable/pqtable.html)
- Faiss - Facebook AI Similarity Search (https://github.com/facebookresearch/faiss/wiki)
-
Faiss tips (https://github.com/matsui528/faiss_tips)
-
-
Faiss on-disk example (https://davidefiocco.github.io/nearest-neighbor-search-with-faiss/)
- NGT - Neighborhood Graph and Tree for Indexing High-dimensional Data (https://github.com/yahoojapan/NGT)
- Milvus Bootcamp Solutions (https://github.com/milvus-io/bootcamp)
- Add Similarity Search to DynamoDB with Faiss (https://medium.com/swlh/add-similarity-search-to-dynamodb-with-faiss-c68eb6a48b08)(https://github.com/ioannist/dynamodb-faiss-builder)
- BERT models with Solr and Elasticsearch (https://github.com/DmitryKey/bert-solr-search)
- Generative Feedback Loops with LLMs for Vector Databases (https://weaviate.io/blog/generative-feedback-loops-with-llms)
- From zero to semantic search embedding model (https://blog.metarank.ai/from-zero-to-semantic-search-embedding-model-592e16d94b61)
- Example of using factory pattern for your vectorstore implementation (https://github.com/trancethehuman/factory-pattern-vectorstore-interface) (https://www.youtube.com/watch?v=v1LyUJ5NFFU)
- Accelerating Vector Search: Fine-Tuning GPU Index Algorithms (https://developer.nvidia.com/blog/accelerating-vector-search-fine-tuning-gpu-index-algorithms/) (https://github.com/rapidsai/raft/blob/HEAD/notebooks/VectorSearch_QuestionRetrieval.ipynb)
- RAFT: Reusable Accelerated Functions and Tools for Vector Search and More (https://github.com/rapidsai/raft)
- RAFT IVF-PQ tutorial (https://github.com/rapidsai/raft/blob/28b789404bedfa8dd82675fc4221f6db927c0422/notebooks/tutorial_ivf_pq.ipynb)
- CAGRA: Highly Parallel Graph Construction and Approximate Nearest Neighbor Search for GPUs (Cuda Anns GRAph-based) (https://arxiv.org/pdf/2308.15136v1.pdf) (https://docs.rapids.ai/api/raft/nightly/pylibraft_api/neighbors/#cagra)
- Geospatial Vector Search: Building an AI-Powered Geo-Aware News Search (https://levelup.gitconnected.com/geospatial-vector-search-building-an-ai-powered-geo-aware-news-search-6cbda8919465)
- MyScale - A Deep Dive into SQL Vector Databases (https://myscale.com/blog/what-is-sql-vector-databases/)
- Vector DB Comparison (https://superlinked.com/vector-db-comparison/)
- How to build a real-time News Search Engine using Vector DBs (https://medium.com/decodingml/how-to-build-a-real-time-news-search-engine-using-serverless-upstash-kafka-and-vector-db-6ba393e55024) (https://decodingml.substack.com/p/how-to-build-a-real-time-news-search) (https://github.com/decodingml/articles-code/tree/main/articles/ml_system_design/real_time_news_search_with_upstash_kafka_and_vector_db)
- A brief history of code search at GitHub (https://github.blog/2021-12-15-a-brief-history-of-code-search-at-github/)
- The technology behind GitHub’s new code search (https://github.blog/2023-02-06-the-technology-behind-githubs-new-code-search/)
- Regular Expression Matching with a Trigram Index or How Google Code Search Worked (https://swtch.com/~rsc/regexp/regexp4.html)
- PROBABILISTIC DATA STRUCTURES FOR WEB ANALYTICS AND DATA MINING (https://highlyscalable.wordpress.com/2012/05/01/probabilistic-structures-web-analytics-data-mining/)
- Big Data Counting: How To Count A Billion Distinct Objects Using Only 1.5KB Of Memory (http://highscalability.com/blog/2012/4/5/big-data-counting-how-to-count-a-billion-distinct-objects-us.html)
- Roaring Bitmap (https://pypi.org/project/roaringbitmap/0.1/) (https://github.com/RoaringBitmap/RoaringBitmap)
- A primer on Roaring bitmaps: what they are and how they work (https://vikramoberoi.com/a-primer-on-roaring-bitmaps-what-they-are-and-how-they-work/) (https://news.ycombinator.com/item?id=32692012)
- Elias-Fano: quasi-succinct compression of sorted integers in C# (2016) (https://wolfgarbe.medium.com/elias-fano-quasi-succinct-compression-of-sorted-integers-in-c-89f92a8c9986)
- Using Bitmaps to Perform Range Queries (https://www.featurebase.com/blog/range-encoded-bitmaps)
- The anatomy of a Druid segment file (https://medium.com/engineers-optimizely/the-anatomy-of-a-druid-segment-file-bed89a93af1e#.46tincja7)
- Information Retrieval resources (https://cwiki.apache.org/confluence/display/LUCENE/InformationRetrieval)
- Information Retrieval - Lectures by Paolo Ferragina (http://didawiki.di.unipi.it/doku.php/magistraleinformatica/ir/ir22/start)
- https://www.slideshare.net/VadimKirilchuk/numeric-rangequeries
- NYU - Introduction to Data Compression - Web Search Engines (http://engineering.nyu.edu/~suel/cs6913/lec4-compress.pdf)
- Decoding Billions of Integers Per Second Through Vectorization (https://people.csail.mit.edu/jshun/6886-s19/lectures/lecture19-1.pdf)
- Smart way of storing data (https://towardsdatascience.com/smart-way-of-storing-data-d22dd5077340)
- Google - Challenges in Building Large-Scale Information Retrieval Systems (http://static.googleusercontent.com/media/research.google.com/en//people/jeff/WSDM09-keynote.pdf)
- A guide to Google Search ranking systems (https://developers.google.com/search/docs/appearance/ranking-systems-guide)
- How to ARCHITECT a search engine like Google Search (https://newsletter.theaiedge.io/p/how-to-architect-a-search-engine)
- Using Cross-Encoders as reranker in multistage vector search (https://weaviate.io/blog/cross-encoders-as-reranker)
- Bi-Encoder vs. Cross-Encoder (https://www.sbert.net/examples/applications/cross-encoder/#bi-encoder-vs-cross-encoder)
- Improving information retrieval in the Elastic Stack: Introducing Elastic Learned Sparse Encoder, our new retrieval model (https://www.elastic.co/search-labs/may-2023-launch-information-retrieval-elasticsearch-ai-model)
- Hybrid Search: SPLADE (Sparse Encoder) (https://medium.com/@sowmiyajaganathan/hybrid-search-splade-sparse-encoder-neural-retrieval-models-d092e5f46913)
- State-of-the-art MS MARCO Models (https://twitter.com/Nils_Reimers/status/1435544757388857345)
-
https://www.sbert.net/examples/training/ms_marco/README.html#marginmse
- Grokking Solr Trie Fields (http://mentaldetritus.blogspot.com/2013/01/grokking-solr-trie-fields.html)
- Mastering ElasticSearch (https://hoclaptrinhdanang.com/downloads/pdf/elasticsearch/Mastering%20ElasticSearch-PDF.pdf)
- Elasticsearch Kernel Analysis - Data Model (https://zhuanlan.zhihu.com/p/34680841)
- Elasticsearch-Principle of Big Data (https://zhuanlan.zhihu.com/p/83961549)
- Scaling Lucene and Solr (https://lucidworks.com/post/scaling-lucene-and-solr/)
- Lucene Papers (https://cwiki.apache.org/confluence/display/LUCENE/LucenePapers)
- Lucene, talk by Doug Cutting (https://lucene.sourceforge.net/talks/pisa/)
- Lucene: The Good Parts (https://blog.parse.ly/lucene/)
- Zhaofeng Zhou (Muluo) - Analysis of Lucene - Basic Concepts (https://alibaba-cloud.medium.com/analysis-of-lucene-basic-concepts-5ff5d8b90a53)
- Zhaofeng Zhou (Muluo) - Lucene IndexWriter: An In-Depth Introduction (https://www.alibabacloud.com/blog/lucene-indexwriter-an-in-depth-introduction_594673)
- What is term vector in Lucene (http://makble.com/what-is-term-vector-in-lucene)
- What is Trie Data Structure in Lucene numeric range query (http://makble.com/what-is-trie-data-structure-in-lucene-numeric-range-query) (https://issues.apache.org/jira/browse/LUCENE-1673)
- Lucene Performance (http://philosophyforprogrammers.blogspot.com/2010/09/lucene-performance.html)
- Frame of Reference and Roaring Bitmaps (https://www.elastic.co/blog/frame-of-reference-and-roaring-bitmaps) (https://juejin.cn/post/7085352076595134494)
- Class RoaringDocIdSet (https://lucene.apache.org/core/6_0_0/core/org/apache/lucene/util/RoaringDocIdSet.html) (https://issues.apache.org/jira/browse/LUCENE-5983)
- Class Lucene50PostingsFormat (https://lucene.apache.org/core/7_2_1/core/org/apache/lucene/codecs/lucene50/Lucene50PostingsFormat.html)
-
Changing Bits - Lucene's PulsingCodec on "Primary Key" Fields (https://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html)
-
Changing Bits - Lucene performance with the PForDelta codec (https://blog.mikemccandless.com/2010/08/lucene-performance-with-pfordelta-codec.html)
-
Changing Bits - Lucene's new BlockPostingsFormat (https://blog.mikemccandless.com/2012/08/lucenes-new-blockpostingsformat-thanks.html)
-
Changing Bits - Using Finite State Transducers in Lucene (https://blog.mikemccandless.com/2010/12/using-finite-state-transducers-in.html)
- Apache Lucene - Index File Formats (https://lucene.apache.org/core/3_0_3/fileformats.html) (https://stackoverflow.com/questions/71816188/what-is-elasticsearch-index-lucene-index-and-inverted-index)
- Lucene-file format (https://zhuanlan.zhihu.com/p/354105864)
- Apache Lucene - Scoring (https://lucene.apache.org/core/3_0_3/scoring.html)
- Lucene JIRA:
-
More fine-grained control over the packed integer implementation that is chosen (https://issues.apache.org/jira/browse/LUCENE-4062)
-
Reduce reads for sparse DocValues (https://issues.apache.org/jira/browse/LUCENE-8374)
-
Simple9 (de)compression (https://issues.apache.org/jira/browse/LUCENE-2189)
- Lucene index modeling - Why are skiplists used instead of btree? (https://stackoverflow.com/questions/66804510/lucene-index-modeling-why-are-skiplists-used-instead-of-btree)
- How does lucene index documents? (https://stackoverflow.com/questions/2602253/how-does-lucene-index-documents#answer-43203339)
- Skip List vs. Binary Search Tree (https://stackoverflow.com/questions/256511/skip-list-vs-binary-search-tree/28270537#28270537)
- Uncle Cang - Lucene Source Code Series (https://juejin.cn/user/2559318800998141)
- Chris - Articles on Lucene Index (https://www.amazingkoala.com.cn/Lucene/Index/)
- Chris - Articles on Lucene Index File (https://www.amazingkoala.com.cn/Lucene/suoyinwenjian/)
- Chris - Articles on Lucene Compressed Storage (https://www.amazingkoala.com.cn/Lucene/yasuocunchu/)
- Chris - Articles on Lucene Tools (https://www.amazingkoala.com.cn/Lucene/gongjulei/)
- Chris - Articles on Lucene Search (https://www.amazingkoala.com.cn/Lucene/Search/)
- LuXugang/Lucene-7.x-9.x/tree/master/blog (https://github.com/LuXugang/Lucene-7.x-9.x/tree/master/blog)
- Lucene Learning Summary (https://blog.csdn.net/jinhong_lu)
- Lucene Underlying Principles and Optimization Experience Sharing (https://blog.csdn.net/njpjsoftdev)
- Lucene PostingsFormat At-a-Glance (https://github.com/mocobeta/lucene-postings-format) (https://github.com/mocobeta/lucene-postings-format/blob/main/indexing_chain.md)
- A Simple Tutorial of Lucene's Indexing and Search Systems (https://github.com/jiepujiang/LuceneTutorial)
- [Video] What is in a Lucene index? Adrien Grand, Software Engineer, Elasticsearch (https://www.youtube.com/watch?v=T5RmMNDR5XI)
- [Video] from Plain Schwarz (https://www.youtube.com/@PlainSchwarzUG)
-
Berlin Buzzwords 2015: Adrien Grand – Algorithms & data-structures that power Lucene & ElasticSearch (https://www.youtube.com/watch?v=eQ-rXP-D80U)
-
Berlin Buzzwords 2015: Ryan Ernst - Compression in Lucene (https://www.youtube.com/watch?v=kCQbFxqusN4&list=PLq-odUc2x7i-_qWWixXHZ6w-MxyLxEC7s&index=21)
-
Berlin Buzzwords 2015: Ivan Mamontov - Fast Decompression Lucene Codec (https://www.youtube.com/watch?v=2HQdbpgHfnQ&list=PLq-odUc2x7i-_qWWixXHZ6w-MxyLxEC7s&index=17)
-
Berlin Buzzwords 2017: Alan Woodward - How does a Lucene Query actually work? (https://www.youtube.com/watch?v=Z-yG-KvIuD8&list=PLq-odUc2x7i-9Nijx-WfoRMoAfHC9XzTt&index=3)
-
Berlin Buzzwords 2017: Adrien Grand - Running slow queries with Lucene (https://www.youtube.com/watch?v=p51vIDWHWqk&list=PLq-odUc2x7i-9Nijx-WfoRMoAfHC9XzTt&index=16)
-
(2020) Bruno Roustant – A Journey to Write a New Lucene PostingsFormat (https://www.youtube.com/watch?v=av0yQY3pklA&list=PLq-odUc2x7i_YTCOTQ6p3m-kqpvEXGvbT&index=44)
-
(2020) Uwe Schindler - Ask Me Anything: Lucene 9 (https://www.youtube.com/watch?v=RvoH_pVvXz0&list=PLq-odUc2x7i_YTCOTQ6p3m-kqpvEXGvbT&index=10)
- A Recipe for Training Neural Networks (http://karpathy.github.io/2019/04/25/recipe/)
- Microsoft - Recommenders (https://github.com/microsoft/recommenders)
- DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inference (https://github.com/harvard-acc/DeepRecSys)
- The best machine learning and deep learning libraries (https://morioh.com/p/73998ba2a04e)
- Nuts and Bolts of Optimization (https://www.linkedin.com/pulse/nuts-bolts-optimization-chandra-mohan-lingam)
- Neural Style Transfer with tf.keras (https://aihub.cloud.google.com/p/products%2F7f7495dd-6f66-4f8a-8c30-15f211ad6957)
- An overview of gradient descent optimization algorithms (https://ruder.io/optimizing-gradient-descent/)
- Why Momentum Really Works (https://distill.pub/2017/momentum/)
- How to Choose an Activation Function for Deep Learning (https://machinelearningmastery.com/choose-an-activation-function-for-deep-learning/)
- Exploring ExplainerDashBoard, the easiest way to Develop Interactive DashBoards (https://towardsdatascience.com/build-dashboards-in-less-than-10-lines-of-code-835e9abeae4b)
- Stock Prices Prediction Using Machine Learning and Deep Learning Techniques (with Python codes) (https://www.analyticsvidhya.com/blog/2018/10/predicting-stock-price-machine-learningnd-deep-learning-techniques-python)
- I trained a model. What is next? (https://ternaus.blog/tutorial/2020/08/28/Trained-model-what-is-next.html)
- How I Built and Deployed a Fun Serverless Machine Learning App (https://towardsdatascience.com/building-and-deploying-cartoonify-b4786b382d7e)
- Applied Deep Learning - Part 3: Autoencoders (https://towardsdatascience.com/applied-deep-learning-part-3-autoencoders-1c083af4d798)
- Finding Good Learning Rate and The One Cycle Policy (https://towardsdatascience.com/finding-good-learning-rate-and-the-one-cycle-policy-7159fe1db5d6)
- Understanding Fastai's fit_one_cycle method (https://iconof.com/1cycle-learning-rate-policy/)
- Approaching (Almost) Any Machine Learning Problem (https://www.linkedin.com/pulse/approaching-almost-any-machine-learning-problem-abhishek-thakur)
- KAGGLE ENSEMBLING GUIDE (https://mlwave.com/kaggle-ensembling-guide/)
- Reading Larger than Memory CSVs with RAPIDS and Dask (https://medium.com/rapids-ai/reading-larger-than-memory-csvs-with-rapids-and-dask-e6e27dfa6c0f)
- 28 Weekly Machine Learning Tricks And Resources That Are Pure Gems #1 (https://ibexorigin.medium.com/28-weekly-machine-learning-tricks-and-resources-that-are-pure-gems-1-8e5259a93c94)
- 26 Weekly ML Tricks And Resources That Are Pure Gems, #2 (https://ibexorigin.medium.com/26-weekly-ml-tricks-and-resources-that-are-pure-gems-2-3be56841b1d9)
- Dealing with Outliers Using Three Robust Linear Regression Models - Huber, RANSAC, Theil-Sen (https://developer.nvidia.com/blog/dealing-with-outliers-using-three-robust-linear-regression-models/)
- Fighting Overfitting With L1 or L2 Regularization: Which One Is Better? (https://neptune.ai/blog/fighting-overfitting-with-l1-or-l2-regularization)
- Understanding Gradient Clipping (and How It Can Fix Exploding Gradients Problem) (https://neptune.ai/blog/understanding-gradient-clipping-and-how-it-can-fix-exploding-gradients-problem)
- Vanishing and Exploding Gradients in Neural Network Models: Debugging, Monitoring, and Fixing (https://neptune.ai/blog/vanishing-and-exploding-gradients-debugging-monitoring-fixing)
- Hadoop vs. Spark vs. Kafka - How to Structure Modern Big Data Architecture? (https://nexocode.com/blog/posts/hadoop-spark-kafka-modern-big-data-architecture/)
- Is it Better to Save Models Using Joblib or Pickle? (https://medium.com/nlplanet/is-it-better-to-save-models-using-joblib-or-pickle-776722b5a095)
- How to Measure Drift in ML Embeddings (https://towardsdatascience.com/how-to-measure-drift-in-ml-embeddings-ee8adfe1e55e) (https://www.evidentlyai.com/blog/embedding-drift-detection)
- Large Language Models in Molecular Biology (https://towardsdatascience.com/large-language-models-in-molecular-biology-9eb6b65d8a30)
- Five mistakes to avoid when modeling with imbalanced datasets (https://medium.com/data-science-at-microsoft/five-mistakes-to-avoid-when-modeling-with-imbalanced-datasets-d58a8c09929c) (https://github.com/PilCAki/imbalanced-dataset-common-errors/blob/main/Imbalanced%20Dataset%20Examples.ipynb)
- Categorically: Don’t explode — encode! (https://github.com/PilCAki/machine-learning-tips/blob/main/Don't%20Explode%20-%20Encode!.ipynb)
- Out-of-bag validation for random forests (https://medium.com/data-science-at-microsoft/out-of-bag-validation-for-random-forests-378f2b292560) (https://github.com/PilCAki/machine-learning-tips/blob/main/Out%20Of%20Bag%20Validation%20for%20Random%20Forests.ipynb)
- Yes you should understand backprop (https://karpathy.medium.com/yes-you-should-understand-backprop-e2f06eab496b) (https://www.youtube.com/watch?v=i94OvYb6noo)
- A Comprehensive Overview of Regression Evaluation Metrics (https://developer.nvidia.com/blog/a-comprehensive-overview-of-regression-evaluation-metrics/)
- Exploring Location Data Using a Hexagon Grid (https://towardsdatascience.com/exploring-location-data-using-a-hexagon-grid-3509b68b04a2) (https://github.com/sktahtin4/Helsinki-city-bikes)
-
H3: Uber’s Hexagonal Hierarchical Spatial Index (https://www.uber.com/en-FI/blog/h3/) (https://h3geo.org/docs/)
-
H3 hexagon data viewer (https://wolf-h3-viewer.glitch.me/)
- Predicting Credit Defaults Using Time-Series Models with Recursive Neural Networks and XGBoost (https://developer.nvidia.com/blog/predicting-credit-defaults-using-time-series-models-with-recursive-neural-networks-and-xgboost/) (https://github.com/daxiongshu/triton_amex)
- Three Approaches to Encoding Time Information as Features for ML Models (https://developer.nvidia.com/blog/three-approaches-to-encoding-time-information-as-features-for-ml-models/)
- A Comprehensive Guide to Interaction Terms in Linear Regression (https://developer.nvidia.com/blog/a-comprehensive-guide-to-interaction-terms-in-linear-regression/)
- A Comprehensive Guide on Interaction Terms in Time Series Forecasting (https://developer.nvidia.com/blog/a-comprehensive-guide-on-interaction-terms-in-time-series-forecasting/)
- Plotly Fundamentals (https://plotly.com/python/plotly-fundamentals/)
- CSS Color (https://developer.mozilla.org/en-US/docs/Web/CSS/color_value)
- Flowing Data (https://flowingdata.com/)
- 📊 Automatic EDA Libraries 📚 Comparisson (https://www.kaggle.com/andreshg/automatic-eda-libraries-comparisson/)
- [TPS-Jun] This is Original EDA & VIZ 😉 (https://www.kaggle.com/subinium/tps-jun-this-is-original-eda-viz/)
- Netflix Data Visualization (https://www.kaggle.com/joshuaswords/netflix-data-visualization?scriptVersionId=58425238)(https://www.kaggle.com/joshuaswords)
- Custom Word Cloud (https://www.kaggle.com/tarzon/custom-word-cloud/notebook)
- RAPIDS cuXfilter - Build a Fully Interactive Dashboard in a Few Lines of Python (https://medium.com/rapids-ai/build-a-fully-interactive-dashboard-in-a-few-lines-of-python-49959fb55fff)
- Facets - visualizations for understanding and analyzing machine learning datasets (https://github.com/PAIR-code/facets)
- Alternatives to box plots: Using beeswarm and raincloud plots to summarise ecological data (https://labs.ala.org.au/posts/2023-08-28_alternatives-to-box-plots/post.html)
- How to Create a Beautiful Polar Histogram With Python and Matplotlib (https://dev.to/oscarleo/how-to-create-a-beautiful-polar-histogram-with-python-and-matplotlib-400l)
- Data Visualisation Guide (https://data.europa.eu/apps/data-visualisation-guide/)
- What is Gephi? Meet this useful network analysis tool (https://medium.com/@vespinozag/what-is-gephi-meet-this-useful-network-analysis-tool-628a1b42428c) (https://github.com/gephi/gephi)
- The Perfect Way to Smooth Your Noisy Data - using Whittaker-Eilers Smoothing (https://towardsdatascience.com/the-perfect-way-to-smooth-your-noisy-data-4f3fe6b44440)
- Scikit-learn Visualization Guide: Making Models Speak (https://www.dataleadsfuture.com/scikit-learn-visualization-guide-making-models-speak/)
- 6 python libraries to make beautiful maps (https://medium.com/@alexroz/6-python-libraries-to-make-beautiful-maps-9fb9edb28b27)
- Google Colab Tips for Power Users (https://amitness.com/2020/06/google-colaboratory-tips/)
- Configuring Google Colab Like A Pro (https://medium.com/@robertbracco1/configuring-google-colab-like-a-pro-d61c253f7573)
- Data manipulation with Python (https://www.mit.edu/~amidi/teaching/data-science-tools/study-guide/data-manipulation-with-python/)
- Data Preprocessing Concepts with Python (https://pub.towardsai.net/data-preprocessing-concepts-with-python-b93c63f14bb6)
- 7 Actionable Tips on How to Use Python to Become a Finance Guru (https://sdsclub.com/7-actionable-tips-on-how-to-use-python-to-become-a-finance-guru/)
- Cross-validation (https://www.datamuni.com/@abhishek/basics-of-cross-validation)
- 7 Cool Python Packages Kagglers Are Using Without Telling You (https://towardsdatascience.com/7-cool-python-packages-kagglers-are-using-without-telling-you-e83298781cf4)
- Fastest Way to Read Excel in Python (https://hakibenita.com/fast-excel-python)
- Efficiently iterating over rows in a Pandas DataFrame (https://towardsdatascience.com/efficiently-iterating-over-rows-in-a-pandas-dataframe-7dd5f9992c01)
- Blogs about streaming database (https://medium.com/@yingjunwu)
- Rethinking Stream Processing and Streaming Databases (https://betterprogramming.pub/rethinking-stream-processing-and-streaming-databases-21076aaec375)
- Why You Shouldn’t Invest In Vector Databases? (https://medium.com/data-engineer-things/why-you-shouldnt-invest-in-vector-databases-c0cd3f59d23c)
- Building and operating a pretty big storage system called S3 (https://www.allthingsdistributed.com/2023/07/building-and-operating-a-pretty-big-storage-system.html)
- MyScale - A Deep Dive into SQL Vector Databases (https://myscale.com/blog/what-is-sql-vector-databases/)
- Building a weather data warehouse part I: Loading a trillion rows of weather data into TimescaleDB (https://aliramadhan.me/2024/03/31/trillion-rows.html)
- Develop Your First AI Agent: Deep Q-Learning (https://towardsdatascience.com/develop-your-first-ai-agent-deep-q-learning-375876ee2472)
- A Visual Guide to Regular Expression (https://amitness.com/regex/)
- Machine Learning Glossary (https://developers.google.com/machine-learning/glossary/)
- ConvNetJS - Deep Learning in your browser (https://cs.stanford.edu/people/karpathy/convnetjs/)
- Classifier Playground (https://www.ccom.ucsd.edu/~cdeotte/programs/classify.html)
- YOLO: Real-Time Object Detection (https://pjreddie.com/darknet/yolo/)
- Coder One - A virtual playground to practice, compete, and experiment with machine learning (https://www.gocoder.one/)
- Interactive demonstrations for ML courses (http://arogozhnikov.github.io/2016/04/28/demonstrations-for-ml-courses.html)
- Tinker With a Neural Network (http://playground.tensorflow.org/)
- Understanding ROC curves (http://www.navan.name/roc/)
- Algorithm Visualizer (https://algorithm-visualizer.org/)
- Sort Visualizer (https://adwait-algorithm-visualizer.netlify.app/)
- VisuAlgo (https://visualgo.net/en)
- Doodles-as-A-Service Repo (https://github.com/girliemac/a-picture-is-worth-a-1000-words)
- AI art generator (https://app.wombo.art/)
- Seeing Theory - Visual introduction to probability and statistics (https://seeing-theory.brown.edu/index.html#firstPage)
- Answer Chat AI (https://www.answerchatai.com/)
- AI Explorables (https://pair.withgoogle.com/explorables/)
-
Do Machine Learning Models Memorize or Generalize? (https://pair.withgoogle.com/explorables/grokking/)
- Tensor Puzzles (https://github.com/srush/Tensor-Puzzles)
- GPU Puzzles (https://github.com/srush/GPU-Puzzles)
- Insanely Useful Websites (https://insanelyusefulwebsites.com)
- The Markov-chain Monte Carlo Interactive Gallery (https://chi-feng.github.io/mcmc-demo/app.html) (https://chi-feng.github.io/mcmc-demo/) (https://github.com/chi-feng/mcmc-demo)
- Interactive Gaussian process regression demo (https://chi-feng.github.io/gp-demo/) (https://github.com/chi-feng/gp-demo)
- Visualizing {dplyr}’s mutate(), summarize(), group_by(), and ungroup() with animations (https://www.andrewheiss.com/blog/2024/04/04/group_by-summarize-ungroup-animations/)
-
Tidy Animated Verbs - inner_join(), left_join(), right_join(), full_join(), semi_join(), anti_join(), union(), union_all(), intersect(), setdiff(), pivot_wider(), pivot_longer(), spread(), gather() (https://www.garrickadenbuie.com/project/tidyexplain/)
- PyTorch Image Models (https://github.com/rwightman/pytorch-image-models)
- EfficientNet PyTorch (https://github.com/lukemelas/EfficientNet-PyTorch)
- blurr - A library that integrates huggingface transformers with version 2 of the fastai framework (https://ohmeow.github.io/blurr/)
- MIT 6.S191: Introduction to Deep Learning - Labs (https://github.com/aamini/introtodeeplearning/)
- GitHub Profile README Generator (https://github.com/rahuldkjain/github-profile-readme-generator)
- Applied ML (https://github.com/eugeneyan/applied-ml)
- A detailed example of how to generate your data in parallel with PyTorch (https://github.com/shervinea/pytorch-data-generator)
- Keras vs. PyTorch: Alien vs. Predator recognition with transfer learning (https://github.com/deepsense-ai/Keras-PyTorch-AvP-transfer-learning)
- Deep Learning with Catalyst (https://github.com/catalyst-team/dl-course)
- sgugger - Deep Learning Notebooks (https://github.com/sgugger/Deep-Learning)
- Yang Zhang - fast.ai machine learning course notes (https://gist.github.com/yang-zhang/7ce6e6e7174c35ae26b7ce0dba1999d2)
- ML Course Notes (https://github.com/dair-ai/ML-Course-Notes)
- Tez: a simple pytorch trainer (https://github.com/abhi1thakur/tez)
- Kaggle-Ensemble-Guide (https://github.com/MLWave/Kaggle-Ensemble-Guide)
- Collection of useful data science topics along with code and articles (https://github.com/khuyentran1401/Data-science)
- Data Science Articles (https://github.com/parulnith/Data-Science-Articles/blob/main/README.md)
- James Le (https://github.com/khanhnamle1994)
- Donne Martin (https://github.com/donnemartin)
- Fraud-Detection-Handbook (https://github.com/Fraud-Detection-Handbook)
- VADER-Sentiment-Analysis (https://github.com/cjhutto/vaderSentiment)
- ECCO - Interfaces for Explaining Transformer (https://github.com/jalammar/ecco)
- MLBoy (https://github.com/john-rocky)
- Criteo (https://github.com/criteo)
- NVIDIA Deep Learning Examples for Tensor Cores (https://github.com/NVIDIA/DeepLearningExamples)
- Pinecone (https://github.com/pinecone-io/examples)
- Surprise - a Python scikit for recommender systems that deal with explicit rating data (https://github.com/NicolasHug/Surprise)
- PQk-means is a Python library for efficient clustering of large-scale data (https://github.com/DwangoMediaVillage/pqkmeans)
- GraphEmbedding (https://github.com/shenweichen/GraphEmbedding)
- Dataquest - Project Walkthroughs (https://github.com/dataquestio/project-walkthroughs)
- Retinal Vessel Segmentation with data augmentation and Keras (https://github.com/onurboyar/Retinal-Vessel-Segmentation)
- Visual Search with MXNet Gluon and HNSW (https://github.com/ThomasDelteil/VisualSearch_MXNet)
- DAIR.AI - Democratizing Artificial Intelligence Research, Education, and Technologies (https://github.com/dair-ai)
-
ML Visuals (https://github.com/dair-ai/ml-visuals)
-
ML Notebooks - examples for all sorts of machine learning tasks and applications (https://github.com/dair-ai/ML-Notebooks)
-
ML YouTube Courses (https://github.com/dair-ai/ML-YouTube-Courses)
-
Transformer Recipe (https://github.com/dair-ai/Transformers-Recipe)
-
Graph Neural Networks (GNNs Recipe) (https://github.com/dair-ai/GNNs-Recipe)
- NannyML estimates performance with Confidence-based Performance estimation (CBPE) - Predict Your Model’s Performance (Without Waiting for the Control Group)(https://towardsdatascience.com/predict-your-models-performance-without-waiting-for-the-control-group-3f5c9363a7da)(https://github.com/NannyML/nannyml)
- Obsei (https://github.com/obsei/obsei)
- nanoGPT - The simplest, fastest repository for training/finetuning medium-sized GPTs (https://github.com/karpathy/nanoGPT)
- Trax — Deep Learning with Clear Code and Speed (https://github.com/google/trax/tree/af3a38917bd1bc69cf5d25ce007e16185f22f050)
- Lightning-Hydra-Template (https://github.com/ashleve/lightning-hydra-template)
- https://github.com/ChanCheeKean/DataScience/
- Tracking Progress in Natural Language Processing (https://github.com/sebastianruder/NLP-progress/tree/master)
- Kaggle Solutions (https://farid.one/kaggle-solutions/)
- https://www.kaggle.com/bextuychiev/notebooks
- Kaggle utils (https://github.com/daxiongshu/kaggle_utils)
- Competition and Community Insights from NVIDIA’s Kaggle Grandmasters (https://developer.nvidia.com/blog/competition-and-community-insights-from-nvidias-kaggle-grandmasters/)
- Abid (https://deepnote.com/@abid)
- Machine Learning Mastery (https://machinelearningmastery.com/start-here/)
- Analytics Vidhya (https://www.analyticsvidhya.com/blog/)
- Cassie Kozyrkov - Decision Intelligence (https://decision.substack.com/)
- Kaggle Winner's Blog (https://medium.com/kaggle-blog)
- Terrence Shin (https://terenceshin.com/)
- Sylvain Gugger (https://sgugger.github.io/)
- Jason Yosinski (https://yosinski.com/)
- Lankinen (Fast.ai Lesson notes) (https://lankinen.medium.com/)
- Zachary Mueller (https://muellerzr.github.io/fastblog/)
- Jay Alammar (http://jalammar.github.io/)
- colah's blog - Christopher Olah (https://colah.github.io/)
- DataMuni (https://www.datamuni.com/)
- Chris McCormick (https://mccormickml.com/)
- Sebastian Ruder (https://ruder.io/)
- Gilbert Tanner (https://ml-explained.com/)
- Andrey Lukyanenko - Paper Review (https://andlukyane.com/blog/)
- Jeremy Jordan (https://www.jeremyjordan.me/data-science/)
- Mario - Kaggle Grandmaster (https://forecastegy.com/)
- Chip Huyen (https://huyenchip.com/blog/)
- Daniel Tunkelang - Search Fundamentals (https://dtunkelang.medium.com/)
- Daniel Lemire - crazily fast code (https://lemire.me/en/)
- NLPlanet (https://www.nlplanet.org/blog/index.html)
- Uncle Cang decoding - Lucene source code series (https://juejin.cn/user/2559318800998141)
- Philschmid's blog on Transformers & SageMaker (by Philipp Schmid) (https://www.philschmid.de/)
- The AiEdge Newsletter (by Damien Benveniste) (https://newsletter.theaiedge.io/archive)
- Ahead of AI (by Sebastian Raschka) (https://magazine.sebastianraschka.com/archive) (https://sebastianraschka.com/blog/)
- The Kaitchup – AI on a Budget - by Benjamin Marie (https://kaitchup.substack.com/archive)
- CheeKean (https://kean-chan.medium.com/)
- Data.gov.sg Blog (https://blog.data.gov.sg/)
- GovTech’s Data Science and Artificial Intelligence Division (DSAID) (https://medium.com/dsaid-govtech)
- Flipkart Tech Blog (https://tech.flipkart.com/)
- RAPIDS AI (https://medium.com/rapids-ai) (https://github.com/rapidsai-community/notebooks-contrib/tree/main/getting_started_materials)
- Pinterest Engineering (https://medium.com/pinterest-engineering)
- Netflix Tech Blog (https://netflixtechblog.com/)
- Uber Engineering (https://eng.uber.com/)
- Meta AI (https://ai.facebook.com/blog)
- Twitter Engineering (https://blog.twitter.com/engineering/en_us)
- DoorDash Engineering Blog (https://doordash.engineering/blog/)
- The Airbnb Tech Blog (https://medium.com/airbnb-engineering)
- Zalando Engineering Blog (https://engineering.zalando.com/)
- Linkedin Engineering Blog (https://engineering.linkedin.com/blog)
- Microsoft Experimentation Platform (https://www.microsoft.com/en-us/research/group/experimentation-platform-exp/)
- Etsy (https://codeascraft.com/)
- The Unofficial Google Data Science Blog (https://www.unofficialgoogledatascience.com/)
- Stitch Fix (https://multithreaded.stitchfix.com/blog/)
- Lyft (https://eng.lyft.com/tagged/data-science)
- Booking.com (https://booking.ai/)
- Yelp Engineering Blog (https://engineeringblog.yelp.com/)
- Spotify (https://engineering.atspotify.com/category/data-science/)
- EXP (https://exp-platform.com/encyclopediamldm/)
- Capital One (https://www.capitalone.com/tech/machine-learning/)
- Google AI Blog (https://ai.googleblog.com/)
- NVIDIA Merlin (https://medium.com/nvidia-merlin)
- Criteo Tech Blog (https://medium.com/criteo-engineering)(https://labs.criteo.com/engineering-blog/)
- Grab (https://engineering.grab.com/categories/data-science/)
- Elucida (https://medium.com/elucidata/tagged/technology)
- Zilliz (https://zilliz.com/learn)
- Neptune (https://neptune.ai/blog)
- Maths is Fun (https://www.mathsisfun.com/)
- Matrix Multiplication (http://matrixmultiplication.xyz/)
- The Matrix Calculus You Need For Deep Learning (https://explained.ai/matrix-calculus/)
- What's behind matrix multiplication? (https://www.tivadardanka.com/blog/behind-matrix-multiplication)
- Descriptive Matrix Operations with Einops - example with multi-query attention (https://www.kolaayonrinde.com/blog/2024/01/08/einops.html)
- Geometric Mean (https://www.mathsisfun.com/numbers/geometric-mean.html)
- Dot Product (https://www.mathsisfun.com/algebra/vectors-dot-product.html)
- Things that confused me about cross-entropy (https://chris-said.io/2020/12/26/two-things-that-confused-me-about-cross-entropy/)
- Mathematics for Machine Learning (https://github.com/dair-ai/Mathematics-for-ML)
- Seeing Theory - Visual introduction to probability and statistics (https://seeing-theory.brown.edu/index.html#firstPage)
- Calculus and Differentiation Primer (https://sebastianraschka.com/pdf/books/dlb/appendix_d_calculus.pdf)
- Understanding Automatic Differentiation in 30 lines of Python (https://vmartin.fr/understanding-automatic-differentiation-in-30-lines-of-python.html)
- What Are Floating-point Numbers? (https://www.baseclass.io/newsletter/floating-point-numbers)
- Monte Carlo Simulation — a practical guide (https://towardsdatascience.com/monte-carlo-simulation-a-practical-guide-85da45597f0e)
- Gradient Descent Algorithm — a deep dive (https://towardsdatascience.com/gradient-descent-algorithm-a-deep-dive-cf04e8115f21)
- 24 Free Datasets for Building an Irresistible Portfolio (2022) (https://www.dataquest.io/blog/free-datasets-for-projects/)
- Best Public Datasets for Machine Learning and Data Science (https://pub.towardsai.net/best-datasets-for-machine-learning-data-science-computer-vision-nlp-ai-c9541058cf4f)
- Amazon Review Data (2018) (https://nijianmo.github.io/amazon/index.html)
- Recommender Systems and Personalization Datasets (https://cseweb.ucsd.edu/~jmcauley/datasets.html)
- Amazon product data (outdated?) (https://jmcauley.ucsd.edu/data/amazon/index.html)
- Amazon Customer Reviews Dataset (https://s3.amazonaws.com/amazon-reviews-pds/readme.html)
- Papers With Code (https://paperswithcode.com/datasets)
- https://research.google/tools/
- Criteo - Terabyte Click Logs (https://labs.criteo.com/2013/12/download-terabyte-click-logs/)
- Components (https://components.one/datasets)
- Texmex (http://www.irisa.fr/texmex/ressources/index_en.php)
-
The MOVI Image Base (http://www.irisa.fr/texmex/ressources/bases/base_images_movi/index.html)
-
INRIA Holidays dataset for evaluation of image search & Copydays dataset for evaluation of copy detection (http://lear.inrialpes.fr/people/jegou/data.php#holidays)
-
Datasets for approximate nearest neighbor search (http://corpus-texmex.irisa.fr/)
- Semantic Text Similarity Dataset Hub (https://github.com/brmson/dataset-sts)
- GroupLens (https://grouplens.org/datasets/movielens/)
- Phishing Detection (Designing a New Net for Phishing Detection with NVIDIA Morpheus)(https://developer.nvidia.com/blog/designing-a-new-net-for-phishing-detection-with-nvidia-morpheus/)
-
SPAM_ASSASSIN dataset (https://spamassassin.apache.org/old/publiccorpus/)
-
Enron Emails dataset (https://www.cs.cmu.edu/~enron/)
-
Clair dataset (https://www.kaggle.com/datasets/rtatman/fraudulent-email-corpus)
- CSE-CIC-IDS2018 on AWS - Dataset for Network Intrusion Detection (https://www.unb.ca/cic/datasets/ids-2018.html)
- MIcrosoft News Dataset (MIND) (https://docs.microsoft.com/en-us/azure/open-datasets/dataset-microsoft-news?tabs=azureml-opendatasets) (https://www.kaggle.com/datasets/arashnic/mind-news-dataset)
- 190k+ Medium Articles (https://www.kaggle.com/datasets/fabiochiusano/medium-articles)
- Unsplash (https://unsplash.com/data)
- Randomizing Very Large Datasets (https://towardsdatascience.com/randomizing-very-large-datasets-e2b14e507725)
- MOSTLY AI, the #1 Synthetic Data Platform (https://mostly.ai/)
- No data? No problem! Generating synthetic training data at scale for NLP tasks using T0PP (https://medium.com/criteo-engineering/no-data-no-problem-generating-synthetic-training-data-at-scale-for-nlp-tasks-using-t0pp-198581643c5b)
- Generating Synthetic Data with Transformers (https://developer.nvidia.com/blog/generating-synthetic-data-with-transformers-a-solution-for-enterprise-data-challenges/)
- https://github.com/NVIDIA/NeMo/blob/r1.8.0/tutorials/nlp/Megatron_Synthetic_Tabular_Data_Generation.ipynb
- Faiss SyntheticDataset (https://github.com/facebookresearch/faiss/blob/main/contrib/datasets.py#L72)(https://gist.github.com/mdouze/551ef6fa0722f2acf58fa2c6fce732d6#file-demo_pytorch_knn-ipynb)
- YData Synthetic - generate synthetic tabular and time-series data (https://github.com/ydataai/ydata-synthetic)
- Generate a synthetic domain-specific Q&A dataset in <30 minutes (https://decodingml.substack.com/p/problems-deploying-your-ml-models)
- Future Tools (https://www.futuretools.io/)
- Scanner App for Math and Science (https://mathpix.com/)
- Readability (https://pypi.org/project/readability/)
- ipdb.set_trace() Commands (https://xxx-cook-book.gitbooks.io/python-cook-book/content/Debug/ipdb.html)
- Writing Math Equations in Jupyter Notebook: A Naive Introduction (https://medium.com/analytics-vidhya/writing-math-equations-in-jupyter-notebook-a-naive-introduction-a5ce87b9a214)
- Carbon - Create and share beautiful images of your source code (https://carbon.now.sh/)
- Manim Community - How to Create Mathematical Animations (https://docs.manim.community/en/stable/tutorials.html)
- diagrams.net (https://www.diagrams.net/)
- PlotNeuralNet: Use LaTex for making neural networks diagrams (https://github.com/HarisIqbal88/PlotNeuralNet)
- Keyword Tool (https://keywordtool.io/)
- Octoparse web scraping (https://www.octoparse.com/blog/10-myths-about-web-scraping)
- Capitalize My Title (https://capitalizemytitle.com/)
- Free IEEE Citation Generator (https://www.citethisforme.com/citation-generator/ieee)
- Shorten URLs (https://bitly.com/)
- Profile Pic Maker (https://pfpmaker.com/)
- Pictory - Online Video Creator (https://pictory.ai/)
- CapCut - Free Video Editor (https://www.capcut.com/)
- Video Creation (https://invideo.io/)
- Free Video Recording & Live Streaming (https://obsproject.com/)
- Boost your YouTube views (https://vidiq.com/)
- Human X AI Generative Music (https://mubert.com/)
- AI Speech Software (https://beta.elevenlabs.io/)
- Digital People Text-to-Video (https://www.d-id.com/)
- BlobCity AI Seed Projects (https://github.com/blobcity/ai-seed)
- Meta AI Frameworks, Tools, Libraries, Models (https://ai.facebook.com/tools)
- Resume Worded (https://resumeworded.com/resume-bullet-points)
- Mockaroo - Random Data Generator (https://mockaroo.com/)
- Flourish - Beautiful and easy data visualization and storytelling (https://flourish.studio/)
- Flameshot - screenshot software (https://github.com/flameshot-org/flameshot)
- Facets - visualizations for understanding and analyzing machine learning datasets (https://github.com/PAIR-code/facets)
- Google - People + AI Research (PAIR) (https://research.google/teams/brain/pair/)
- Google Research resources (https://research.google/tools/)
- Frederik Brasz - Voronoi generator (https://cfbrasz.github.io/programs.html)
- handcalcs - Python calculations in Jupyter (https://github.com/connorferster/handcalcs)
- Towhee - open-source ML pipeline to encode unstructured data into embeddings (https://towhee.io/)
- Gradio - Build & Share Delightful Machine Learning Apps (https://gradio.app/)
-
🤗 Gradio example (https://huggingface.co/spaces/gradio/xgboost-income-prediction-with-explainability)
-
🤗 Gradio example (https://huggingface.co/spaces/gradio/chatbot_dialogpt/blob/main/run.py)
-
🤗 Gradio example (https://huggingface.co/spaces/gradio/chatbot/blob/main/app.py)
- CurlWget (https://www.analyticsvidhya.com/blog/2021/08/load-dataset-directly-into-colab-from-anywhere-on-the-browser-using-curlwget-extension/)
- GLTR - detect automatically generated text (http://gltr.io/)
- Keenious - research explorer (https://keenious.com/) (https://medium.com/keenious/knowledge-graph-search-of-60-million-vectors-with-weaviate-7964657ec911)
- AutoRegex - Effortless conversions between English and RegEx (https://www.autoregex.xyz/)
- Stable Diffusion (https://huggingface.co/spaces/stabilityai/stable-diffusion)
- Quarto - open-source scientific and technical publishing system (https://quarto.org/)
- JSON Crack - Visualize JSON with graphs (https://jsoncrack.com/)
- markmap (markdown + mindmap) (https://markmap.js.org/repl)
- Newspaper3k: Article scraping & curation (https://github.com/codelucas/newspaper)
- Trafilatura - Python package and command-line tool to gather text on the Web (https://trafilatura.readthedocs.io/en/latest/)
- GoogleNews (https://github.com/Iceloof/GoogleNews)
- Interactive network visualizations (https://pyvis.readthedocs.io/en/latest/index.html)
- markdownify - Convert HTML to markdown (https://pypi.org/project/markdownify/)
- ResumeWizard (https://resume-wizard.vercel.app/)
- 12 AI Copywriting Tools to Improve Efficiency (https://ahrefs.com/blog/ai-copywriting/)
- Icons, backgrounds, templates, graphics, etc for presentations:
-
-
- News API - Search worldwide news with code (https://newsapi.org/)
- MkDocs: static site generator that's geared towards building project documentation (https://www.mkdocs.org/) (https://github.com/mkdocs/mkdocs)
- Material for MkDocs (https://www.youtube.com/watch?v=Q-YA_dA8C20) (https://squidfunk.github.io/mkdocs-material/) (https://github.com/squidfunk/mkdocs-material) (https://www.stevemar.net/five-things-about-mkdocs/) (https://docs.markhh.com/pages/tools/mkdocs_demo/) (https://www.starfallprojects.co.uk/blog/mkdocs-material-blog-cover-image/) (https://www.codeinsideout.com/blog/site-setup/create-site-project/) (https://blog.ktz.me/making-mkdocs-tables-look-like-github-markdown-tables/)
- data load tool - dlt (https://dlthub.com/docs/intro)
- cleanlab automatically detects data and label issues in your ML datasets (https://docs.cleanlab.ai/stable/index.html)
- Public APIs (https://github.com/public-apis/public-apis)
- How to configure VS Code for AI, ML and MLOps development in Python 🛠️️ (https://mlops.community/how-to-configure-vs-code-for-ai-ml-and-mlops-development-in-python-%F0%9F%9B%A0%ef%b8%8f%ef%b8%8f/)
- Great Tables - Absolutely Delightful Table-making in Python (https://posit-dev.github.io/great-tables/articles/intro.html)
-
The Design Philosophy of Great Tables (https://posit-dev.github.io/great-tables/blog/design-philosophy/)
- Chunk visualizer (https://huggingface.co/spaces/m-ric/chunk_visualizer)
- Bytewax - Stream processing purely in Python (https://bytewax.io/)
- Unstructured - Preprocess and structure unstructured text documents (such as PDFs, XML and HTML) for use in downstream machine learning tasks (https://unstructured-io.github.io/unstructured/core/cleaning.html)
- Upstash Serverless Kafka & Vector Database (https://upstash.com/)
- Meet the NiceGUI: Your Soon-to-be Favorite Python UI Library (https://towardsdatascience.com/meet-the-nicegui-your-soon-to-be-favorite-python-ui-library-fb69f14bb0ac) (https://nicegui.io/documentation)
- imodels - Interpretable ML package (https://github.com/csinva/imodels)
- Interview Query - Questions and Blogs (https://www.interviewquery.com/p/data-science-interview-questions)(https://www.interviewquery.com/articles)(https://www.interviewquery.com/blog)
- Machine Learning FAQ (https://sebastianraschka.com/faq/)
- Mastering the Deep Learning Interview: Top 35 Questions and Expert Answers (https://medium.com/@riteshgupta.ai/mastering-the-deep-learning-interview-top-35-questions-and-expert-answers-aabb701f6e45)
- Machine Learning Interviews Book (https://huyenchip.com/ml-interviews-book/)
- Deep Learning Interviews: Hundreds of fully solved job interview questions from a wide range of key topics in AI (https://github.com/BoltzmannEntropy/interviews.ai)
- Machine Learning Interviews (https://github.com/khangich/machine-learning-interview/)
- NLP Interview Question and Answers in 2022 (https://www.mygreatlearning.com/blog/nlp-interview-questions/)
- Machine Learning Interview Question and Answers in 2022 (https://www.mygreatlearning.com/blog/machine-learning-interview-questions/)
- Machine learning algorithms interview - tips & resources (https://workera.ai/resources/machine-learning-algorithms-interview/)
- Crack Data Science Interviews: Essential Machine Learning Concepts (https://towardsdatascience.com/crack-data-science-interviews-essential-machine-learning-concepts-afd6a0a6d1aa)
- Top 20 AB Test Interview Questions And Answers (https://grabngoinfo.com/top-20-ab-test-interview-questions-and-answers/)
- AB Testing 101 (https://medium.com/jonathans-musings/ab-testing-101-5576de6466b)
- How to use Causal Inference when A/B testing is not available (https://towardsdatascience.com/how-to-use-causal-inference-when-a-b-testing-is-not-possible-c87c1252724a)
- Glassdoor machine learning interview questions (https://www.glassdoor.sg/Interview/machine-learning-interview-questions-SRCH_KO0,16.htm?countryRedirect=true)
- subreddit - ML question (https://www.reddit.com/r/MLQuestions/)
- subreddit - Learning machine learning (https://www.reddit.com/r/learnmachinelearning/)
- Ten Advanced SQL Concepts You Should Know for Data Science Interviews (https://towardsdatascience.com/ten-advanced-sql-concepts-you-should-know-for-data-science-interviews-4d7015ec74b0)
- SQL Group By and Partition By Scenarios: When and How to Combine Data in Data Science (https://www.kdnuggets.com/sql-group-by-and-partition-by-scenarios-when-and-how-to-combine-data-in-data-science)
- Creating a Data Science Portfolio (https://towardsdatascience.com/creating-a-data-science-portfolio-bd485382f49)
- Data Science Portfolio (https://github.com/MaartenGr/projects)
- Github digital cv example (https://github.com/March-08/digital-cv)
- How to Build a Data Science Portfolio Website using Python (https://towardsdatascience.com/how-to-build-a-data-science-portfolio-website-using-python-79531426fde5)
- How to create a Medium-like personal blog for free in a day (https://medium.com/geekculture/how-to-create-a-medium-like-personal-blog-for-free-in-a-day-55ebd9551d9c)
- How to Swiftly Launch a Free Website With GitHub Pages (https://www.stephenvinouze.com/how-to-swiftly-launch-a-free-website-with-github-pages/)
- TYJ (https://tengyeejing.com/)
- The Portfolio that Got Me a Data Scientist Job (https://towardsdatascience.com/the-portfolio-that-got-me-a-data-scientist-job-513cc821bfe4)
- Set Up Your Portfolio Website in Less Than 10 Minutes with Github Pages (https://medium.com/@evanca/set-up-your-portfolio-website-in-less-than-10-minutes-with-github-pages-d0efa8ff56fd) (https://github.com/evanca/quick-portfolio)
- Use Python and NLP to Boost Your Resume (https://medium.com/data-marketing-philosophy/use-python-and-nlp-to-boost-your-resume-e4691a58bcc9)
- Facebook Field Guide to Machine Learning (https://research.facebook.com/blog/2018/05/the-facebook-field-guide-to-machine-learning-video-series/)
- A Guide to Production Level Deep Learning (https://github.com/alirezadir/Production-Level-Deep-Learning)
- Rules of ML (https://developers.google.com/machine-learning/guides/rules-of-ml)
- Impactful and widely cited papers and literature on ML/DL/RL/AI (https://github.com/tirthajyoti/Papers-Literature-ML-DL-RL-AI)
- Definitive Interview prep ROADMAP (https://www.codinginterview.com/interview-roadmap)
- interviewing.io - Watch technical mock interviews (https://interviewing.io/recordings)
- Interview Cake (https://www.interviewcake.com/)
- Interview Questions (https://www.tryexponent.com/questions)
- Free Interview Practice (https://www.pramp.com/)
- Big-O Cheat Sheet (https://www.bigocheatsheet.com/)
- Leetcode list by topics (more comprehensive): https://protegejj.gitbook.io/oj-practices/chapter1/dynamic-programming
- Leetcode Company Tag (https://github.com/xizhengszhang/Leetcode_company_frequency)
- Python_LeetCode_Coding (https://github.com/LeihuaYe/Python_LeetCode_Coding)
- Software Engineering Interview Preparation (GitBook) (https://orrsella.gitbooks.io/soft-eng-interview-prep/content/)
- 4 questions to ask in interviews to assess codebase health (https://www.educative.io/blog/questions-assess-codebase-health-interviews)
- How to Tackle The 7 Most Common Types of Interview Questions (https://www.youtube.com/watch?v=vFdkhsN1PJo&t=137s)
- Instamentor's Knowledge Center (https://instamentor.com/articles/)
- Experimentation is a major focus of Data Science across Netflix (https://netflixtechblog.com/experimentation-is-a-major-focus-of-data-science-across-netflix-f67923f8e985)
- Fighting Overfitting With L1 or L2 Regularization: Which One Is Better? (https://neptune.ai/blog/fighting-overfitting-with-l1-or-l2-regularization)
- Machine Learning Projects You NEVER Knew Existed (https://www.youtube.com/watch?v=sw3o0rAazMg&list=RDCMUCHXa4OpASJEwrHrLeIzw7Yg&index=6)
- https://resumaker.ai/app/build/templates/
- Scarlet Ink by Dave Anderson - Interview Advice (https://www.scarletink.com/tag/interviewing/)
-
How can I recover a job offer I rejected, or a job I quit? (https://www.scarletink.com/questions-answers-reddit-cscareerquestions-experiment/)
-
Writing and Speaking Clearly and Concisely (https://www.scarletink.com/writing-speaking-clearly-concisely/)
- The Top 3 Resume Mistakes Costing You the Job (https://blog.bytebytego.com/p/the-top-3-resume-mistakes-costing)
- The Amazon “secret” of controllable inputs for your career (https://levelupwithethanevans.substack.com/p/the-amazon-secret-of-controllable)
- ML Systems Design Interview Guide (http://patrickhalina.com/posts/ml-systems-design-interview-guide/)
- level.fyi (https://www.levels.fyi/Salaries/Data-Scientist/Singapore/)
- Ten Rules for Negotiating a Job Offer (https://haseebq.com/my-ten-rules-for-negotiating-a-job-offer/)
- HOW TO NEGOTIATE SALARY: 9 TIPS FROM A PRO SALARY NEGOTIATOR (https://fearlesssalarynegotiation.com/salary-negotiation-guide/)
- HOW TO WRITE A SALARY NEGOTIATION EMAIL (WITH 11 PROVEN TEMPLATES AND A SAMPLE) (https://fearlesssalarynegotiation.com/salary-negotiation-email-sample/#ask-for-time-template)
- Salary Negotiation: Make More Money, Be More Valued (https://www.kalzumeus.com/2012/01/23/salary-negotiation/)
- ByteByteGo - System Design 101 (https://blog.bytebytego.com/) (https://github.com/ByteByteGoHq/system-design-101)
- System Design Interview (https://systeminterview.com/scale-from-zero-to-millions-of-users.php)
- SYSTEM DESIGN INTERVIEW PREPARATION SERIES (https://www.codekarle.com/)
- The complete guide to system design in 2022 (https://www.educative.io/blog/complete-guide-to-system-design#filestorage)
- Systems Design Crash Course for ML Engineers (https://towardsdatascience.com/systems-design-crash-course-for-ml-engineers-aafae1cf1890)
- The System Design Primer (https://github.com/donnemartin/system-design-primer)
- awesome-scalability (https://github.com/binhnguyennus/awesome-scalability)
- System Architecture (https://orrsella.gitbooks.io/soft-eng-interview-prep/content/topics/system-architecture.html)
- Dynamo: Amazon’s Highly Available Key-value Store (https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf)
- System Design Series by Sanil Khurana (https://medium.com/@sanilkhurana7)
-
System Design Series: The Ultimate Guide for Building High-Performance Data Streaming Systems from Scratch! (https://towardsdatascience.com/system-design-series-0-to-100-guide-to-data-streaming-systems-3dd584bd28fa)
- ML Systems Design Interview Guide (http://patrickhalina.com/posts/ml-systems-design-interview-guide/)
- Dynamic Programming Patterns (https://leetcode.com/discuss/general-discussion/458695/dynamic-programming-patterns)
- Binary Search Template (https://leetcode.com/discuss/general-discussion/786126/Python-Powerful-Ultimate-Binary-Search-Template.-Solved-many-problems)
- Binary Search (https://www.youtube.com/watch?v=tgVSkMA8joQ)
- Partition subset problem - all approaches explained (https://leetcode.com/problems/partition-equal-subset-sum/solutions/462699/Whiteboard-Editorial.-All-Approaches-explained)
- Neet Code (https://www.youtube.com/c/NeetCode/playlists)
- MIT 6.006 Introduction to Algorithms, Fall 2011 (https://www.youtube.com/playlist?list=PLUl4u3cNGP61Oq3tWYp6V_F-5jb5L2iHb)
- Vivekanand Khyade - Algorithm Every Day (https://www.youtube.com/user/vivekanandkhyade/playlists)
- Gaurav Sen (https://www.youtube.com/channel/UCRPMAqdtSgd0Ipeef7iFsKw)
- Coding Interview Solutions (https://www.youtube.com/playlist?list=PLot-Xpze53leF0FeHz2X0aG3zd0mr1AW_)
- Kevin Naughton Jr. (https://www.youtube.com/c/KevinNaughtonJr/playlists)
- Sai Anish Malla (https://www.youtube.com/channel/UCFBorf0jHu-1WNHGJZDNtew/playlists)
- Back To Back SWE (https://www.youtube.com/c/BackToBackSWE/playlists)
- Tech Dummies Narendra L (https://www.youtube.com/c/TechDummiesNarendraL/videos)
- Data Interview Pro - Emma (https://www.youtube.com/c/DataInterviewPro/playlists)
- Bit Hacks (https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-172-performance-engineering-of-software-systems-fall-2018/lecture-slides/MIT6_172F18_lec3.pdf)
- Bit Twiddling Hacks (http://graphics.stanford.edu/~seander/bithacks.html)
- Introduction to Low Level Bit Hacks (https://catonmat.net/low-level-bit-hacks)
- Bitwise operations cheat sheet ()
- Bits, Bytes, Building With Binary (https://medium.com/basecs/bits-bytes-building-with-binary-13cb4289aafa)
- The Binary Cheatsheet (https://timseverien.github.io/binary-cheatsheet/)
- Binary cheatsheet for coding interviews (https://www.techinterviewhandbook.org/algorithms/binary/)
- Bitwise Hacks for Competitive Programming (https://www.geeksforgeeks.org/bitwise-hacks-for-competitive-programming/)
- Bitwise Operators in Python (https://realpython.com/python-bitwise-operators/)
- Signed number representations (https://en.wikipedia.org/wiki/Signed_number_representations)
- Pack and Unpack data (Masking) (https://www.youtube.com/watch?v=MWFwM9-2nlI)
- Bit Packing or How to Love AND, OR or XOR (https://learnmongodbthehardway.com/article/bitflipping/)
- BIt Packing (https://www.cs.waikato.ac.nz/~tcs/COMP317/bitpacking.html)
- https://codeforces.com/blog/entry/50841
- Foobar Challenge: Google’s Secret Hiring Process (https://towardsdatascience.com/how-to-get-hired-by-google-b19806ad3c62)
- Google Has a Secret Hiring Challenge Called Foobar (https://betterprogramming.pub/google-has-a-secret-hiring-challenge-called-foobar-14625bfcea7a)(https://github.com/FBosler/GoogleFoobar)
- Dodge The Lasers — Fantastic Question From Google’s hiring challenge (https://towardsdatascience.com/dodge-the-lasers-fantastic-question-from-googles-hiring-challenge-72363d95fec)
- PyTorch 2 Internals (https://blog.christianperone.com/2023/12/pytorch-2-internals-talk/)
- Torch Tensor Operations (https://jhui.github.io/2018/02/09/PyTorch-Basic-operations/)
- Optimize PyTorch Performance for Speed and Memory Efficiency (18 Tips) - (https://towardsdatascience.com/optimize-pytorch-performance-for-speed-and-memory-efficiency-2022-84f453916ea6)
- Faster Deep Learning Training with PyTorch – a 2021 Guide (https://efficientdl.com/faster-deep-learning-in-pytorch-a-guide/)
- How to fine tune VERY large model if it doesn’t fit on your GPU (https://bestasoff.medium.com/how-to-fine-tune-very-large-model-if-it-doesnt-fit-on-your-gpu-3561e50859af)
- Finetuning Torchvision Models (https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html)
- Transfer Learning with ResNet in PyTorch (https://www.pluralsight.com/guides/introduction-to-resnet)
- Notes in pytorch to deal with ConvNets (https://github.com/mortezamg63/Accessing-and-modifying-different-layers-of-a-pretrained-model-in-pytorch/blob/master/README.md)
- Some important Pytorch tasks - A concise summary from a vision researcher (https://spandan-madan.github.io/A-Collection-of-important-tasks-in-pytorch/)
- PyTorch layer dimensions: what size and why? (https://towardsdatascience.com/pytorch-layer-dimensions-what-sizes-should-they-be-and-why-4265a41e01fd)
- PyTorch :: Understanding Tensors (Part 1) (https://dev.to/tbhaxor/pytorch-understanding-tensors-part-1-od8)
- VGG16 Transfer Learning - Pytorch (https://www.kaggle.com/carloalbertobarbano/vgg16-transfer-learning-pytorch)
- Pytorch to fastai, Bridging the Gap (https://muellerzr.github.io/fastblog/2021/02/14/Pytorchtofastai.html)
- A detailed example of how to generate your data in parallel with PyTorch (https://stanford.edu/~shervine/blog/pytorch-how-to-generate-data-parallel)
- Cyclic Learning Rates and One Cycle Policy (https://github.com/nachiket273/One_Cycle_Policy)
- Cyclical LR and momentums (https://github.com/sgugger/Deep-Learning/blob/master/Cyclical%20LR%20and%20momentums.ipynb)
- Pytorch Loss Functions in Plain Python (https://zhang-yang.medium.com/pytorch-loss-funtions-in-plain-python-b79c05f8b53f)
- Automatic Mixed Precision Training for Deep Learning using PyTorch (https://debuggercafe.com/automatic-mixed-precision-training-for-deep-learning-using-pytorch/)
- A developer-friendly guide to mixed precision training with PyTorch (https://spell.ml/blog/mixed-precision-training-with-pytorch-Xuk7YBEAACAASJam)
- Making Pytorch Transformer Twice as Fast on Sequence Generation (https://scale.com/blog/pytorch-improvements)
- Transformer Details Not Described in The Paper (https://tunz.kr/post/4)
- A collection of tips for speeding up learning and reasoning with PyTorch (https://qiita.com/sugulu_Ogawa_ISID/items/62f5f7adee083d96a587)
- Implement Early Stopping in PyTorch (https://qiita.com/ku_a_i/items/ba33c9ce3449da23b503)
- BERT Fine-Tuning Tutorial with PyTorch (https://mccormickml.com/2019/07/22/BERT-fine-tuning/)
- Utilizing Transformer Representations Efficiently (https://www.kaggle.com/rhtsingh/utilizing-transformer-representations-efficiently)
- Visualize BERT sequence embeddings: An unseen way (https://towardsdatascience.com/visualize-bert-sequence-embeddings-an-unseen-way-1d6a351e4568)
- PyTorch Tutorial: Paddy Disease Identification (https://www.kaggle.com/code/manabendrarout/pytorch-tutorial-paddy-disease-identification)
- PYTORCH LIGHTNING TUTORIALS (https://pytorch-lightning.readthedocs.io/en/stable/tutorials.html)
-
TUTORIAL 3: INITIALIZATION AND OPTIMIZATION (https://pytorch-lightning.readthedocs.io/en/stable/notebooks/course_UvA-DL/03-initialization-and-optimization.html)
-
TUTORIAL 5: TRANSFORMERS AND MULTI-HEAD ATTENTION (https://lightning.ai/docs/pytorch/stable/notebooks/course_UvA-DL/05-transformers-and-MH-attention.html)
- University of Amsterdam - UvA Deep Learning Tutorials (https://uvadlc-notebooks.readthedocs.io/en/latest/) (https://www.youtube.com/playlist?list=PLdlPlO1QhMiAkedeu0aJixfkknLRxk1nA)
- Debugging in PyTorch (https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/guide3/Debugging_PyTorch.html)
- TorchMetrics - How do we use it, and what's the difference between .update() and .forward()? (https://sebastianraschka.com/blog/2022/torchmetrics.html) (https://github.com/rasbt/torchmetrics-blog/blob/main/torchmetrics-update-forward.ipynb)
- PyTorch training codes with AverageMeter & ProgressMeter (https://docs.openvino.ai/2023.0/notebooks/302-pytorch-quantization-aware-training-with-output.html)
- Hooks: the one PyTorch trick you must know (https://tivadardanka.com/blog/hooks-the-one-pytorch-trick-you-must-know)
- How to modify a pretrained model (https://discuss.pytorch.org/t/how-to-modify-a-pretrained-model/60509)
Module.children()
vsModule.modules()
(https://discuss.pytorch.org/t/module-children-vs-module-modules/4551)- What is the distinct usage of the
AdaptiveConcatPool2d
layer? (https://forums.fast.ai/t/what-is-the-distinct-usage-of-the-adaptiveconcatpool2d-layer/7600) - Splitting a pretrained model in groups of layers (https://forums.fast.ai/t/splitting-a-pretrained-model-in-groups-of-layers/33012/2)
x.data
(https://stackoverflow.com/questions/51743214/is-data-still-useful-in-pytorch)model.eval()
vs withtorch.no_grad()
(https://discuss.pytorch.org/t/model-eval-vs-with-torch-no-grad/19615)- How do we resume training by using the last LR? (https://www.kaggle.com/c/seti-breakthrough-listen/discussion/247574)
- Tensor view vs. permute (https://stackoverflow.com/questions/51143206/difference-between-tensor-permute-and-tensor-view-in-pytorch)
- Torch
stack()
vs.cat()
(https://stackoverflow.com/questions/54307225/whats-the-difference-between-torch-stack-and-torch-cat-functions/54307331) - Why do transformers use layer norm instead of batch norm? (https://stats.stackexchange.com/questions/474440/why-do-transformers-use-layer-norm-instead-of-batch-norm)
- Deep Learning normalization methods (https://tungmphung.com/deep-learning-normalization-methods/)
- PyTorch vs TensorFlow in 2022 (https://www.assemblyai.com/blog/pytorch-vs-tensorflow-in-2022/)
- What is the difference between
register_buffer
andregister_parameter
ofnn.Module
(https://discuss.pytorch.org/t/what-is-the-difference-between-register-buffer-and-register-parameter-of-nn-module/32723) (https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module.register_buffer) Model.named_parameters()
will lose some layer modules (https://discuss.pytorch.org/t/model-named-parameters-will-lose-some-layer-modules/14588)model.parameters()
,model.named_parameters()
,model.state_dict()
(https://blog.csdn.net/qq_36429555/article/details/118609604)
- Jeremy Howard - Kaggle Notebooks (https://www.kaggle.com/jhoward/notebooks)
- fastkaggle (https://fastai.github.io/fastkaggle/)
- A Beginner’s Guide to Large Language Models (https://resources.nvidia.com/en-us-large-language-model-ebooks)
- End-To-End Speech AI Pipelines (https://resources.nvidia.com/en-us-speech-ai-ebooks-nurture/part-2)
- Numbers every LLM Developer should know (https://github.com/ray-project/llm-numbers#1-mb-gpu-memory-required-for-1-token-of-output-with-a-13b-parameter-model)
- Transformer Math 101 (https://blog.eleuther.ai/transformer-math/)
- LLM Parameter Counting (https://kipp.ly/transformer-param-count/)
- Transformer Inference Arithmetic (https://kipp.ly/transformer-inference-arithmetic/)
- AI Canon - a curated list of resources (https://a16z.com/2023/05/25/ai-canon/)
- A Very Gentle Introduction to Large Language Models without the Hype - 38 min read (https://mark-riedl.medium.com/a-very-gentle-introduction-to-large-language-models-without-the-hype-5f67941fa59e)
- The Animated Transformer (https://prvnsmpth.github.io/animated-transformer/)
- Transformer Anatomy guide (https://www.kaggle.com/code/pastorsoto/transformer-anatomy-guide)
- Notebooks for the book: Natural Language Processing with Transformers (https://github.com/nlp-with-transformers/notebooks)
- The Annotated Transformer - with PyTorch codes
-
Original: (http://nlp.seas.harvard.edu/2018/04/03/attention.html)
-
- Transformers: How Do They Transform Your Data? - explanation with codes (https://towardsdatascience.com/transformers-how-do-they-transform-your-data-72d69e383e0d) (https://github.com/maxime7770/Transformers-Insights)
- Attention? Attention! (https://lilianweng.github.io/posts/2018-06-24-attention/)
- The Transformer Family (https://lilianweng.github.io/posts/2020-04-07-the-transformer-family/)
- The Transformer Family Version 2.0 (https://lilianweng.github.io/posts/2023-01-27-the-transformer-family-v2/)
- Beautifully Illustrated: NLP Models from RNN to Transformer (https://towardsdatascience.com/beautifully-illustrated-nlp-models-from-rnn-to-transformer-80d69faf2109)
- Understanding Transformers: A Step-by-Step Math Example — Part 1 (https://medium.com/@fareedkhandev/understanding-transformers-a-step-by-step-math-example-part-1-a7809015150a)
-
Solving Transformer by Hand: A Step-by-Step Math Example (https://levelup.gitconnected.com/understanding-transformers-from-start-to-end-a-step-by-step-math-example-16d4e64e6eb1)
-
Building a Million-Parameter LLM from Scratch Using Python - A Step-by-Step Guide to Replicating LLaMA Architecture (https://levelup.gitconnected.com/building-a-million-parameter-llm-from-scratch-using-python-f612398f06c2)
- Understanding and Coding Self-Attention, Multi-Head Attention, Cross-Attention, and Causal-Attention in LLMs (https://magazine.sebastianraschka.com/p/understanding-and-coding-self-attention)
-
https://github.com/rasbt/LLMs-from-scratch/blob/main/ch03/01_main-chapter-code/ch03.ipynb
-
-
https://github.com/rasbt/LLMs-from-scratch/tree/main/ch03/02_bonus_efficient-multihead-attention
- Concept of self-attention (https://www.linkedin.com/posts/ugcPost-6882314741088321536-PNqG) (https://www.linkedin.com/feed/update/urn:li:activity:6879010048421445633)
- Transformers from Scratch - great explanation on dot products and matrix multiplication (https://e2eml.school/transformers.html)
- TRANSFORMERS FROM SCRATCH (https://peterbloem.nl/blog/transformers)
- Transformers From Scratch (https://blog.matdmiller.com/posts/2023-06-10_transformers/notebook.html)
- How Transformers work in deep learning and NLP: an intuitive introduction (https://theaisummer.com/transformer/)
- Getting Meaning from Text: Self-attention Step-by-step Video (https://pub.towardsai.net/getting-meaning-from-text-self-attention-step-by-step-video-7d8f49694f89) (https://www.youtube.com/watch?v=-9vVhYEXeyQ&t=570s)
- Transformer Architecture: The Positional Encoding (https://kazemnejad.com/blog/transformer_architecture_positional_encoding/)
- Explained: Multi-head Attention (Part 1) (https://storrs.io/attention/)
- Explained: Multi-head Attention (Part 2) (https://storrs.io/multihead-attention/)
- Deep learning explainer: a simple single cell classification model (https://storrs.io/sc-deep-learning-explainer/)
- Accelerating Large Language Models with Accelerated Transformers (https://pytorch.org/blog/accelerating-large-language-models/)
- Step-by-Step Illustrated Explanations of Transformer (https://medium.com/@yulemoon/detailed-explanations-of-transformer-step-by-step-dc32d90b3a98)
-
An In-Depth Look at the Transformer Based Models (https://medium.com/@yulemoon/an-in-depth-look-at-the-transformer-based-models-22e5f5d17b6b)
- Anti-hype LLM reading list (https://gist.github.com/veekaybee/be375ab33085102f9027853128dc5f0e)
- GPT in 60 Lines of NumPy (https://jaykmody.com/blog/gpt-from-scratch/)
- x-transformers - A concise but fully-featured transformer, complete with a set of promising experimental features from various papers (https://github.com/lucidrains/x-transformers/tree/main)
- Transformer Taxonomy (https://kipp.ly/transformer-taxonomy/)
- Large Language Models: SBERT — Sentence-BERT (https://towardsdatascience.com/sbert-deb3d4aef8a4)
- How context sizes of 100k tokens and longer are achieved (https://www.linkedin.com/posts/andriyburkov_in-case-you-were-wondering-how-context-sizes-activity-7154922110354604032-Tx1a)
- KV caching (https://medium.com/@plienhar/llm-inference-series-3-kv-caching-unveiled-048152e461c8) (https://medium.com/@plienhar/llm-inference-series-4-kv-caching-a-deeper-look-4ba9a77746c8)
- BertViz - tool for visualizing attention in Transformer model (https://github.com/jessevig/bertviz)
- LLM Visualization (https://bbycroft.net/llm)
- AttentionViz: A Global View of Transformer Attention (https://catherinesyeh.github.io/attn-docs/)
- Inside the Matrix: Visualizing Matrix Multiplication, Attention and Beyond (https://pytorch.org/blog/inside-the-matrix/)
- Explainable AI: Visualizing Attention in Transformers (https://generativeai.pub/explainable-ai-visualizing-attention-in-transformers-4eb931a2c0f8) (https://www.topbots.com/deconstructing-bert-part-1/) (https://www.topbots.com/deconstructing-bert-part-2/) (https://www.topbots.com/openai-gpt-2-visualization/)
- Deconstructing BERT, Part 2: Visualizing the Inner Workings of Attention (https://towardsdatascience.com/deconstructing-bert-part-2-visualizing-the-inner-workings-of-attention-60a16d86b5c1)
- Transformers Explained Visually (Part 3): Multi-head Attention, deep dive (https://towardsdatascience.com/transformers-explained-visually-part-3-multi-head-attention-deep-dive-1c1ff1024853)
- A Visual Guide to Vision Transformers (https://blog.mdturp.ch/posts/2024-04-05-visual_guide_to_vision_transformer.html)
- llama3 implemented from scratch (https://github.com/naklecha/llama3-from-scratch/blob/main/README.md)
- AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration [Paper] (https://github.com/mit-han-lab/llm-awq)
- AutoAWQ (https://github.com/casper-hansen/AutoAWQ)
- Microsoft DeepSpeed - Deep learning optimization software suite for both training and inference (https://github.com/microsoft/DeepSpeed) (https://www.deepspeed.ai/)
- DeepSpeed Chat - Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales (https://github.com/microsoft/DeepSpeedExamples/tree/master/applications/DeepSpeed-Chat) (https://github.com/microsoft/DeepSpeed/tree/master/blogs/deepspeed-chat)
- DeepSpeed's Bag of Tricks for Speed & Scale (https://www.kolaayonrinde.com/blog/2023/07/14/deepspeed-train.html)
- 🤗 PEFT: Parameter-Efficient Fine-Tuning of Billion-Scale Models on Low-Resource Hardware (https://huggingface.co/blog/peft) (https://github.com/huggingface/peft)
-
🤗 PEFT Documentation (https://huggingface.co/docs/peft/index)
-
🤗 PEFT Examples (https://github.com/huggingface/peft/tree/main/examples)
-
🤗 PEFT Patch Release (https://github.com/huggingface/peft/releases)
- 🤗 TRL - Transformer Reinforcement Learning (https://github.com/huggingface/trl) (https://huggingface.co/docs/trl/index)
- bitsandbytes - 8-bit optimizers, matrix multiplication (LLM.int8()), and quantization functions (https://github.com/TimDettmers/bitsandbytes)
- GPTQ: Accurate Post-training Compression for Generative Pretrained Transformers (https://github.com/IST-DASLab/gptq)
- AutoGPTQ: LLMs quantization package with user-friendly apis, based on GPTQ algorithm (https://github.com/PanQiWei/AutoGPTQ)
- Quantize 🤗 Transformers models (https://huggingface.co/docs/transformers/main_classes/quantization)
- Which Quantization Method is Right for You? (GPTQ vs. GGUF vs. AWQ) (https://maartengrootendorst.substack.com/p/which-quantization-method-is-right)
- Optimum-Benchmark 🏋️ (https://github.com/huggingface/optimum-benchmark)
- NEFTune - add random noise to the embedding vectors of the training data during the forward pass of fine-tuning (https://github.com/neelsjain/NEFTune)
- LoRA+: Efficient Low Rank Adaptation of Large Models (https://github.com/nikhil-ghosh-berkeley/loraplus)
- DoRA: Weight-Decomposed Low-Rank Adaptation (https://github.com/catid/dora/tree/main)
tensor_parallel
- much faster than huggingface'sdevice_map
and lightweight than vLLM? (https://github.com/BlackSamorez/tensor_parallel)- Nanotron - Minimalistic large language model 3D-parallelism training (https://github.com/huggingface/nanotron/)
- FastChat - platform for training, serving, and evaluating large language model based chatbots (https://github.com/lm-sys/FastChat)
- Half-Quadratic Quantization (HQQ) (https://mobiusml.github.io/hqq_blog/) (https://github.com/mobiusml/hqq)
- AQLM - Extreme Compression of Large Language Models via Additive Quantization (https://github.com/Vahe1994/AQLM) (https://towardsdatascience.com/the-aqlm-quantization-algorithm-explained-8cf33e4a783e)
- torchtune - A Native-PyTorch Library for LLM Fine-tuning (https://github.com/pytorch/torchtune)
- Prefect - Modern workflow orchestration for data and ML engineers (https://www.prefect.io/)
- Modal - serverless platform to run generative AI models, large-scale batch jobs, job queues, etc (https://modal.com/)
-
Beating Proprietary Models with a Quick Fine-Tune - Finetuning Quora Embeddings (https://modal.com/blog/fine-tuning-embeddings) (https://github.com/567-labs/fastllm/blob/main/applications/finetune-quora-embeddings/Readme.md)
-
Fine-tune an LLM in minutes (ft. Llama 2, CodeLlama, Mistral, etc.) (https://modal.com/docs/examples/llm-finetuning) (https://github.com/modal-labs/llm-finetuning)
- Model Explorer - a powerful graph visualization tool that helps one understand, debug, and optimize ML models (https://ai.google.dev/edge/model-explorer) (https://research.google/blog/model-explorer/)
- 🤗 The Large Language Model Training Handbook (https://github.com/huggingface/llm_training_handbook)
- 🤗 The Large Language Model Training Playbook (https://github.com/huggingface/large_language_model_training_playbook)
- 🤗 Dataset map method - how to pass argument to the function (https://discuss.huggingface.co/t/dataset-map-method-how-to-pass-argument-to-the-function/16274)
- 🤗 Quantization (https://huggingface.co/docs/transformers/quantization)
- 🤗 Generation with LLMs - Common pitfalls, etc (https://huggingface.co/docs/transformers/main/llm_tutorial)
- 🤗 Getting the most out of LLMs - Optimizing LLMs for Speed and Memory (https://huggingface.co/docs/transformers/main/llm_tutorial_optimization)
- 🤗 LLM prompting guide (https://huggingface.co/docs/transformers/main/tasks/prompting)
- 🤗 Templates for Chat Models (https://huggingface.co/docs/transformers/main/chat_templating)
- 🤗 Text generation strategies & Decoding strategies (https://huggingface.co/docs/transformers/main/generation_strategies)
- Efficient Training Techniques (https://huggingface.co/docs/transformers/perf_train_gpu_one)
- Multimodal Toolkit - Transformers with Tabular Data (https://github.com/georgian-io/Multimodal-Toolkit)
- LiGO - Learning to grow machine-learning models - New LiGO technique accelerates training of large machine-learning models (https://news.mit.edu/2023/new-technique-machine-learning-models-0322)
- Finetuning Large Language Models (https://magazine.sebastianraschka.com/p/finetuning-large-language-models)
- Adapters (https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/v1.10.0/core/adapters/intro.html) (https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/02_NeMo_Adapters.ipynb)
- P-tuning and prompt tuning (https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/nlp/nemo_megatron/prompt_learning.html) (https://github.com/NVIDIA/NeMo/blob/stable/tutorials/nlp/Multitask_Prompt_and_PTuning.ipynb)
- Difference between P-tuning and Prefix Tuning (https://www.reddit.com/r/MachineLearning/comments/14pkibg/comment/jqkdam8/)
-
- RLHF: Reinforcement Learning from Human Feedback (https://huyenchip.com/2023/05/02/rlhf.html)
- Building LLM applications for production (https://huyenchip.com/2023/04/11/llm-engineering.html)
- Instruction tuning datasets to train (text and multi-modal) chat-based LLMs (GPT-4, ChatGPT,LLaMA,Alpaca) (https://github.com/yaodongC/awesome-instruction-dataset)
- A Gentle Introduction to 8-bit Matrix Multiplication for transformers at scale using Hugging Face Transformers, Accelerate and bitsandbytes (https://huggingface.co/blog/hf-bitsandbytes-integration)
- Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA (https://huggingface.co/blog/4bit-transformers-bitsandbytes)
-
General notebook (https://colab.research.google.com/drive/1ge2F1QSK8Q7h0hn3YKuBCOAS0bK8E0wf?usp=sharing#scrollTo=OQdUx-aQScdR)
-
Training notebook (https://colab.research.google.com/drive/1VoYNfYDKcKRQRor98Zbf2-9VQTtGJ24k?usp=sharing#scrollTo=jq0nX33BmfaC)
- Making LLMs lighter with AutoGPTQ and transformers (https://huggingface.co/blog/gptq-integration) (https://arxiv.org/pdf/2210.17323.pdf)
- Introduction to Quantization cooked in 🤗 with 💗🧑🍳 (https://huggingface.co/blog/merve/quantization)
- LLM.int8() and Emergent Features (https://timdettmers.com/2022/08/17/llm-int8-and-emergent-features/)
- [Video] State of GPT - Learn about the training pipeline of GPT assistants (https://www.youtube.com/watch?v=bZQun8Y4L2A&t=1s)
- Google "We Have No Moat, And Neither Does OpenAI" (https://www.semianalysis.com/p/google-we-have-no-moat-and-neither)
- Emerging Architectures for LLM Applications (https://a16z.com/2023/06/20/emerging-architectures-for-llm-applications/)
- The Secret Sauce behind 100K context window in LLMs: all tricks in one place (https://blog.gopenai.com/how-to-speed-up-llms-and-use-100k-context-window-all-tricks-in-one-place-ffd40577b4c)
- All You Need to Know to Build Your First LLM App (https://towardsdatascience.com/all-you-need-to-know-to-build-your-first-llm-app-eb982c78ffac) $$
- Neural Networks: Zero to Hero - A course by Andrej Karpathy - Syllabus & links to videos (https://karpathy.ai/zero-to-hero.html)
-
[Video] Let's build GPT: from scratch, in code, spelled out. (https://www.youtube.com/watch?v=kCc8FmEb1nY)
- Decoding Strategies in Large Language Models (https://towardsdatascience.com/decoding-strategies-in-large-language-models-9733a8f70539) $$
- Interactively fine-tune Falcon-40B and other LLMs on Amazon SageMaker Studio notebooks using QLoRA (https://aws.amazon.com/blogs/machine-learning/interactively-fine-tune-falcon-40b-and-other-llms-on-amazon-sagemaker-studio-notebooks-using-qlora/) (https://github.com/aws-samples/amazon-sagemaker-generativeai/blob/main/studio-notebook-fine-tuning/falcon-40b-qlora-finetune-summarize.ipynb)
- [Codes] CVPR 2023 - Scaling PyTorch Model Training With Minimal Code Changes (https://github.com/rasbt/cvpr2023)
- [Codes] LLM-finetuning-scripts (https://github.com/rasbt/LLM-finetuning-scripts/)
- [Codes] LoRA (https://github.com/rasbt/low-rank-adaptation-blog) (https://lightning.ai/lightning-ai/studios/code-lora-from-scratch?utm_source=substack&utm_medium=email)
- [Codes] Gradient Accumulation (https://github.com/rasbt/gradient-accumulation-blog/)
- [Codes] Optimizing PyTorch Memory Usage (https://github.com/rasbt/pytorch-memory-optim/)
- Patterns for Building LLM-based Systems & Products (https://eugeneyan.com/writing/llm-patterns/)
- Fine-tuning Alpaca and LLaMA: Training on a Custom Dataset - for sentiment analysis, using 🤗 PEFT LoRA (https://www.mlexpert.io/machine-learning/tutorials/alpaca-fine-tuning) (https://colab.research.google.com/drive/1X85FLniXx_NyDsh_F_aphoIAy63DKQ7d?usp=sharing)
- Extended Guide: Instruction-tune Llama 2 - focus on creating the instruction dataset (https://www.philschmid.de/instruction-tune-llama-2)
- Fine-tuning LLMs - in simple notes - ROUGE, BLEU metrics - (https://teetracker.medium.com/fine-tuning-llms-9fe553a514d0)
- Fine-tune Llama 2 with DPO - Direct Preference Optimization (https://huggingface.co/blog/dpo-trl)
- Fine-Tuning Llama 2.0 with Single GPU Magic (https://ai.plainenglish.io/fine-tuning-llama2-0-with-qloras-single-gpu-magic-1b6a6679d436)
-
Understanding Llama2: KV Cache, Grouped Query Attention, Rotary Embedding and More (https://ai.plainenglish.io/understanding-llama2-kv-cache-grouped-query-attention-rotary-embedding-and-more-c17e5f49a6d7)
-
Mastering BERT Model: Building it from Scratch with Pytorch (https://medium.com/data-and-beyond/complete-guide-to-building-bert-model-from-sratch-3e6562228891)
-
Exploring Dolly 2.0: Fine Tuning Your Own ChatGPT-like Model (https://ai.plainenglish.io/exploring-dolly-2-0-a-guide-to-training-your-own-chatgpt-like-model-dd9b785ff1df)
-
GPT Model Behind the Scene: Exploring it from scratch with Pytorch (https://ai.plainenglish.io/creating-and-exploring-gpt-from-scratch-ffe84ac415a9)
- Fine-Tune Your Own Llama 2 Model in a Colab Notebook (https://towardsdatascience.com/fine-tune-your-own-llama-2-model-in-a-colab-notebook-df9823a04a32) $$ (https://colab.research.google.com/drive/1PEQyJO1-f6j0S_XJ8DV50NkpzasXkrzd?usp=sharing) (https://mlabonne.github.io/blog/posts/Fine_Tune_Your_Own_Llama_2_Model_in_a_Colab_Notebook.html)
- Fine tune Llama v2 models on Guanaco Dataset (https://gist.github.com/younesbelkada/9f7f75c94bdc1981c8ca5cc937d4a4da)
- Fine-tune Llama 2 with dolly-15k on a free Google Colab instance (https://colab.research.google.com/drive/134o_cXcMe_lsvl15ZE_4Y75Kstepsntu?usp=sharing)
- Fine-tune Llama 2 with SFT and DPO (https://medium.com/@anchen.li/fine-tune-llama-2-with-sft-and-dpo-8b57cf3ec69)
- Fine-Tuning LLaMA 2 Models using a single GPU, QLoRA and AI Notebooks (https://blog.ovhcloud.com/fine-tuning-llama-2-models-using-a-single-gpu-qlora-and-ai-notebooks/)
- Fine-Tuning Embedding Model with PEFT and LoRA (https://medium.com/@kelvin.lu.au/fine-tuning-embedding-model-with-peft-and-lora-3b6f08987c24)
- [Video] Jeremy Howard: A Hackers' Guide to Language Models (https://www.youtube.com/watch?v=jkrNMKz9pWU)
- 🤗 The Alignment Handbook - Robust recipes to align language models with human and AI preferences (https://github.com/huggingface/alignment-handbook/tree/main) (https://www.linkedin.com/posts/philipp-schmid-a6a2bb196_what-if-we-could-distill-the-alignment-from-activity-7123567109128806400-jgEg) (https://www.linkedin.com/posts/thom-wolf_ai-knowledgesharing-opensource-activity-7123588019839713280-EqhV) (https://arxiv.org/pdf/2310.16944.pdf)
- Efficient Fine-Tuning with LoRA: A Guide to Optimal Parameter Selection for Large Language Models (https://www.databricks.com/blog/efficient-fine-tuning-lora-guide-llms)
- LoRA — Intuitively and Exhaustively Explained (https://towardsdatascience.com/lora-intuitively-and-exhaustively-explained-e944a6bff46b) $$ (https://github.com/DanielWarfield1/MLWritingAndResearch/blob/main/LoRA.ipynb)
- Fine-Tuning LLMs: LoRA or Full-Parameter? An in-depth Analysis with Llama 2 (https://www.anyscale.com/blog/fine-tuning-llms-lora-or-full-parameter-an-in-depth-analysis-with-llama-2)
- Out-of-Domain Finetuning to Bootstrap Hallucination Detection (https://eugeneyan.com/writing/finetuning/) (https://github.com/eugeneyan/visualizing-finetunes)
- Faster debug and development with tiny models, tokenizers and datasets (https://github.com/stas00/ml-engineering/blob/master/transformers/make-tiny-models.md) (https://huggingface.co/stas/tiny-random-llama-2) (https://huggingface.co/stas/tiny-random-llama-2/blob/main/make_tiny_model.py)
- Mastering LLM Techniques: Inference Optimization (https://developer.nvidia.com/blog/mastering-llm-techniques-inference-optimization/)
- The Novice's LLM Training Guide (https://rentry.org/llm-training)
- Fast Llama 2 on CPUs With Sparse Fine-Tuning and DeepSparse (https://neuralmagic.com/blog/fast-llama-2-on-cpus-with-sparse-fine-tuning-and-deepsparse/)
- LLM Distillation Playbook (by Predibase) - Practical best practices for distilling large language models (https://github.com/predibase/llm_distillation_playbook)
- Preference Tuning LLMs with Direct Preference Optimization Methods - evaluation of Direct Preference Optimization (DPO), Identity Preference Optimisation (IPO) and Kahneman-Tversky Optimisation (KTO) (https://huggingface.co/blog/pref-tuning)
- Merge Large Language Models with mergekit (https://towardsdatascience.com/merge-large-language-models-with-mergekit-2118fb392b54)
- 🤗 How to Fine-Tune LLMs in 2024 with Hugging Face (https://www.philschmid.de/fine-tune-llms-in-2024-with-trl)
- 🤗 makeMoE: Implement a Sparse Mixture of Experts Language Model from Scratch (https://huggingface.co/blog/AviSoori1x/makemoe-from-scratch)
- Function Calling Datasets, Training and Inference (https://www.youtube.com/watch?v=hHn_cV5WUDI)
- [Video] Fine tuning Optimizations - DoRA, NEFT, LoRA+, Unsloth (https://www.youtube.com/watch?v=ae2lbmtTY5A)
- DoRA Demystified: Visualising Weight-Decomposed Low-Rank Adaptation (https://shreyassk.substack.com/p/visualising-dora-weight-decomposed) (https://github.com/shreyassks/DoRA/)
- GaLore: Advancing Large Model Training on Consumer-grade Hardware (https://huggingface.co/blog/galore) (https://github.com/jiaweizzhao/galore)
-
Memory-efficient LLM Training with GaLore (https://medium.com/@geronimo7/llm-training-on-consumer-gpus-with-galore-d25075143cfb) (https://github.com/geronimi73/3090_shorts/blob/main/nb_galore_llama2-7b.ipynb)
- An Overview of the LoRA Family (https://towardsdatascience.com/an-overview-of-the-lora-family-515d81134725)
- Efficiently fine-tune Llama 3 with PyTorch FSDP and Q-Lora (https://www.philschmid.de/fsdp-qlora-llama3) (https://github.com/philschmid/deep-learning-pytorch-huggingface/blob/main/training/scripts/run_fsdp_qlora.py)
- A Guide on 12 Tuning Strategies for Production-Ready RAG Applications (https://towardsdatascience.com/a-guide-on-12-tuning-strategies-for-production-ready-rag-applications-7ca646833439#341d)
- Advanced RAG Techniques: an Illustrated Overview (https://pub.towardsai.net/advanced-rag-techniques-an-illustrated-overview-04d193d8fec6)
- Advanced RAG Techniques (https://www.pinecone.io/learn/advanced-rag-techniques/)
- A Cheat Sheet and Some Recipes For Building Advanced RAG (https://blog.llamaindex.ai/a-cheat-sheet-and-some-recipes-for-building-advanced-rag-803a9d94c41b)
-
https://d3ddy8balm3goa.cloudfront.net/llamaindex/rag-cheat-sheet-final.svg
- RAG cheatsheet (https://miro.com/app/board/uXjVNvklNmc=/)
- From paper to prod! A guide to improving your semantic search with HyDE (https://aimodels.substack.com/p/from-paper-to-prod-a-guide-to-improving)
- Improving the Semantic Search Tool (https://puddles-of-water.medium.com/improving-the-semantic-search-tool-ef0442f7e972)
- Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval (https://huggingface.co/blog/embedding-quantization)
- 100x Faster — Scaling Your RAG App for Billions of Embeddings - Computing Cosine Similarity in parallel - using Chunkdot (https://medium.com/gitconnected/100x-faster-scaling-your-rag-app-for-billions-of-embeddings-ded34fccd16a) (https://github.com/rragundez/chunkdot/)
- Advanced Retrieval for AI with Chroma (https://www.deeplearning.ai/short-courses/advanced-retrieval-for-ai/)
- Chat with your code: RAG with Weaviate and LlamaIndex (https://lightning.ai/weaviate/studios/chat-with-your-code-rag-with-weaviate-and-llamaindex)
- The 4 Advanced RAG Algorithms You Must Know to Implement (https://medium.com/decodingml/the-4-advanced-rag-algorithms-you-must-know-to-implement-5d0c7f1199d2)
- 📖 Lightning Fabric Documentation (https://lightning.ai/docs/fabric/stable/)
- [Codes] Build Your Own Trainer (https://github.com/Lightning-AI/lightning/tree/master/examples/fabric/build_your_own_trainer)
- ✏️ Ahead of AI (by Sebastian Raschka) (https://magazine.sebastianraschka.com/archive) (https://sebastianraschka.com/blog/)
- [Course] Deep Learning Fundamentals (https://lightning.ai/pages/courses/deep-learning-fundamentals/) (https://github.com/Lightning-AI/dl-fundamentals)
- Accelerate PyTorch Code with Fabric (https://lightning.ai/pages/blog/accelerate-pytorch-code-with-fabric/)
- How to Speed Up PyTorch Model Training (https://lightning.ai/pages/community/tutorial/how-to-speed-up-pytorch-model-training/)
- Finetuning LLMs on a Single GPU Using Gradient Accumulation (https://lightning.ai/pages/blog/gradient-accumulation/)
- Accelerating LLaMA with Fabric: A Comprehensive Guide to Training and Fine-Tuning LLaMA (https://lightning.ai/pages/community/tutorial/accelerating-llama-with-fabric-a-comprehensive-guide-to-training-and-fine-tuning-llama/)
- Understanding Parameter-Efficient Finetuning of Large Language Models: From Prefix Tuning to LLaMA-Adapters (https://lightning.ai/pages/community/article/understanding-llama-adapters/)
- How To Finetune GPT Like Large Language Models on a Custom Dataset (https://lightning.ai/pages/blog/how-to-finetune-gpt-like-large-language-models-on-a-custom-dataset/)
- Code LoRA From Scratch (https://lightning.ai/lightning-ai/studios/code-lora-from-scratch)
- Parameter-Efficient LLM Finetuning With Low-Rank Adaptation (LoRA) (https://lightning.ai/pages/community/tutorial/lora-llm/)
- Efficient Initialization of Large Models (https://lightning.ai/pages/community/efficient-initialization-of-large-models/)
- Accelerating Large Language Models with Mixed-Precision Techniques (https://lightning.ai/pages/community/tutorial/accelerating-large-language-models-with-mixed-precision-techniques/)
- Faster PyTorch Training by Reducing Peak Memory (combining backward pass + optimizer step) (https://lightning.ai/pages/community/tutorial/faster-pytorch-training-by-reducing-peak-memory/)
- Falcon – A guide to finetune and inference (https://lightning.ai/pages/blog/falcon-a-guide-to-finetune-and-inference/)
- Finetuning Falcon LLMs More Efficiently With LoRA and Adapters (https://lightning.ai/pages/community/finetuning-falcon-efficiently/)
- The Falcon has landed in the Hugging Face ecosystem (https://huggingface.co/blog/falcon) (https://colab.research.google.com/drive/1BiQiw31DT7-cDp1-0ySXvvhzqomTdI-o?usp=sharing)
- Improve LLMs With Proxy-Tuning (https://lightning.ai/lightning-ai/studios/improve-llms-with-proxy-tuning)
- 🤗 Open LLM Leaderboard (https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
- 🤗 Massive Text Embedding Benchmark (MTEB) Leaderboard (https://huggingface.co/spaces/mteb/leaderboard)
- Hughes Hallucination Evaluation Model (HHEM) leaderboard (https://huggingface.co/spaces/vectara/leaderboard) (https://github.com/vectara/hallucination-leaderboard)
- FastChat - platform for training, serving, and evaluating large language model based chatbots (https://github.com/lm-sys/FastChat)
- LLM Evaluation Metrics: Everything You Need for LLM Evaluation (https://www.confident-ai.com/blog/llm-evaluation-metrics-everything-you-need-for-llm-evaluation)
- DeepEval - open-source LLM evaluation framework specialized for unit testing LLM outputs (https://github.com/confident-ai/deepeval)
- Language Model Evaluation Harness - based on tasks (https://github.com/EleutherAI/lm-evaluation-harness) (https://github.com/EleutherAI/lm-evaluation-harness/blob/main/docs/interface.md)
- LLM Task-Specific Evals that Do & Don't Work (https://eugeneyan.com/writing/evals/)
- 7 Basic NLP Models to Empower Your ML Application (https://zilliz.com/learn/7-nlp-models)
- 7 models on HuggingFace you probably didn’t know existed (https://towardsdatascience.com/7-models-on-huggingface-you-probably-didnt-knew-existed-f3d079a4fd7c)
- ChatGPT, GenerativeAI and LLMs Timeline (https://github.com/hollobit/GenAI_LLM_timeline)
- AI / ML / LLM / Transformer Models Timeline and List (https://ai.v-gar.de/ml/transformer/timeline/index.html)
- Comprehensive LLM model zoo (https://crfm.stanford.edu/ecosystem-graphs/index.html?mode=table)
- Totally Open Chatgpt (https://github.com/nichtdax/awesome-totally-open-chatgpt)
- Open LLMs (https://github.com/eugeneyan/open-llms)
- GPT4All (https://github.com/nomic-ai/gpt4all)
- TII open-sourced Falcon LLM (https://huggingface.co/tiiuae)
- Question generator with context (https://huggingface.co/voidful/bart-eqg-question-generator)
- T5 One Line Summary (https://huggingface.co/snrspeaks/t5-one-line-summary)
- T5-base fine-tuned on SQuAD for Question Generation by just prepending the answer to the context (https://huggingface.co/mrm8488/t5-base-finetuned-question-generation-ap)
- Parrot_paraphraser_on_T5 (https://huggingface.co/prithivida/parrot_paraphraser_on_T5)
- T0pp (https://huggingface.co/bigscience/T0pp) (https://huggingface.co/GroNLP/T0pp-sharded)
- Persimmon-8B: The best fully permissively-licensed model in the 8B class (https://www.adept.ai/blog/persimmon-8b)
- Mistral Transformer (https://github.com/mistralai/mistral-src) (https://mistral.ai/news/announcing-mistral-7b/)
- MiniGPT-v2: Large Language Model as a Unified Interface for Vision-Language Multi-task Learning (https://minigpt-v2.github.io/)
- 8 Top Open-Source LLMs for 2024 and Their Uses (https://www.datacamp.com/blog/top-open-source-llms)
- 01-ai/Yi-34B (https://huggingface.co/01-ai/Yi-34B)
- OLMo: Open Language Model (https://github.com/allenai/OLMo/tree/main) (https://github.com/allenai/OLMo/blob/main/scripts/train.py)
- Genstruct-7B, an instruction-generation model (https://huggingface.co/NousResearch/Genstruct-7B) (https://huggingface.co/NousResearch/Genstruct-7B/blob/main/notebook.ipynb)
- Grok-1, a 314 billion parameter Mixture-of-Experts model (https://github.com/xai-org/grok-1)
- Qwen1.5-MoE: Matching 7B Model Performance with 1/3 Activated Parameters (https://qwenlm.github.io/blog/qwen-moe/)
- Tiny but mighty: The Phi-3 small language models with big potential (https://news.microsoft.com/source/features/ai/the-phi-3-small-language-models-with-big-potential/)
-
TinyStories: How Small Can Language Models Be and Still Speak Coherent English? (https://arxiv.org/pdf/2305.07759)
- OpenELM - An Efficient Language Model Family with Open-source Training and Inference Framework - using layer-wise scaling strategy (https://arxiv.org/abs/2404.14619) (https://huggingface.co/apple/OpenELM) (https://github.com/apple/corenet)
- 7 Ways To Speed Up Inference of Your Hosted LLMs (https://betterprogramming.pub/speed-up-llm-inference-83653aa24c47)
- vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention (https://github.com/vllm-project/vllm) (https://vllm.ai/)
- 🤗 TGI: Text Generation Inference - Fast optimized inference for LLMs (https://github.com/huggingface/text-generation-inference)
- LMDeploy: a toolkit for compressing, deploying, and serving LLM (https://github.com/InternLM/lmdeploy)
- OpenVINO: an open-source toolkit for optimizing and deploying AI inference (https://github.com/openvinotoolkit) (https://docs.openvino.ai/2023.0/home.html)
- How continuous batching enables 23x throughput in LLM inference while reducing p50 latency (https://www.anyscale.com/blog/continuous-batching-llm-inference)
- Squeeze more out of your GPU for LLM inference—a tutorial on Accelerate & DeepSpeed (https://preemo.medium.com/squeeze-more-out-of-your-gpu-for-llm-inference-a-tutorial-on-accelerate-deepspeed-610fce3025fd)
- Performance bottlenecks in deploying LLMs—a primer for ML researchers (https://preemo.medium.com/performance-bottlenecks-in-deploying-llms-a-primer-for-ml-researchers-c2b51c2084a8)
- Inference using the pre-trained Alpaca-LoRA (https://www.mlexpert.io/machine-learning/tutorials/alpaca-and-llama-inference) (https://colab.research.google.com/drive/15VstUxU48CT3mRudFrj3FIv6Z4cIXnon?usp=sharing)
- Optimizing your LLM in production (https://huggingface.co/blog/optimize-llm)
- StreamingLLM: Efficient Streaming Language Models with Attention Sinks (https://github.com/mit-han-lab/streaming-llm) (https://arxiv.org/pdf/2309.17453.pdf)
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters (https://github.com/s-lora/s-lora)
-
Recipe for Serving Thousands of Concurrent LoRA Adapters (https://lmsys.org/blog/2023-11-15-slora/)
- DeepSparse by Neural Magic - Sparsity-aware deep learning inference runtime for CPUs (https://github.com/neuralmagic/deepsparse/tree/main)
- SparseML by Neural Magic - an open-source model optimization toolkit that enables you to create inference-optimized sparse models using pruning, quantization, and distillation algorithms (https://github.com/neuralmagic/sparseml)
- Marlin - Mixed Auto-Regressive Linear kernel, an extremely optimized FP16xINT4 matmul kernel aimed at LLM inference (https://github.com/IST-DASLab/marlin)
- LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning (https://github.com/datamllab/LongLM) (https://www.reddit.com/r/LocalLLaMA/comments/18x8g6c/llm_maybe_longlm_selfextend_llm_context_window/)
- Flash-Decoding for long-context inference (https://pytorch.org/blog/flash-decoding/)
- 10 Ways To Run LLMs Locally And Which One Works Best For You (https://matilabs.ai/2024/02/07/run-llms-locally/)
- Towards 100x Speedup: Full Stack Transformer Inference Optimization (https://yaofu.notion.site/Towards-100x-Speedup-Full-Stack-Transformer-Inference-Optimization-43124c3688e14cffaf2f1d6cbdf26c6c)
- Deploy Deep Learning Models at Scale using NVIDIA Triton Inference Server (https://github.com/decodingml/articles-code/tree/main/articles/computer_vision/deploy_deep_learning_at_scale_nvidia_triton_server)
- LMDeploy - a toolkit for compressing, deploying, and serving LLM (https://github.com/InternLM/lmdeploy)
- LLM Inference Series: 5. Dissecting model performance (https://medium.com/@plienhar/llm-inference-series-5-dissecting-model-performance-6144aa93168f)
- How to compute LLM embeddings 3X faster with model quantization - with ONNX model quantization / ONNX transformer optimization (https://medium.com/nixiesearch/how-to-compute-llm-embeddings-3x-faster-with-model-quantization-25523d9b4ce5)
- A Hitchhiker’s Guide to Speculative Decoding (https://pytorch.org/blog/hitchhikers-guide-speculative-decoding/)
- Curating Trillion-Token Datasets: Introducing NVIDIA NeMo Data Curator (https://developer.nvidia.com/blog/curating-trillion-token-datasets-introducing-nemo-data-curator/)
- ftfy: fixes text for you (https://github.com/rspeer/python-ftfy)
- Cosmopedia: how to create large-scale synthetic data for pre-training (https://huggingface.co/blog/cosmopedia) (https://github.com/huggingface/cosmopedia)
- DataTrove - a library to process, filter and deduplicate text data at a very large scale (https://github.com/huggingface/datatrove/)
- Guidance - control how LLM output is structured (https://github.com/guidance-ai/guidance)
- Large-scale Near-deduplication Behind BigCode (https://huggingface.co/blog/dedup)
- Dolma Toolkit - curation of large datasets for (pre)-training ML models (https://github.com/allenai/dolma)
- DBPedia (https://www.dbpedia.org/resources/individual/) (http://downloads.dbpedia.org/wiki-archive/dbpedia-version-2016-04.html) (http://downloads.dbpedia.org/2016-04/core/)
- Common Crawl (https://commoncrawl.org/the-data/get-started/)
- c4 - A colossal, cleaned version of Common Crawl's web crawl corpus (https://tensorflow.org/datasets/catalog/c4)
- c4 processed version with five variants of the data: en, en.noclean, en.noblocklist, realnewslike, and multilingual (mC4). (https://huggingface.co/datasets/allenai/c4)
- RedPajama-V2: An open dataset with 30 trillion tokens for training large language models (https://www.together.ai/blog/redpajama-data-v2) (https://huggingface.co/datasets/togethercomputer/RedPajama-Data-V2)
- SlimPajama-627B - Extensively deduplicated, multi-corpora, open-source dataset for training LLM (https://www.cerebras.net/blog/slimpajama-a-627b-token-cleaned-and-deduplicated-version-of-redpajama) (https://huggingface.co/datasets/cerebras/SlimPajama-627B)
- Falcon RefinedWeb - An English large-scale dataset (5 trillion tokens ) for the pretraining of LLM, built through stringent filtering and extensive deduplication of CommonCrawl (https://huggingface.co/datasets/tiiuae/falcon-refinedweb) (https://arxiv.org/abs/2306.01116)
- Instruction tuning datasets to train (text and multi-modal) chat-based LLMs (GPT-4, ChatGPT, LLaMA, Alpaca) (https://github.com/yaodongC/awesome-instruction-dataset)
- Python-Code-23k-ShareGPT (https://huggingface.co/datasets/ajibawa-2023/Python-Code-23k-ShareGPT)
- UltraFeedback Binarized - A pre-processed version of the UltraFeedback dataset and was used to train Zephyr-7Β-β (https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized)
- [Blog post] FineWeb: decanting the web for the finest text data at scale (https://huggingface.co/spaces/HuggingFaceFW/blogpost-fineweb-v1)
-
FineWeb: 15T tokens (44TB disk space) of cleaned and deduplicated english web data from CommonCrawl, for LLM pretraining (https://huggingface.co/datasets/HuggingFaceFW/fineweb)
-
FineWeb-Edu: 1.3T tokens of educational web pages filtered from 🍷 FineWeb dataset (https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu)
-
FineWeb-Edu-score-2: 5.4T tokens of educational web pages filtered from 🍷 FineWeb dataset (https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu-score-2)
- Dolma - Used to train OLMo on 3 trillion tokens from a diverse mix of web content, academic publications, code, books, and encyclopedic materials, with quality filtering, fuzzy deduplication (https://blog.allenai.org/dolma-3-trillion-tokens-open-llm-corpus-9a0ff4b8da64) (https://huggingface.co/datasets/allenai/dolma)
- Beyond Classification With Transformers and Hugging Face (https://towardsdatascience.com/beyond-classification-with-transformers-and-hugging-face-d38c75f574fb)
- Faster Text Classification with Naive Bayes and GPUs (https://developer.nvidia.com/blog/faster-text-classification-with-naive-bayes-and-gpus)
- Classifying Multimodal Data using Transformers (https://github.com/dsaidgovsg/multimodal-learning-hands-on-tutorial)
- Chris McCormick
-
[Video] BERT Document Classification Tutorial with Code - including Semantic Similarity Search (https://www.youtube.com/watch?v=_eSGWNqKeeY)
-
Combining Categorical and Numerical Features with Text in BERT (https://mccormickml.com/2021/06/29/combining-categorical-numerical-features-with-bert/)
-
Smart Batching Tutorial - Speed Up BERT Training (https://mccormickml.com/2020/07/29/smart-batching-tutorial/)
- [Discussion] How to use Bert for long text classification? (https://stackoverflow.com/questions/58636587/how-to-use-bert-for-long-text-classification/63413589#63413589)
- Sentiment Analysis with BERT and Transformers by Hugging Face using PyTorch and Python (https://curiousily.com/posts/sentiment-analysis-with-bert-and-hugging-face-using-pytorch-and-python/)
- Custom Named Entity Recognition with BERT (https://towardsdatascience.com/custom-named-entity-recognition-with-bert-cf1fd4510804)
- The Guide to Multi-Tasking with the T5 Transformer (https://towardsdatascience.com/the-guide-to-multi-tasking-with-the-t5-transformer-90c70a08837b)
- A Full Guide to Finetuning T5 for Text2Text and Building a Demo with Streamlit (https://medium.com/nlplanet/a-full-guide-to-finetuning-t5-for-text2text-and-building-a-demo-with-streamlit-c72009631887) (https://colab.research.google.com/drive/1RFBIkTZEqbRt0jxpTHgRudYJBZTD3Szn?usp=sharing)
- Building a Knowledge Base from Texts: a Full Practical Example (https://medium.com/nlplanet/building-a-knowledge-base-from-texts-a-full-practical-example-8dbbffb912fa)
- Building a Personal Assistant from Scratch - intent classification, speech-to-text, and text-to-speech (https://medium.com/nlplanet/building-a-personal-assistant-from-scratch-db0814e62d34)
- How I Turned My Company’s Docs into a Searchable Database with OpenAI (https://towardsdatascience.com/how-i-turned-my-companys-docs-into-a-searchable-database-with-openai-4f2d34bd8736)
- How I Turned ChatGPT into an SQL-Like Translator for Image and Video Datasets (https://towardsdatascience.com/how-i-turned-chatgpt-into-an-sql-like-translator-for-image-and-video-datasets-7b22b318400a)
- 10 Exciting Project Ideas Using Large Language Models (LLMs) for Your Portfolio (https://towardsdatascience.com/10-exciting-project-ideas-using-large-language-models-llms-for-your-portfolio-970b7ab4cf9e) $$
- Build a Telegram chatbot with any AI model under the hood (web scraping summarizer bot) (https://medium.com/@galperovich/build-a-telegram-chatbot-with-any-ai-model-under-the-hood-62f9a8675d81) (https://github.com/galinaalperovich/ai_summary_tg_bot)
- Put ChatGPT right into your messenger: build a Telegram bot with the new official OpenAI API (https://blog.gopenai.com/put-chatgpt-right-into-your-messenger-build-a-telegram-bot-with-the-new-official-openai-api-84f7c005de7f) (https://github.com/galinaalperovich/chatgpt-api-tg-bot)
- LangChain + Streamlit🔥+ Llama 🦙: Bringing Conversational AI to Your Local Machine 🤯 (https://ai.plainenglish.io/%EF%B8%8F-langchain-streamlit-llama-bringing-conversational-ai-to-your-local-machine-a1736252b172)
- Zero to One: A Guide to Building a First PDF Chatbot with LangChain & LlamaIndex — Part 1 (https://medium.com/how-ai-built-this/zero-to-one-a-guide-to-building-a-first-pdf-chatbot-with-langchain-llamaindex-part-1-7d0e9c0d62f)
- Choosing the Right Embedding Model: A Guide for LLM Applications (https://medium.com/@ryanntk/choosing-the-right-embedding-model-a-guide-for-llm-applications-7a60180d28e3)
- Financial Document Classification with LayoutLMv3 - using OCR on document images (https://www.mlexpert.io/machine-learning/tutorials/document-classification-with-layoutlmv3) (https://colab.research.google.com/drive/1I0Qyajp_DFzKvQfUwwRc9p6fs6NfI-Kx?usp=sharing)
- Making a web app generator with open ML models (https://huggingface.co/blog/text-to-webapp)
- Topic Modeling with Llama 2 (https://maartengrootendorst.substack.com/p/topic-modeling-with-llama-2) (https://towardsdatascience.com/topic-modeling-with-llama-2-85177d01e174) $$ (https://colab.research.google.com/drive/1QCERSMUjqGetGGujdrvv_6_EeoIcd_9M?usp=sharing)
-
https://maartengr.github.io/BERTopic/algorithm/algorithm.html
-
https://maartengr.github.io/BERTopic/getting_started/best_practices/best_practices.html
- Mastering PDFs: Extracting Sections, Headings, Paragraphs, and Tables with Cutting-Edge Parser (https://blog.llamaindex.ai/mastering-pdfs-extracting-sections-headings-paragraphs-and-tables-with-cutting-edge-parser-faea18870125)
-
LayoutPDFReader (https://github.com/nlmatics/llmsherpa#layoutpdfreader)
- Multimodal Retrieval with Text Embedding and CLIP Image Embedding for Backyard Birds (https://github.com/wenqiglantz/multi_modal_retrieval_backyard_birds)
- LlamaIndex Chat (https://github.com/run-llama/chat-llamaindex)
- LoRA for semantic similarity tasks (https://huggingface.co/docs/peft/task_guides/semantic-similarity-lora)
- Forging a Personal Chatbot with OpenAI API, Chroma DB, HuggingFace Spaces, and Gradio 🔥 (https://mlops.community/forging-a-personal-chatbot-with-openai-api-chroma-db-huggingface-spaces-and-gradio-%f0%9f%94%a5/)
- How to Convert Any Text Into a Graph of Concepts (https://towardsdatascience.com/how-to-convert-any-text-into-a-graph-of-concepts-110844f22a1a) (https://github.com/rahulnyk/knowledge_graph)
-
Text to Knowledge Graph Made Easy with Graph Maker (https://towardsdatascience.com/text-to-knowledge-graph-made-easy-with-graph-maker-f3f890c0dbe8)
- Embed English Wikipedia under 5 dollars (https://lightning.ai/lightning-ai/studios/embed-english-wikipedia-under-5-dollars~01hg0zg8fyybp7p1sma6g9dkzm)
- Fine-Tuning Mistral 7b in Google Colab with QLoRA (complete guide) (https://medium.com/@codersama/fine-tuning-mistral-7b-in-google-colab-with-qlora-complete-guide-60e12d437cca)
- Hands-on LLMs Course - Learn to Train and Deploy a Real-Time Financial Advisor (https://github.com/iusztinpaul/hands-on-llms)
- LLM Twin Course: Building Your Production-Ready AI Replica (https://github.com/decodingml/llm-twin-course)
- The Full Stack 7-Steps MLOps Framework (https://github.com/iusztinpaul/energy-forecasting)
- Building DoorDash’s Product Knowledge Graph with Large Language Models (https://doordash.engineering/2024/04/23/building-doordashs-product-knowledge-graph-with-large-language-models/)
- Musings on building a Generative AI product - at LinkedIn (https://www.linkedin.com/blog/engineering/generative-ai/musings-on-building-a-generative-ai-product)
- How We Finetuned a Large Language Model to Search Patents & Generate New Patents (https://www.activeloop.ai/resources/how-we-finetuned-a-large-language-model-to-search-patents-generate-new-patents/)
- Structured LLM Output and Function Calling with Guidance - and Tool Use (https://lightning.ai/lightning-ai/studios/structured-llm-output-and-function-calling-with-guidance)
- Danswer: OpenSource Enterprise Question-Answering tool (https://github.com/danswer-ai/danswer) (https://docs.danswer.dev/introduction)
- DocsGPT: GPT-powered chat for documentation (https://docsgpt.arc53.com/) (https://github.com/arc53/DocsGPT)
-
Host a Llama 2 API on GPU for Free (https://medium.com/@yuhongsun96/host-a-llama-2-api-on-gpu-for-free-a5311463c183)
-
How to Augment LLMs with Private Data (https://medium.com/@yuhongsun96/how-to-augment-llms-with-private-data-29349bd8ae9f)
- How to build an AI that can answer questions about your website (https://platform.openai.com/docs/tutorials/web-qa-embeddings)
- openai-cookbook
-
-
- Fine-Tune Transformer Models For Question Answering On Custom Data (https://towardsdatascience.com/fine-tune-transformer-models-for-question-answering-on-custom-data-513eaac37a80)
- Revolutionise Your Q&A Bot with GPT-J: The Open-Source Game Changer as a Replacement for GPT-3 (https://medium.com/@maliahrajan/revolutionise-your-q-a-bot-with-gpt-j-the-open-source-game-changer-as-a-replacement-for-gpt-3-216bc4362b53)
- How to build a Q&A web application using Python and Anvil (https://www.section.io/engineering-education/building-a-qa-web-application/)
- Build a Deep Q&A Web App with Transformers and Anvil | Python Deep Learning App (https://www.youtube.com/watch?v=G1uGSkANZjQ) (https://github.com/nicknochnack/Q-A-Anvil-App/blob/main/Anvil-Tutorial.ipynb)
- Open Source Generative AI in Question-Answering (NLP) using Python (https://www.youtube.com/watch?v=L8U-pm-vZ4c) (https://docs.pinecone.io/docs/abstractive-question-answering)
- Build a Question Answering Engine (Towhee & Gradio Chatbot) (https://github.com/towhee-io/examples/blob/main/nlp/question_answering/1_build_question_answering_engine.ipynb)
- Question Generation using 🤗transformers (https://github.com/patil-suraj/question_generation) (https://colab.research.google.com/gist/nrjvarshney/39ed6c80e2fe293b9e7eca5bc3a45b7d/quiz.ipynb) (https://huggingface.co/mrm8488/t5-base-finetuned-question-generation-ap)
- Running Llama 2 on CPU Inference Locally for Document Q&A (https://towardsdatascience.com/running-llama-2-on-cpu-inference-for-document-q-a-3d636037a3d8) $$ (https://github.com/kennethleungty/Llama-2-Open-Source-LLM-CPU-Inference)
- Using LLaMA 2.0, FAISS and LangChain for Question-Answering on Your Own Data (https://medium.com/@murtuza753/using-llama-2-0-faiss-and-langchain-for-question-answering-on-your-own-data-682241488476)
- How to Fine-tune Llama 2 with LoRA for Question Answering: A Guide for Practitioners (https://deci.ai/blog/fine-tune-llama-2-with-lora-for-question-answering/)
- Generative AI Lifecycle Patterns (https://dr-arsanjani.medium.com/the-generative-ai-lifecycle-1b0c7d9463ec)
- Why does the falcon QLoRA tutorial code use eos_token as pad_token? - use TemplateProcessing (https://discuss.huggingface.co/t/why-does-the-falcon-qlora-tutorial-code-use-eos-token-as-pad-token/45954/14?u=brando)
- Pad and eos distinction. (https://chat.openai.com/share/ebb9a9a2-71d3-4c97-a727-b6042494b9a9)
- LLaMA FastTokenizer does not add eos_token_id at the end. #22794 (huggingface/transformers#22794)
- data_collator.py (https://github.com/huggingface/transformers/blob/main/src/transformers/data/data_collator.py#L747)
- https://huggingface.co/docs/transformers/main/llm_tutorial#wrong-padding-side
- https://huggingface.co/docs/transformers/main/model_doc/llama2#resources
- Challenges in Stop Generation within Llama 2 (https://towardsdatascience.com/challenges-in-stop-generation-within-llama-2-25f5fea8dea2)
- Padding Large Language Models — Examples with Llama 2 (https://towardsdatascience.com/padding-large-language-models-examples-with-llama-2-199fb10df8ff) $$
- [SFTTrainer] Fix non packed dataset #444 - Example of formatting_func on alpaca dataset (huggingface/trl#444)
- Merging QLoRA weights with quantized model (https://gist.github.com/ChrisHayduk/1a53463331f52dca205e55982baf9930)
- LoRA Adapters: When a Naive Merge Leads to Poor Performance (https://kaitchup.substack.com/p/lora-adapters-when-a-naive-merge)
- FuseChat: Knowledge Fusion of Chat Models (https://github.com/fanqiwan/FuseLLM/tree/main/FuseChat)
- Enprompt 360 - AI Prompts Generator (https://www.kickstarter.com/projects/enprompt360/enprompt-360)
- Awesome ChatGPT Prompts (https://prompts.chat/)
- Prompt Engineering Guide (https://www.promptingguide.ai/techniques)
- The Power of Prompt Engineering: Building Your Own Personal Assistant (https://ai.plainenglish.io/the-power-of-prompt-engineering-personalizing-your-ai-model-5a1b9671b8c5)
- What I Learned Pushing Prompt Engineering to the Limit (https://towardsdatascience.com/what-i-learned-pushing-prompt-engineering-to-the-limit-c40f0740641f)
- Fixing Hallucinations in LLMs (https://betterprogramming.pub/fixing-hallucinations-in-llms-9ff0fd438e33)
- New ChatGPT Prompt Engineering Technique: Program Simulation (https://towardsdatascience.com/new-chatgpt-prompt-engineering-technique-program-simulation-56f49746aa7b)
- AutoGPT Agent Custom Instruction (https://shard-tsunami-ffe.notion.site/AutoGPT-Agent-Custom-Instruction-9826d664c53e4f50a5f814378c19a89d)
- Practitioners guide to fine-tune LLMs for domain-specific use case (https://cismography.medium.com/practitioners-guide-to-fine-tune-llms-for-domain-specific-use-case-part-1-4561714d874f)
- Practical insights while fine-tuning LLMs for domain-specific use cases and best practices (https://cismography.medium.com/practical-insights-while-fine-tuning-llms-for-domain-specific-use-cases-and-best-practices-aa986c799777)
- A New Prompt Engineering Technique Has Been Introduced Called Step-Back Prompting (https://cobusgreyling.medium.com/a-new-prompt-engineering-technique-has-been-introduced-called-step-back-prompting-b00e8954cacb)
- The LangChain Implementation Of DeepMind’s Step-Back Prompting (https://cobusgreyling.medium.com/the-langchain-implementation-of-deepminds-step-back-prompting-9d698cf3e0c2)
- Open AI - Prompt engineering guide (https://platform.openai.com/docs/guides/prompt-engineering)
- Inside the Leaked System Prompts of GPT-4, Gemini 1.5, Claude 3, and More (https://medium.com/gitconnected/inside-the-leaked-system-prompts-of-gpt-4-gemini-1-5-claude-3-and-more-4ecb3d22b447?sk=7e053318c47b260ee482a5c8b319dd83)
-
https://gist.github.com/kennethleungty/74c8f1ad0c39ca006fddea5da449c390 / https://gist.github.com/kennethleungty/00b5a5d809fdda94eafe5d49ccff7729 / https://gist.github.com/kennethleungty/80ceeba091d7c777abe861ef46558363 / https://gist.github.com/kennethleungty/587693681583da71f90d2da28e733ec3
- DSPy - a framework for algorithmically optimizing LM prompts and weights (https://github.com/stanfordnlp/dspy)
-
Your Language Model Deserves Better Prompting (https://weaviate.io/blog/dspy-optimizers) (https://github.com/weaviate/recipes/blob/main/integrations/weights_and_biases/wandb_logging_RAG_dspy_cohere.ipynb)
- Prompting Fundamentals and How to Apply them Effectively (https://eugeneyan.com/writing/prompting/)
- AutoGPT: the heart of the open-source agent ecosystem (https://github.com/Significant-Gravitas/AutoGPT)
- The Official AutoGPT Forge Tutorial Series (https://aiedge.medium.com/autogpt-forge-e3de53cc58ec)
- AutoGPT Tutorial: Creating an Agent Powered Research Assistant with Auto-GPT-Forge (https://lablab.ai/t/autogpt-tutorial-creating-a-research-assistant-with-auto-gpt-forge)
- Decoding Auto-GPT (https://maartengrootendorst.substack.com/p/decoding-auto-gpt)
- How Do Language Models put Attention Weights over Long Context? (https://yaofu.notion.site/How-Do-Language-Models-put-Attention-Weights-over-Long-Context-10250219d5ce42e8b465087c383a034e) (https://github.com/FranxYao/Long-Context-Data-Engineering)
- AI Watermarking 101: Tools and Techniques (https://huggingface.co/blog/watermarking)
- GGUF, the long way around (https://vickiboykis.com/2024/02/28/gguf-the-long-way-around/)
- A Comprehensive Guide to Modeling Techniques in Mixed-Integer Linear Programming - Convert ideas into mathematical expressions to solve operations research problems (https://towardsdatascience.com/a-comprehensive-guide-to-modeling-techniques-in-mixed-integer-linear-programming-3e96cc1bc03d)
- Mastering ML Configurations by leveraging OmegaConf and Hydra (https://decodingml.substack.com/p/mastering-ml-configurations-by-leveraging)
- UltraChat - example training script with Accelerator (https://github.com/thunlp/UltraChat/blob/main/train/train_legacy/train.py)
- Using LESS Data to Tune Models (https://www.cs.princeton.edu/~smalladi/blog/2024/04/04/dataselection/)
- Techniques for training large neural networks - Data parallelism, Pipeline parallelism, Tensor parallelism, Mixture-of-Experts (MoE), and other memory saving designs (https://openai.com/research/techniques-for-training-large-neural-networks)
-
Large Scale Transformer model training with Tensor Parallel (TP) (https://pytorch.org/tutorials/intermediate/TP_tutorial.html)
- Best Embedding Model — OpenAI / Cohere / Google / E5 / BGE - An In-depth Comparison of Multilingual Embedding Models (https://medium.com/@lars.chr.wiik/best-embedding-model-openai-cohere-google-e5-bge-931bfa1962dc) topic dataset (https://github.com/LarsChrWiik/lars_datasets/tree/main/topics_dataset_50)
- Training and Finetuning Embedding Models with Sentence Transformers v3 (https://huggingface.co/blog/train-sentence-transformers)
- What can LLMs never do? (https://www.strangeloopcanon.com/p/what-can-llms-never-do)
- Make LLM Fine-tuning 2x faster with Unsloth and 🤗 TRL (https://huggingface.co/blog/unsloth-trl)
- https://unsloth.ai/blog
- https://github.com/unslothai/unsloth
- https://www.kaggle.com/code/danielhanchen/kaggle-mistral-7b-unsloth-notebook
- Retentive Networks (RetNet) Explained: The much-awaited Transformers-killer is here (https://medium.com/ai-fusion-labs/retentive-networks-retnet-explained-the-much-awaited-transformers-killer-is-here-6c17e3e8add8)
- Mamba Explained - The State Space Model taking on Transformers (https://www.kolaayonrinde.com/blog/2024/02/11/mamba.html)
- Introducing Jamba: AI21's Groundbreaking SSM-Transformer Model (https://www.ai21.com/blog/announcing-jamba)
- ModuleFormer: a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward experts (https://github.com/IBM/ModuleFormer)
- JetMoE: Reaching LLaMA2 Performance with 0.1M Dollars (https://research.myshell.ai/jetmoe)
- SUPRA: Scalable UPtraining for Recurrent Attention - uptrain a transformer to a linear RNN (https://github.com/TRI-ML/linear_open_lm) (https://huggingface.co/TRI-ML/mistral-supra)