advaitsamudralwar / machine-learning

Machine Learning: Perceptron and K-Nearest Neighbors

Numpy Implementation

This project focuses on implementing two fundamental machine learning algorithms using the powerful Numpy library.

Perceptron and K-Nearest Neighbors

In the realm of supervised learning, we delve into two essential algorithms: the Perceptron and K-Nearest Neighbors (KNN). Our objective is to create efficient implementations utilizing the Numpy library.

Optimal Margin Classifier and K-Nearest Neighbors

For advanced classification tasks, we explore the realm of optimal margin classifiers and KNN. Leveraging both Numpy and CVXPY, we bring forth two key components:

1. Optimal Margin Classifier: We present an implementation of the dual form of the optimal margin classifier, both with and without the Radial Basis Function (RBF) kernel. The utilization of CVXPY allows us to handle complex optimization problems elegantly. This classifier seeks to optimize the margin between classes, enhancing the model's robustness.

2. K-Nearest Neighbors with RBF Kernel: Further enriching our repertoire, we delve into KNN enhanced with the RBF kernel. Combining the power of Numpy and CVXPY, we develop an implementation that extends KNN's capabilities by leveraging the RBF kernel to transform data into a higher-dimensional space, enabling more accurate classification.

Reference: https://www.cvxpy.org

Gradient Descent: Stochastic and Batch Approaches

In the realm of optimization, we shift our focus to gradient descent techniques, particularly Stochastic Gradient Descent (SGD) and Batch Gradient Descent.

Numpy Implementation

Our implementation revolves around the two prominent gradient descent techniques: Stochastic Gradient Descent and Batch Gradient Descent. With Numpy, we craft efficient and streamlined versions of these optimization strategies.

These techniques play a pivotal role in machine learning optimization, enabling models to iteratively learn and adapt to the data. Our Numpy-based implementation ensures both speed and accuracy, contributing to the efficiency of the optimization process.

By mastering these gradient descent approaches, we equip ourselves with powerful tools to fine-tune machine learning models and achieve optimal performance.

For more information and inspiration, refer to our code and the Numpy documentation.

References:

• Numpy Documentation: https://numpy.org/doc/stable/
• CVXPY: https://www.cvxpy.org