liethon / machine-learning-algorithm

:star2:【Numpy 手写实现】SVM 支持向量机 | KNN K近邻 | Kmeans | Logistic Regression 逻辑回归 | Maximum Entropy 最大熵 | Naive Bayes 朴素贝叶斯 | Perception 感知机 | Decision Tree 决策树 | Random Forest 随机森林 | DBDT | GDA 高斯判别分析 | PCA 主成分分析 | LDA 线性判别分析

Machine-Learning-Algorithm

注意: 每个文件只有开始的 class 是模型本身，其它代码都是用来测试的，每个模型的实现都在 100 行以内

Note: Only the class at the beginning of each file is the model itself, the rest of the code is for testing, and the implementation of each model is within 100 lines

---

1. Logistic Regression

File - logistic_regression.py

Cost Function -

Optimization Algorithm - Gradient descent method

---

2. Support Vector Machine

File - support_vector_machine.py

Example -

Cost Function -

Optimization Algorithm - Sequential minimal optimization (SMO)

---

3. Perception

File - perception.py

Example -

---

4. Naive Bayes

File - naive_bayes.py

Example -

---

5. K-Nearest Neighbor

File - k_nearest_neighbor.py | util_kd_tree.py

Example -

---

6. Decision Tree

File - decision_tree.py

Optimization Algorithm - Generalized Iterative Scaling (GIS)

Example -

---

7. Random Forest

File - random_forest.py | | decision_tree.py

Example -

---

8. Gradient Boosting Decision Tree

File - gradient_boosting_decision_tree.py | decision_tree.py

---

9. Linear Discriminant Analysis

File - linear_discriminant_analysis.py

---

10. Maximum Entropy

File - maximum_entropy.py

Example -

---

11. Gaussian Discriminant Analysis

File - gaussian_discriminant_analysis.py

---

12. Principal Component Analysis

File - principal_component_analysis.py

Example -

---

13. K-means

File - kmeans.py | util_kd_tree.py