wei-1 / scala-machine-learning

No Dependency Scala Machine Learning Algorithm Gallery

Scala Machine Learning

Branch	Status	CodeCov
master
develop

Light Weight Scala Machine Learning Library

A very light weight Scala machine learning library that provide some basic ML algorithms in Scala. The repo is served as a algorithm gallery. Please enjoy and dive into the algorithm that you will like to learn in its basic level.

Dev-Environment

• Scala 2.13

• Sbt 2.1

This package includes

Classification :

• Naive Bayesian Decision [Code] [Usage]

• K-Nearest Neighborhood (KNN) [Code] [Usage]

• Gaussian Process Classification [Code] [Usage]

• Linear Classification [Code] [Usage]

• Linear Support Vector Machine (linear-SVM) [Code] [Usage]

• Perceptron [Code] [Usage]

• Decision Tree [Code] [Usage]

• Random Forest [Code] [Usage]

• Extreme Learning Machine [Code] [Usage]

Boost :

• Naive Boost [Code] [Usage]

• Weighted Boost [Code] [Usage]

• Gradient Boost [Code] [Usage]

Regression :

• Multiple Linear Regression [Code] [Usage]

• Multivariate Linear Regression - GD [Code] [Usage]

• Stochastic Gradient Decent [Code] [Usage]

• Regression Tree [Code] [Usage]

Clustering :

• Hierarchical [Code] [Usage]

• DBSCAN [Code] [Usage]

• HDBSCAN [Code] [Usage]

• BIRCH [Code] [Usage]

• K-Means [Code] [Usage]

• EM Cluster [Code] [Usage]

• Density Peak Cluster [Code] [Usage]

Neural Net & Deep Learning :

• Neural Network (NN) [Code] [Usage]

• Restricted Boltzmann Machine (RBM) [Code] [Usage]

• Deep Belief Network (DBN) [Code] [Usage]

• Long Short-Term Memory (LSTM) [Code] [Usage]

• Neural Turing Machine - Memory Searching Cognition [Code (in another Repo)]

Optimization :

• Gene Algorithm (GA) [Code] [Usage]

• Minimax [Code] [Usage]

• Monte Carlo Tree Search (MCTS) [Code] [Usage]

• Epsilon Greedy Search [Code] [Usage]

• Upper Confidence Bound [Code] [Usage]

Reinforcement Learning :

• Naive Feedback [Code] [Usage]

• Q-Learning [Code] [Usage]

• Q-Neural Learning [Code] [Usage]

• Deep Q-Network (DQN) [Code] [Usage]

• Dueling DQN (D-DQN) [Code] [Usage]

• Asynchronous Advantage Actor-Critic (A3C) [Code] [Usage]

• Prioritized Experience Replay (PER-DQN) [Code] [Usage]

Feature Analysis :

• Student-T Test [Code] [Usage]

• ANOVA [Code] [Usage]

• Linear Discriminant Analysis [Code] [Usage]

• Quadratic Discriminant Analysis [Code] [Usage]

Feature Transformation :

• One Hot Encoding [Code] [Usage]

Abnormal Detection :

• Isolation Tree [Code] [Usage]

• Isolation Forest [Code] [Usage]

• Random Cut Tree [Code] [Usage]

• Random Cut Forest [Code] [Usage]

TODO

• Rainbow - Deep Reinforcement Learning

• Alpha-go Zero (MCTS-NN) - Deep Reinforcement Learning

• Neural Architect Search (NAS) - Neural Network & Deep Learning

Installation Process

1. Install sbt

2. Clone this project

Test

sbt test

Build Jar

sbt assembly

scala-machine-learning's Issues

Feature Analysis

• Student-T
• ANOVA
• MANOVA
• PCA
• LDA
• QDA

LinearSVM not in Classifier trait yet

• restruct LinearSVM into Classifier trait

[Code] - Scala-Machine-Learning - ExtremeLearning Training Bug

https://travis-ci.org/github/Wei-1/Scala-Machine-Learning/builds/720148266

• identify the error
• solve
• test

Neural Turing Machine

• Neural Turing Machine

Alpha-go Zero (MCTS-NN)

Reinforcement Learning need to catch up with the state of the art

• Alpha-go Zero (MCTS-NN)

Ref: https://github.com/grimmer0125/alphago-zero-tictactoe-js

Create the Regression Section

Currently, I put Linear Regression in Classification.
And we actually do Linear Regressing Classification in the code.
For actually Linear Regression, we should start a new section and write new code.

• Multiple Linear Regression
• Multivariant Linear Regression

Move from Travis-CI to Github-Action

Travis-CI keep breaking because of dependencies download errors, we might need to find a more robust method to test the repo.

Codecov implementation

• codecov
• badge

Boost for classifier

let's build a classifier trait and implement boost algorithms with it.

• naive boost
• weighted boost

Upgrade Neural Network

Upgrade neural network and related deep learning network based on my own implementation
https://github.com/Wei-1/Scala-NN-Playground/tree/basic-nn

---

Deep Learning

• NN
• DBN

Reinforcement Learning

• Q-Neural Learning
• DQN
• DDQN
• A3C

Depth Control to Avoid Overfitting

While testing some dataset with my algorithms, I found that my current implementations overfit a lot because there is no depth control or fuzzy logic been taken into consideration.
(especially for tree-based algorithms)
It shouldn't be hard to fix.

Restructure with Unified IO Class or Trait (Classifier)

Since this one single card is too big.
I will separate it into several different cards.
This card will be kept for Classifier only

• Classifier

Handle categorical data (One Hot)

One Hot Encoding should be implemented for categorical data.

• One Hot Encoding
• Decoding (back to category)

Rainbow

Ref: https://github.com/Curt-Park/rainbow-is-all-you-need
Leave a comment if you are willing to help

Neural Architecture Search

Ref: https://github.com/melodyguan/enas
Leave a comment if you are willing to help

Reinforcement Learning

• Naive Feedback
• Q-Learning
• Q-Neural Learning
• Deep Q Network
• Double DQN
• A3C