w4k2 / dse

Deterministic Sampling Ensemble

Deterministic Sampling Ensemble

---

Deterministic Sampling Ensemble diagram

Deterministic Sampling diagram

---

Experiment 1 - Evaluating the best sampling method

Experiment files:

• Experiment
• Analyze

Methods:

• DSE - Deterministic Sampling Ensemble

Base classifiers:

• Support Vector Machines (SVM)
• K-Nearest Neighbors (KNN)
• Gaussian Naive Bayes (GNB)
• Decision Tree Classifier (DTC)

Data streams:

• Generators:
  • stream-learn
  • MOA
• Concept drift:
  • sudden
  • incremental
• Objects: 15 000
• Features: 10
• Imbalance Ratio: 10%
• Noise: 10%
• Random samples: 333

Results:

Results of Random Under Sampling combination with oversampling methods. Darker is better, best value isbold and underscored

Results of SVMSMOTE combination with undersampling methods. Darker is better, best value is bold andunderscored

Results of NCR combination with oversampling methods. Darker is better, best value is bold and underscored

---

Experiment 2 - Evaluating the best balance ratio param

Files:

• Experiment
• Analyze

Methods:

• DSE - Deterministic Sampling Ensemble

Base classifiers:

• Support Vector Machines (SVM)
• K-Nearest Neighbors (KNN)
• Gaussian Naive Bayes (GNB)
• Decision Tree Classifier (DTC)

Data streams:

• Generators:
  • stream-learn
  • MOA
• Concept drift:
  • sudden
  • incremental
• Objects: 15 000
• Features: 10
• Imbalance Ratio: 10%
• Noise: 10%
• Random samples: 333

Results:

Balance parameter setup experiment. Darker is better, best value bold and underscore

---

Experiment 3 - Evaluating the performance on different noise ratio data stream

Files:

• Experiment
• Analyze
• Plots

Methods:

• DSE - Deterministic Sampling Ensemble
• REA - Recursive ensemble approach
• KMC - K-mean clustering undersampling ensemble
• L++CDS - Learn++CDS
• L++NIE - Learn++NIE
• OUSE - Over and under-sampling ensemble
• MLPC - Multi-layer perceptron classifier

Base classifiers:

• Support Vector Machines (SVM)
• K-Nearest Neighbors (KNN)
• Gaussian Naive Bayes (GNB)
• Decision Tree Classifier (DTC)

Data streams:

• Generator: stream-learn
• Concept drift: incremental
• Objects: 10 000
• Features: 10
• Imbalance Ratio: 10%
• Noise: 5%, 10%, 15%, 20%, 25%, 30%, 35%, 40%
• Random samples: 111, 222, 333, 444, 555

Results:

Selected mean results from noise experiments

---

Experiment 4 - Evaluating the performance on different balance ratio data stream

Files:

• Experiment
• Analyze
• Plots

Base classifiers:

• Support Vector Machines (SVM)
• K-Nearest Neighbors (KNN)
• Gaussian Naive Bayes (GNB)
• Decision Tree Classifier (DTC)

Methods:

• DSE - Deterministic Sampling Ensemble
• REA - Recursive ensemble approach
• KMC - K-mean clustering undersampling ensemble
• L++CDS - Learn++CDS
• L++NIE - Learn++NIE
• OUSE - Over and under-sampling ensemble
• MLPC - Multi-layer perceptron classifier

Data streams:

• Generator: stream-learn
• Concept drift: incremental
• Objects: 10 000
• Features: 10
• Imbalance Ratio: 5%, 10%, 15%, 20%, 25%, 30%, 35%, 40%, 45%
• Noise: 10%
• Random samples: 111, 222, 333, 444, 555

Results:

Selected mean results from noise and balance experiments

---

Experiment 5 - Main evaluation (synthetic data)

Files:

• Experiment stream-learn 1 drift
• Experiment stream-learn 5 drifts
• Experiment moa 1 drift
• Experiment moa 5 drifts
• Analyze
• Plots

Base classifiers:

• Support Vector Machines (SVM)
• K-Nearest Neighbors (KNN)
• Gaussian Naive Bayes (GNB)
• Decision Tree Classifier (DTC)

Methods:

• DSE - Deterministic Sampling Ensemble
• REA - Recursive ensemble approach
• KMC - K-mean clustering undersampling ensemble
• L++CDS - Learn++CDS
• L++NIE - Learn++NIE
• OUSE - Over and under-sampling ensemble
• MLPC - Multi-layer perceptron classifier

Data streams:

• Generators:
  • stream-learn
  • MOA
• Concept drifts:
  • 1 sudden
  • 1 incremental
  • 5 sudden
  • 5 incremental
• Objects: 100 000
• Features: 10
• Imbalance Ratio: 10%, 20%, 30%
• Noise: 0%, 10%
• Random samples: 111, 222

Results:

Wilcoxon pair rank-sum tests for synthetic data streams with incremental concept drift. Dashed vertical line isa critical value with a confidence level 0.05 (green – win, yellow – tie, red – loss)

Wilcoxon pair rank-sum tests for synthetic data streams with sudden concept drift. Dashed vertical line is acritical value with a confidence level 0.05 (green – win, yellow – tie, red – loss)

---

Experiment 5 - Main evaluation (real data)

Files:

• Experiment
• Analyze
• Plots

Base classifiers:

• Support Vector Machines (SVM)
• K-Nearest Neighbors (KNN)
• Gaussian Naive Bayes (GNB)
• Decision Tree Classifier (DTC)

Methods:

• DSE - Deterministic Sampling Ensemble
• REA - Recursive ensemble approach
• KMC - K-mean clustering undersampling ensemble
• L++CDS - Learn++CDS
• L++NIE - Learn++NIE
• OUSE - Over and under-sampling ensemble

Data streams:

• covtypeNorm-1-2vsAll-pruned
• poker-lsn-1-2vsAll-pruned

Results:

F-score metric over the data chunks for covtypeNorm-1-2vsAll data stream with SVM base classifier

F-score metric over the data chunks for poker-lsn-1-2vsAll data stream with SVM base classifier