luk-f / pycfof

CFOF developed in Python. Based on Angiulli's works : https://arxiv.org/pdf/1901.04992v2.pdf

👋 Hello everyone

💬 I'm Lucas and I'm a machine learning engineer

I like to use Python 🐍 for personal and professional projects (but this is not my native language 😝).

Here is a quick list of my most recent or significant projects: 💻

• Just recently, I have create a small app to play with a buzzer 🚨 on each personal device. I developed it to try Flask API ! You can find it here: luk-f/BuzzerApp
• I'm currently work on a algorithm used to aggregate movement data (in 2D) and generalize spatial information: luk-f/AggOfMassiveMvtData,
  • and I've implemented a specific package with pybind11: luk-f/pyhaversine
• And obviously, I should refer to my PhD work 🎓 that just finished last year, an unsupervised method of anomaly detection on multidimensional stream data (a CFOF approximation with iSAX tree): luk-f/pyCFOFiSAX
• Also developed the CFOF method to compare it to my thesis work: luk-f/pyCFOF

pycfof's Issues

Error with rho value egal to 0.1 (float)

import numpy as np
from pyCFOF import ConcentrationFreeOutlierFactor as CFOF

dataset = np.random.normal(loc=0.0, scale=1.0, size=(10_000, 2))
cfof = CFOF(n_neighbors=len(dataset), rho=0.1)

>>> Rho type error: must be a float: 0.1.
>>> Default value applied. <class 'TypeError'>: 'float' object is not iterable

Scores not modified for few intermediate rho value

In this example, some objects obtain the minimal score (0.0001) for the second last rho value.

`dataset = np.concatenate((np.random.normal(loc=0.0, scale=1.0, size=(9_000, 2)),
np.random.normal(loc=6.0, scale=1.0, size=(1_000, 2))), axis=0)

rho_list = [0.001, 0.01, 0.02, 0.05, 0.09, 0.1, 0.5]
cfof = CFOF(n_neighbors=len(dataset), rho=rho_list)

for row in cfof.outlier_factor_:
score_prec = row[0]
for score in row[1:]:
if score < score_prec:
print(row)`

To wonder if...

Is it useful to do this ?

not_same_neighbor = (np.diff(np.concatenate((np.zeros_like(self._distances_fit_X_.T[0]).reshape(-1, 1),
                                                     self._distances_fit_X_), axis=1)) != 0).T
not_same_neighbor = np.concatenate(([np.zeros_like(not_same_neighbor[0]).T], not_same_neighbor))

IndexError: index 9999 is out of bounds for axis 0 with size 9999

Overflow with 2 clusters and rho value at 0.99 :

dataset = np.concatenate((np.random.normal(loc=0.0, scale=1.0, size=(9_000, 2)), 
                          np.random.normal(loc=6.0, scale=1.0, size=(1_000, 2))), axis=0)

rho_list = [0.01, 0.1, 0.5]
cfof = CFOF(n_neighbors=len(dataset), rho=rho_list)

_ = cfof.fit_predict(dataset)

Adapt contamination to multirho value