vkhamesi / ocpdet

📦 A Python package for online changepoint detection, implementing state-of-the-art algorithms and a novel approach based on neural networks.

Hello there 👋

I'm a Ph.D. student in Statistics and Machine Learning at Imperial College London working on distribution shift detection in multivariate sequential data. Here is a brief recap of my journey:

• 📦 Amazon Science - Data Scientist - Representation Learning, Recommender Systems, Statistical Machine Learning, Natural Language Processing (1 year)
• 🔬 Imperial College London - M.Sc. in Statistics - Research Master Thesis on Convolutional Neural Networks for Online Changepoint Detection (1 year)
• 🔎 Research Project on Machine Learning for Selection and Ranking, applied to Portfolio Optimisation - Awarded First Prize by Google, HSBC, Société Générale, Sia Partners (1 year)
• 🎓 École Centrale de Lyon - French Engineering Grandes Écoles - Major in Computer Science, Applied Mathematics, Machine Learning (2 years)
• 📚 Lycée Janson de Sailly - Classe Préparatoire aux Grandes Écoles MPSI/MP - Mathematics, Physics, Computer Science (2 years)

ocpdet's Issues

Corret the part of pip install

The letters p and d were changed.

The correct is pip install ocpdet and in the example is writen pip install ocdpet.

TwoSample using sliding windows for X if N=1 ?!

I reviewed the TwoSample class.

I fail to understand why you would loop thru all the iterations of sliding windows of the vector X in your process_batch function:

When printing where you collect the test-statistics you seem to compare sliding windows of N=1 ?!.
That does not make sense, even though the changepoint detection seems to work, besides it is very slow run time due to the permutations.

Can you mabye comment on this, what is the statistical reasoning behind it? I was looking into refactoring this to make it faster an leaner.

while self.t < len(X):
        Dkn = []
        for k in range(1, self.t):
                x, y = X[tau:k], X[k:self.t]
                # Different syntax for each Scipy test
                print(f"X[{tau}:{k}] vs X[{k}:{self.t}]")

X[0:1] vs X[1:2]
X[0:1] vs X[1:3]
X[0:2] vs X[2:3]
X[0:1] vs X[1:4]
X[0:2] vs X[2:4]
X[0:3] vs X[3:4]
X[0:1] vs X[1:5]
X[0:2] vs X[2:5]
X[0:3] vs X[3:5]
X[0:4] vs X[4:5]
X[0:1] vs X[1:6]
X[0:2] vs X[2:6]
X[0:3] vs X[3:6]
X[0:4] vs X[4:6]

should the `S[-1]` be `T[-1]` in the following line

https://github.com/vkhamesi/ocpdet/blob/26ad9ff239447f10dbcbe5dfb1c3fa419c8bd1e8/ocpdet/CUSUM.py#L100C30-L100C36

should the S[-1] be T[-1] in the following line?

        self.T.append(
            max(0, self.S[-1] - (data_new - self.mu) / self.sigma - self.k))