baraline / convst Goto Github PK

The performance on ASCF1 is very low in a cross validation setting for RDST + Ridge, behaviour changed after deprecation of the normalize argument for Ridge classifier.
TODO :
Check wheter or not the "with_mean" argument on StandardScaler should be step to False. Changing it to True seems to solve the issue.

In the multivariate transforms, the alpha similarity mask of shape (2,n_dilations,n_samples,n_features,n_timestamps) is shared independently of the feature subset used. A shapelet using features 1,2,3 and another features 2,3,4 will then share 2/3 of the mask information, which may be restrictive for selecting some interesting patterns.

• Change the alpha similarity mask logic for multivariate time series

Design a helper function to run all numba compilations with very small data to avoid issues with first-time uses.

• Helper function must compile all possible function signatures (32/64)
• Ideally, must be run during the installation process (is it possible ?), not during imports to avoid repeating it. If not, provide it as part of the API
• Update documentations and readme to mention this function

How do i extract distance vector or transformed shapelets for each of my time series?

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

During some, but not all, runs (e.g. FordA / FordB datasets) RDST Ensemble classifier fails with the following error dump :

joblib.externals.loky.process_executor._RemoteTraceback: 
Traceback (most recent call last):
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/numba/core/caching.py", line 469, in save
    data_name = overloads[key]
KeyError: ((array(float64, 3d, C), Tuple(array(float64, 2d, C), array(int64, 1d, C), array(int64, 1d, C), array(float64, 1d, C), array(bool, 1d, C)), type(CPUDispatcher(<function manhattan at 0x7fce883fe0d0>)), bool), ('x86_64-unknown-linux-gnu', 'cascadelake', '+64bit,+adx,+aes,-amx-bf16,-amx-int8,-amx-tile,+avx,+avx2,-avx512bf16,-avx512bitalg,+avx512bw,+avx512cd,+avx512dq,-avx512er,+avx512f,-avx512ifma,-avx512pf,-avx512vbmi,-avx512vbmi2,+avx512vl,+avx512vnni,-avx512vp2intersect,-avx512vpopcntdq,+bmi,+bmi2,-cldemote,+clflushopt,+clwb,-clzero,+cmov,+cx16,+cx8,-enqcmd,+f16c,+fma,-fma4,+fsgsbase,+fxsr,-gfni,+invpcid,-lwp,+lzcnt,+mmx,+movbe,-movdir64b,-movdiri,-mwaitx,+pclmul,-pconfig,+pku,+popcnt,-prefetchwt1,+prfchw,-ptwrite,-rdpid,+rdrnd,+rdseed,-rtm,+sahf,-serialize,-sgx,-sha,-shstk,+sse,+sse2,+sse3,+sse4.1,+sse4.2,-sse4a,+ssse3,-tbm,-tsxldtrk,-vaes,-vpclmulqdq,-waitpkg,-wbnoinvd,-xop,+xsave,+xsavec,+xsaveopt,+xsaves'), ('00e465fe82fb9c04ee9ece12d3d459a0d4fe0a0d451df090bccee8dc666d02b2', 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 436, in _process_worker
    r = call_item()
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 288, in __call__
    return self.fn(*self.args, **self.kwargs)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/joblib/_parallel_backends.py", line 595, in __call__
    return self.func(*args, **kwargs)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/joblib/parallel.py", line 262, in __call__
    return [func(*args, **kwargs)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/joblib/parallel.py", line 262, in <listcomp>
    return [func(*args, **kwargs)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/sklearn/utils/fixes.py", line 117, in __call__
    return self.function(*args, **kwargs)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/convst-0.2.1-py3.8.egg/convst/classifiers/rdst_ensemble.py", line 56, in _parallel_fit
    return model.fit(X, y)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/sklearn/pipeline.py", line 378, in fit
    Xt = self._fit(X, y, **fit_params_steps)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/sklearn/pipeline.py", line 336, in _fit
    X, fitted_transformer = fit_transform_one_cached(
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/joblib/memory.py", line 349, in __call__
    return self.func(*args, **kwargs)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/sklearn/pipeline.py", line 870, in _fit_transform_one
    res = transformer.fit_transform(X, y, **fit_params)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/sklearn/base.py", line 870, in fit_transform
    return self.fit(X, y, **fit_params).transform(X)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/convst-0.2.1-py3.8.egg/convst/transformers/rdst.py", line 270, in transform
    X_new = self.transformer(
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/numba/core/dispatcher.py", line 487, in _compile_for_args
    raise e
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/numba/core/dispatcher.py", line 420, in _compile_for_args
    return_val = self.compile(tuple(argtypes))
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/numba/core/dispatcher.py", line 972, in compile
    self._cache.save_overload(sig, cres)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/numba/core/caching.py", line 652, in save_overload
    self._save_overload(sig, data)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/numba/core/caching.py", line 662, in _save_overload
    self._cache_file.save(key, data)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/numba/core/caching.py", line 478, in save
    self._save_index(overloads)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/numba/core/caching.py", line 522, in _save_index
    data = self._dump(data)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/numba/core/caching.py", line 550, in _dump
    return dumps(obj)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/numba/core/serialize.py", line 57, in dumps
    p.dump(obj)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/numba/cloudpickle/cloudpickle_fast.py", line 568, in dump
    return Pickler.dump(self, obj)
  File "/home/prof/guillaume/anaconda3/envs/convst/lib/python3.8/site-packages/numba/core/types/functions.py", line 486, in __getnewargs__
    raise ReferenceError("underlying object has vanished")
ReferenceError: underlying object has vanished

• Create a new util file, containing the private code used to generate critical difference and pairwise plots.
• Add examples to the documentation and links

Working on multivariate time series, with shape (6202x2x33), when running the interpreter model, I am getting this error
SystemError: _PyEval_EvalFrameDefault returned a result with an error set
Please let me know if you need more information
With univariate time series, I dont see the error

v0.2.0 introduced major performance loss for some dataset, such as PigCVP (0.87→0.37). Cause is unknown for now.

Notes to self :

• Some changes in the sktime API with 0.11+, along with deprecation warnings to fix
• Check required version is still up-to-date

Potential impacts :

• Dataset loading utilities
• Some utils function for input checks
• Some models API used in the experiments for the paper

The current formula for alpha similarity was not adapted to multiple length shapelets. This does not cause any exception, but can harm the shapelet sampling process.

• Change alpha similarity in all transformers to $max(1,(1-\alpha) min(L))$

Additional bug with multiple lengths:

• Debug tuple creation for length and dilation parameters in transformers

Introduced in v0.2, RDST Ensemble is sensibly slower than on the experimental build. Further inspection is needed to find the source of the problem.

The results folder is not up-to-date with the latest results, fetch CSV files and replace old ones. CD figure is already updated, check for other figures to update.

In some cases, using prime_dilation=True is slower than prime_dilation=False for RDST Ensemble. This can happen for example on the Rock dataset.

• Investigate the issue (it only happens for Ensemble)

Hello,

I'm trying to use the RDST implementation for TSC task, however, I'm interested in the interpretability of the method. So, I would like to confirm what it is the correct way to make the extraction of the shapelets of the model and the time series that generate them.

Thanks in advance,

Known bug : n_jobs = -1 will return the number of CPU + 2 instead of the number of CPU, fixed by #14

• Close this issue when a new release is out

Add the video presentation made at ICPRAI to the Readme page and readthedocs as it is a good introduction to Shapelets and the components introduced by RDST.

• Compile all video slides into one
• Upload the video (is it possible to integrate it directly in readme ? Or link to a video hosting)
• Add dedicated section to readme and docs

In some use case, it might be interesting to only sample shapelets from a subset of the input dataset. For example in an imbalanced binary classification context, if class 0 represent 99% of samples, the majority of shapelets will be sampled from class 0 with the actual random sampling scheme. It might be more interesting to sample shapelets only from class 1 or to downsample class 0.

While it is simple to add this as a step before doing an RDST transform, it would require reimplementing or hacking through the RDST classifiers.

• Replace the current n_samples parameter of RDST to allow different sampling options, such as :

• Only sample shapelets from a specified subset of the data (boolean mask on samples and/or timestamps)

• String options such as "balanced"

• Current option with float to indicate a percentage for up/downsampling

• Update API and docs of the rdst transformer and linked classifiers. Clarify which parameter (e.g class_weight) impact the classifier and which impact the shapelet sampling.

• Add unit tests for each options

In some case, shapelet initialization for multivariate time series return mostly channel 0. The problem is removed by setting Numba parallel to False.

• Find the origin of the bug in the multivariate initialization with parallel option to True.

When running test_on_arff_resamples.py, the following error occurred. How to solve it?
PaperScripts/test_on_arff_resamples.py:None (PaperScripts/test_on_arff_resamples.py) test_on_arff_resamples.py:54: in <module> pipeline_RDST_rdg = model_class(n_jobs=3, n_jobs_rdst=100//3) E TypeError: __init__() got an unexpected keyword argument 'n_jobs_rdst'
'

When installing convst for Python 3.11, error occurs due to some dependencies version not matching with the absence of upper bound version for dependencies.

• Update lower dependencies version
• Add upper version
• Add Python 3.11 to test pipeline
• Add deprecations check in test pipeline (Is it possible to make test for both lower and upper versions specified ?)
• Move test, doc and graphic dependencies to optional dependencies

Describe the bug
A clear and concise description of what the bug is.
There is data loading error in 'Wafer' dataset, while 'GunPoint' is ok.

To Reproduce
Steps to reproduce the behavior:
You can reproduce the error by running following file.
https://1drv.ms/f/s!AuU5Lmr0utymk4IKdA1WNoxzkD0tNg

Expected behavior
A clear and concise description of what you expected to happen.
Accuracy should be 1 on wafer dataset.

Code example
If applicable, add code example to help explain your problem.
Same as above

Environment (please complete the following information):

• OS: [e.g. iOS] Google colab
• Version of the convst package [e.g. 0.15] : Latest (0.3.0)

Additional context
Add any other context about the problem here.

In some rare edge cases (e.g., on ACSF1), _get_subsequence function in RDST transform may return nan values after normalization, due to the standard deviation being computed as nan. This was unseen in the experiments, as the StandardScaler is dealing with the nan values before the Ridge classifier.

• Fix implementation of _get_subsequence to avoid nan value on standard deviation.