alan-turing-institute / tcpd Goto Github PK

First of all, thank you for your effort in creating a change point detection benchmark. I really appreciate this as the benchmarking of algorithms is not standardized in most publications. Therefore, I wanted to use your dataset to test different algorithms.

I used a virtual environment to install all required packages as described in the Readme. Then I use
python build_tcpd.py -v collect within the activated virtual environment to start the download of data.

After downloading the first files for the Apple dataset (AAPL.csv and apple.json appear in the directory ./datasets/apple). The script throws and Validation error:

Running collect action for dataset: apple ... Traceback (most recent call last):
  File "C:\Users\Lucas\PycharmProjects\TCPD\build_tcpd.py", line 115, in <module>
    main()
  File "C:\Users\Lucas\PycharmProjects\TCPD\build_tcpd.py", line 110, in main
    func(name, script)
  File "C:\Users\Lucas\PycharmProjects\TCPD\build_tcpd.py", line 85, in collect_dataset
    return run_dataset_func(name, script, "collect")
  File "C:\Users\Lucas\PycharmProjects\TCPD\build_tcpd.py", line 81, in run_dataset_func
    func(output_dir=dir_path)
  File "./datasets\apple\get_apple.py", line 213, in collect
    write_patch(json_path, target_path=json_path)
  File "./datasets\apple\get_apple.py", line 68, in wrapper
    raise ValidationError(target)
tcpd.apple.ValidationError: ./datasets\apple\apple.json

If I delete the Apple Dataset from TARGETS in build_tcpd.py, it then proceeds with everything for bee_waggle_6. There it also notes, that the checksum does not match. The script then throws the next Validation error at bitcoin.

Running collect action for dataset: bee_waggle_6 ... Warning: Generated dataset ./datasets\bee_waggle_6\bee_waggle_6.json didn't match a known checksum. This is likely due to rounding differences caused by different system architectures. Minor differences in algorithm performance can occur for this dataset. 
ok
Running collect action for dataset: bitcoin ... Traceback (most recent call last):
  File "C:\Users\Lucas\PycharmProjects\TCPD\build_tcpd.py", line 114, in <module>
    main()
  File "C:\Users\Lucas\PycharmProjects\TCPD\build_tcpd.py", line 109, in main
    func(name, script)
  File "C:\Users\Lucas\PycharmProjects\TCPD\build_tcpd.py", line 84, in collect_dataset
    return run_dataset_func(name, script, "collect")
  File "C:\Users\Lucas\PycharmProjects\TCPD\build_tcpd.py", line 80, in run_dataset_func
    func(output_dir=dir_path)
  File "./datasets\bitcoin\get_bitcoin.py", line 131, in collect
    write_json(csv_path, target_path=json_path)
  File "./datasets\bitcoin\get_bitcoin.py", line 66, in wrapper
    raise ValidationError(target)
tcpd.bitcoin.ValidationError: Validating the file './datasets\bitcoin\bitcoin.json' failed.
Please raise an issue on the GitHub page for this project if the error persists.

Could you help me out, what is going wrong here? I would really appreciate it. Maybe this has something to do with the validation checksum computation, which might be different on my PC. The venv is using Python 3.9.13 on a Windows machine.

Best regards
Lucas

I noticed that annotation file is the map from annotator id to change points, but I do not get what "annotator id" is meaning. Which one I should use?

When trying to build the file, using python build_tcpd.py -v collect I encounter the following error for

Running collect action for dataset: iceland_tourism ... Traceback (most recent call last):
and the error message I get is:
xlrd.biffh.XLRDError: Excel xlsx file; not supported

This is because the package xlrd has explicitly removed support for anything other than xls files.

I am getting the following error while downloading the data sets

  File "/home/anand/dirichlet/TCPD/build_tcpd.py", line 115, in <module>
    main()
  File "/home/anand/dirichlet/TCPD/build_tcpd.py", line 110, in main
    func(name, script)
  File "/home/anand/dirichlet/TCPD/build_tcpd.py", line 85, in collect_dataset
    return run_dataset_func(name, script, "collect")
  File "/home/anand/dirichlet/TCPD/build_tcpd.py", line 81, in run_dataset_func
    func(output_dir=dir_path)
  File "./datasets/ratner_stock/get_ratner_stock.py", line 170, in collect
    write_patch(json_path, target_path=json_path)
  File "./datasets/ratner_stock/get_ratner_stock.py", line 65, in wrapper
    raise ValidationError(target)
tcpd.ratner_stock.ValidationError: ./datasets/ratner_stock/ratner_stock.json

Please see the downloaded dataset here

I am also facing issue #3

Running collect action for dataset: apple ... Traceback (most recent call last):
File "build_tcpd.py", line 115, in
main()
File "build_tcpd.py", line 110, in main
func(name, script)
File "build_tcpd.py", line 85, in collect_dataset
return run_dataset_func(name, script, "collect")
File "build_tcpd.py", line 81, in run_dataset_func
func(output_dir=dir_path)
File "./datasets/apple/get_apple.py", line 134, in collect
write_json(csv_path, target_path=json_path)
File "./datasets/apple/get_apple.py", line 65, in wrapper
raise ValidationError(target)
tcpd.apple.ValidationError: ./datasets/apple/apple.json
make: *** [collect] Error 1
Trying to run make command. It gives this error. What possible reason?

Hi,

upon running "make", I am getting the following error:

tcpd.bee_waggle_6.ValidationError: Validating the file './datasets/bee_waggle_6/bee_waggle_6.json' failed.
Please raise an issue on the GitHub page for this project if the error persists.
make: *** [collect] Error 1

Can I do this dataset manually?

How come the summary for average table (in case of univariate) and the rank plot be different?
amoc 0.702
binseg 0.706
pelt .689
These are the scores for default cover metric. Then why the rankplot shows AMOC after PELT?