IOError: [Errno 2] No such file or directory: './dialogs/10/974.tsv' about ubuntu-ranking-dataset-creator

[EmielStoelinga]

I am having the same issue. Is there a fix already?

[wucz]

i met same question too.
I download the data manually, not by script. After decompress the .tar ,I find some folders is empty, such as 10, but in .tar it's not.
The question might be wrong way decompress, or downloading the data.

[kuhanw]

I am having a similar issue. My file size appears to be only 250 MB, but I gather the full size is over 500 MB?

Error below:

IOError: CRC check failed 0x7bc3be24 != 0x13e71207L
0
Traceback (most recent call last):
File "create_ubuntu_dataset.py", line 407, in
args.func(args)
File "create_ubuntu_dataset.py", line 358, in test_cmd
create_eval_dataset(args, "testfiles.csv")
File "create_ubuntu_dataset.py", line 287, in create_eval_dataset
lambda context_dialog, candidates : create_single_dialog_test_example(context_dialog, candidates, rng,
File "create_ubuntu_dataset.py", line 228, in create_examples
examples.append(creator_function(context_dialog, candidate_dialog_paths))
File "create_ubuntu_dataset.py", line 288, in
args.n, args.max_context_length))
File "create_ubuntu_dataset.py", line 187, in create_single_dialog_test_example
negative_responses = get_random_utterances_from_corpus(candidate_dialog_paths,rng,distractors_num)
File "create_ubuntu_dataset.py", line 82, in get_random_utterances_from_corpus
dialog = translate_dialog_to_lists(dialog_path)
File "create_ubuntu_dataset.py", line 36, in translate_dialog_to_lists
dialog_file = open(dialog_filename, 'r')
IOError: [Errno 2] No such file or directory: '.\dialogs\4\21894.tsv'
0
Traceback (most recent call last):
File "create_ubuntu_dataset.py", line 407, in
args.func(args)
File "create_ubuntu_dataset.py", line 355, in valid_cmd
create_eval_dataset(args, "valfiles.csv")
File "create_ubuntu_dataset.py", line 287, in create_eval_dataset
lambda context_dialog, candidates : create_single_dialog_test_example(context_dialog, candidates, rng,
File "create_ubuntu_dataset.py", line 228, in create_examples
examples.append(creator_function(context_dialog, candidate_dialog_paths))
File "create_ubuntu_dataset.py", line 288, in
args.n, args.max_context_length))
File "create_ubuntu_dataset.py", line 180, in create_single_dialog_test_example
dialog = translate_dialog_to_lists(context_dialog_path)
File "create_ubuntu_dataset.py", line 36, in translate_dialog_to_lists
dialog_file = open(dialog_filename, 'r')
IOError: [Errno 2] No such file or directory: '.\dialogs\3\4347.tsv'

[nabihach]

Same issue here.

[blazezhazha]

I believe something went wrong with the request package used in the code, while I have no idea how it works indeed...

But here comes an alternative to overcome this:

1. You may download the dataset here by yourself (maybe with some download toolkits) and put it into the /src folder manually. The size of this maybe-complete dataset archive is about 555.3MB.
2. After that, you can follow the README.md and generate the data you want. For me, I just run the command ./generate.sh -t -s -l and it did work!

Hope this would help you!
Good luck!

[JasonForJoy]

@blazezhazha I have tried the method you have mentioned, but I can only download a .tgz file about 435MB. Could you share the complete file about 555.3MB with me?
Thank you!