Fail to load "stas/c4-en-10k" dataset since 2.16 version about datasets

[albertvillanova]

I am not able to reproduce the error with datasets 2.19.1:

In [1]: from datasets import load_dataset; ds = load_dataset("stas/c4-en-10k", streaming=True); item = next(iter(ds["train"])); item
Out[1]: {'text': 'Beginners BBQ Class Taking Place in Missoula!\nDo you want to get better at making delicious BBQ? You will have the opportunity, put this on your calendar now. Thursday, September 22nd join World Class BBQ Champion, Tony Balay from Lonestar Smoke Rangers. He will be teaching a beginner level class for everyone who wants to get better with their culinary skills.\nHe will teach you everything you need to know to compete in a KCBS BBQ competition, including techniques, recipes, timelines, meat selection and trimming, plus smoker and fire information.\nThe cost to be in the class is $35 per person, and for spectators it is free. Included in the cost will be either a t-shirt or apron and you will be tasting samples of each meat that is prepared.'}

In [2]: from datasets import load_dataset; ds = load_dataset("stas/c4-en-10k", download_mode="force_redownload"); ds
Downloading data: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 13.3M/13.3M [00:00<00:00, 18.7MB/s]
Generating train split: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10000/10000 [00:00<00:00, 78548.55 examples/s]
Out[2]: 
DatasetDict({
    train: Dataset({
        features: ['text'],
        num_rows: 10000
    })
})

Looking at your error traceback, I notice that the code line numbers do not correspond to the ones of datasets 2.19.1.

Additionally, I can't reproduce the issue with HfFileSystem:

In [1]: from huggingface_hub import HfFileSystem

In [2]: fs = HfFileSystem()

In [3]: with fs.open("datasets/stas/c4-en-10k/c4-en-10k.py", "rb") as f:
   ...:     data = f.read()
   ...: 

In [4]: data[:20]
Out[4]: b'# coding=utf-8\n# Cop'

Could you please verify the datasets and huggingface_hub versions you are indeed using?

import datasets; print(datasets.__version__)

import huggingface_hub; print(huggingface_hub.__version__)

[guch8017]

Thanks for your reply! After I update the datasets version from 2.15.0 back to 2.19.1 again, it seems everything work well. Sorry for bordering you!