[FEATURE] Add option to print or store each generation during a `pipeline` run about distilabel

[dvsrepo]

Hi!

We should make more clear that users need to use this:

https://distilabel.argilla.io/latest/technical-reference/pipeline/#dataset-checkpoints

[kcentric]

Ah! Yes, I did see that argument was available, but I didn't know what exactly it was for. That would be helpful.

Might it be possible it add a default value for dataset-checkpoints, just in case users don't use it?

[plaguss]

Hi @kcentric the default for the value is to have a DatasetCheckpoint with the default values as can be seen here in the API reference, which results in the dataset being returned at the end of the pipeline by default. The reason for this was that we couldn't set a "sensible" value for the checkpointing (for example, we can set the checkpoint strategy to save the dataset every 100 records always, but that number implies a different frequency for a dataset of 100, 1K or 10K records for example, and for a bigger dataset, the default could mean that the dataset is saved to often, which could also be undesired). Thanks for your feedback, will try to improve the default behaviour.

[k-neev]

No worries, thanks @plaguss. Maybe we could do something like making the argument a required one (but that could add some more complication) or set it to updating every 1/10th of whatever dataset size we have (which might have its own implications but personally for me it's better to save more frequently than less frequently 😅). Anyways, I see now.