Track input data with md5 about mlf-core

[edmundmiller]

@Zethson gotcha. That was what I was thinking as well. Do you want it just printed out or how do you want it logged into mlflow specifically? I've gotten the MD5_sum function from @KevinMenden nf-core module PR.

[Zethson]

[Zethson]

@emiller88 Just assigning it to you for now. If you want and have time you can tackle this after #220 .
But it's not urgent! So don't stress yourself, please.

[edmundmiller]

Where do you want to track the md5 sums? And I'm guess this is more of an added mlf-core feature, not using mlflow.log_artifacts?

[Zethson]

I want to track them with mlflow. I was thinking that every file that gets downloaded (E. G. Pytorch data loader) or supplied as input parameter should get its md5sum calculated and then this is logged with mlflow (not as artifact). Alternatively, we log the data Uri like suggested in the slack post above? I myself am not perfectly sure yet. Wednesday, 03 February 2021, 11:32PM +01:00 from Edmund Miller [email protected] :

…

Where do you want to track the md5 sums? And I'm guess this is more of an added mlf-core feature, not using mlflow.log_artifacts ? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub , or unsubscribe .

[Zethson]

@ggabernet and @KevinMenden any thoughts and ideas?

[KevinMenden]

Hmm Not immediately, I will give it some though ... I'm thinking you might have to do this in the data loader because that's usually the only one that sees the original data. But I also haven't had a coffee yet so :)

Coffee ✔️

Okay I like this but I think it should be kept simple. We could write a small functions that calculates for instance a list of md5 sums for a file system (in case of image data) or just one md5 sum in case it's just a file (might be the case for scRNA-seq data, for instance). And then just issue a warning and/or save the result in somewhere in the mlflow logs output?

This should cover the scenario where you really want to reproduce something by running the model with the same dataset. I don't think we need to cover other cases.
What do you think? Is this what you had in mind @Zethson ?

[Zethson]

@KevinMenden @emiller88 yes exactly. This is what I had in mind as well.
There might just be many edge cases. Dataloaders downloading the data to somewhere. Downloading the data on the fly or accessing it from e.g. S3
We could force dataloaders at least to download to /data, but then we would need to ensure that the tensorboard logs are not in /data but somewhere else.

@emiller88 let's start simple. For now we can just provide the functionality and then slowly see how we can use it while we are developing the use-cases. I would suggest that you add a function to our shiny new MLFCore object which given a single path (string not Path to keep it simple) or a list of paths calculates the MD5 sums and logs them.

[Zethson]

Must be logged with mlflow

Please refer to https://www.mlflow.org/docs/latest/tracking.html

Think that we should then also update our docs to note this to our users somewhere.

[KevinMenden]

@Zethson gotcha. That was what I was thinking as well. Do you want it just printed out or how do you want it logged into mlflow specifically? I've gotten the MD5_sum function from @KevinMenden nf-core module PR.

(And I got it from the net :D )