Invalid provenance file crashes task execution about cloud-volume

[william-silversmith]

Thanks for reporting this Nico. Initially I was torn on this, since it encourages you to fix the provenance file, but it might make more sense to issue a warning.

[william-silversmith]

On second thought, the warning should be better, but this should be an irrecoverable error, and not only for motivational purposes. If we continue to allow processing, say using an empty version of the provenance file, it's likely that it will get committed and overwrite the extremely useful but ill formatted notes already existing.

Alternatively, we could ban editing the provenance file from CloudVolume in cases where the JSON is corrupted. What do you think @nkemnitz?

[william-silversmith]

I have another idea that I think I'm just going to go with. We should use json5 (http://json5.org/) instead to make it easier to write the provenance files and make them accept things like trailing commas.

[nkemnitz]

I am not sure if I understand: Why is the provenance file loaded? I thought that is a human-readable info file with contact information. For running tasks, I should only need the info file, right?

If it is necessary to return an error message, then I would say it should go to the maintainers of the dataset/layer, not to the end-user :P

[william-silversmith]

You can also use the provenance file to detail how the dataset is being modified and, yes, send messages to maintainers. I'm including a way to override the provenance file in the constructor in another PR.

[nkemnitz]

The json5 solution is definitely a suitable workaround for the specific issue I had - thanks.
But I think it does not fix the underlying design flaw: If someone wants to work with the data alone (e.g. getting cutouts), then there is no need to download and parse the (possibly invalid) provenance file.

[william-silversmith]

No one has complained about this since this issue was opened, so I'm going to consider my solution a success. :) Only valid provenance files are written!