Compatibility issues about viscy

[ziw-liu]

We'd fine-tune the VSCyto3D model with data acquired on Mantis as we discussed at the lab meeting yesterday.

I think @edyoshikun is already training these models without needing code changes. (Recall that VSCyto3D and VSNeuromast are the same architecture trained on different data.) The incompatibility I mentioned in OP for VSCyto3D is between the old config files we published (e.g. on zenodo for the mantis preprint) and the current code base. This is a minor issue since that zenodo record points to a pinned version of VisCy that works properly with the files.

[ziw-liu]

My reading of the code suggests that all of these can be instantiated by passing parameters to UNet21d.

The VSCyto2D models are made from the FCMAE class for both pre-training and fine-tuning.

My suggestion is:

1. Merge #71
2. Merge #84
3. Merge #88
4. Tag v0.1.0rc0
5. Prepare v0.1.0 release artifacts (weights and config files) based on the new code base

[mattersoflight]

@ziw-liu. This order of operation makes sense to me. Are you renaming what we thought of release 0.2.0 as 0.1.0? Totally fine to call the next release 0.1.0. Please update the milestone for clarity if you mean something else.

[ziw-liu]

@edyoshikun can you confirm the details for the neuromast model and comment on compatibility requirements of the mantis revision?

[edyoshikun]

Correct! Neuromast model was trained with the current HEAD of the main. I confirm.

[mattersoflight]

Hi @ziw-liu We'd fine-tune the VSCyto3D model with data acquired on Mantis as we discussed at the lab meeting yesterday. Please start a PR with changes that enable training of both VSCyto3D and VSNeuromast models with the main. This PR should be a priority since we need it to iterate on the models for mantis Fig. 1 next.

It seems like 2D FCMAE branch will require development such that the weights are compatible with the main. I don't follow this point, can you point to a few lines of code to illustrate? This can be the second PR.

Thanks for iterating quickly on these.

[ziw-liu]

It seems like 2D FCMAE branch will require development such that the weights are compatible with the main. I don't follow this point, can you point to a few lines of code to illustrate?

The network module changed, among other things. For example:
https://github.com/mehta-lab/VisCy/pull/71/files#diff-9af609b0fdd9a3358d7099a616e29f1fefdab379a92388bca856ab99944fc67c

[mattersoflight]

@ziw-liu How about the following order of operation?

• Create a release candidate 0.2.0 that works with current weights of VSCyto3D, VSNeuromast, and VSCyto2D models. My reading of the code suggests that all of these can be instantiated by passing parameters to UNet21d. You can rename this model UNeXt2 to match the nomenclature in the preprint.
• Update FullyConvolutionalMAE model such that the weights can be transferred from FCAMAE to VSCyto2D models.

[edyoshikun]

Let's also update the Readme.md and add the preprint link here as well.

[ziw-liu]

Let's also update the Readme.md and add the preprint link here as well.

#85