Discussion on the issue of the basis for the selection of 5-fold cross-validation results and the preservation of trained models. about survpath

[guillaumejaume]

I recommend using better feature extractor, such as our recently published UNI (Nature Medicine). Instructions can be found here: https://github.com/mahmoodlab/UNI.

[guillaumejaume]

Sorry for the late reply. Each fold is trained for a fixed number of epochs. There is no early stopping based on validation c-index (results wouldn't be fair).

[JinchaoChen112]

Sorry for the late reply. Each fold is trained for a fixed number of epochs. There is no early stopping based on validation c-index (results wouldn't be fair).

Thank you for your reply. I have another small problem, while reproducing the code I found that for all the samples in the validation of each fold, the difference in the range of their risk values that I predicted for each validation sample under that fold is very small. For example, case1, risk=-2.33666, case2, risk=-2.33222. but I feel that in this case, case1, risk=-2.33666, case2, risk=-3.48883 is reasonable. Have you encountered such a problem? It would help me a lot if you can answer it? Thanks again.

hazards = torch.sigmoid(h)
survival = torch.cumprod(1 - hazards, dim=1)
risk = -torch.sum(survival, dim=1).detach().cpu().numpy()

[guillaumejaume]

I don't recall having this issue. I agree that having very similar risk scores is surprising. How are the c-index and loss behaving?

[JinchaoChen112]

I don't recall having this issue. I agree that having very similar risk scores is surprising. How are the c-index and loss behaving?

I don't recall having this issue. I agree that having very similar risk scores is surprising. How are the c-index and loss behaving?

Regarding cancer stad, I found that if you use the pre-trained model CTransPath in your article to extract features and then predict survival, the risk values ​​predicted by your model will indeed be different. But switch to resnet101 to extract features and input them into your model. The risk values ​​will be very close. If you choose the maximum C-index to save the model and put the features extracted by CTransPath into your model, the C-index can reach about 0.6. Using resnet101 can reach about 0.62. I'm really a little confused, but the survival curve plotted based on the risk value can be less than 0.05 in some cases. But the small difference in risk value makes me depressed.

[guillaumejaume]

Are you sure that the resnet101 features were correctly extracted -- that'd be my guess

[JinchaoChen112]

Are you sure that the resnet101 features were correctly extracted -- that'd be my guess

Thank you very much for your quick reply. How can I determine whether resnet101 correctly extracted valid features? Do you have any suggestions? Then I noticed that the PORPOISE published by your team used resnet50 to extract image features for survival prediction. As for why resnet101 is used, I think it may extract more detailed image features.

[JinchaoChen112]

I recommend using better feature extractor, such as our recently published UNI (Nature Medicine). Instructions can be found here: https://github.com/mahmoodlab/UNI.

Thank you immensely for the remarkable outcomes of your team's research efforts. I am keenly hopeful that, inspired by your team's achievements, I will be able to attain my own research milestones. Furthermore, I have submitted an application to access UNI's pre-trained model weights, eagerly awaiting approval. Lastly, I am profoundly grateful for any assistance you can provide.

Could we possibly exchange contact details? I believe that having a direct line for academic exchanges could immensely benefit my research and foster potential collaborative opportunities. Your guidance and insights would be invaluable as I navigate through my research journey. Thank you once again for your pioneering work and consideration.