Question about Colab environments affecting results about big-sleep

[rynmurdock]

It could definitely be something in the BigSleep itself, but you may want to try seeing if CLIP's output varies wildly between Colab environments, as I've had issues like this as well -- and using a different PyTorch version seemed to fix it, but I don't know what's going on under the hood in CLIP.

[enricoros]

Have you checked the deterministic flag for CUDA operation and is this affecting your initial issue? I've seen the convergence/not convergence to be a function of the text. I.e. anything puppies converges very easily for me.

[glennamarshall]

In the Story2Hallucination notebook, it asks you to restart runtime after cuda install, then run all cells again - I found that this fixed the problem.

[stedn]

I haven't been able to trace it to anything specific. I've just accepted the process of restarting a few times until it looks good.