how to run cvivit trainer or phenaki trainer about phenaki-pytorch

[gmegh]

Hi @lucidrains ! I am also having trouble understanding how both trainer classes relate to the not trainer classes and how to called the trained ones when initializing the other ones.

in that sense, is the readme example code up to date?

thanks so muchh

[gmegh]

I will try over the weekend! Sorry, I have been busy this week with research

[lucidrains]

@gmegh sorry, been busy with some medical stuff, but circling back to video this week to deal some more damage..

yup, i'll write up the examples today, still missing the token critic trainer as well, and will get that done

[lucidrains]

@gmegh i got the cvivit trainer example in the readme now if you'd like to try it out

there's still a few bells and whistles from vision transformers literature that i'd like to apply to cvivit for faster training, but i think you'll see some results even with the vanilla version

i'll add the phenaki and token critic trainers tomorrow

[lucidrains]

ok, was able to get unconditional images and video training working fine for PhenakiTrainer, similar to CViViTTrainer, but still need to wire up the logic for training with text

will need another day

[lucidrains]

@gmegh do you want to try? https://github.com/lucidrains/phenaki-pytorch#phenaki-trainer-wip should be in a better state by weekend's end

[lucidrains]

@gmegh ok! take your time as it is not quite polished yet 😅

[lucidrains]

@9B8DY6 let me know if https://github.com/lucidrains/phenaki-pytorch#phenaki-trainer helps!