pytorch model & ring attention about lwm

[apexspyche]

What do you have in mind? Is this model suitable for tokenized ecosystem and bridging liquidity and creating a smart algorithm for bridging / blending / mending and growth hacking liquidity across and between multiple TOKENS

[kabachuha]

Lucidrains has a pytorch implementation of RingAttention https://github.com/lucidrains/ring-attention-pytorch

[LzhinFdu]

Lucidrains has a pytorch implementation of RingAttention https://github.com/lucidrains/ring-attention-pytorch

Have you tried this repo? I don’t know whether the experimental results are as expected.
Seems that the model posted on huggingface cannot use it directly to call ring attention.

[LzhinFdu]

What do you have in mind? Is this model suitable for tokenized ecosystem and bridging liquidity and creating a smart algorithm for bridging / blending / mending and growth hacking liquidity across and between multiple TOKENS

I just want to call ring attention when using the trained pytorch LLM for inference.