xunboo / llama2win

Run baby Llama 2 model in windows

llama2win

Run baby Llama 2 model in windows

llama2win.exe model.bin

Run exe @ AMD Ryzen 7 PRO 5850U

Once upon a time, there was a big fish named Bubbles. Bubbles had a best friend named Sally, who was a small fish. Sally loved to cook yummy food for Bubbles. One day, Sally brought some jelly for Bubbles to eat. Bubbles was very happy and ate the jelly quickly. Suddenly, the jelly fell on Sally's face and blew her hair! Sally was surprised and didn't know what to do. But then she started to laugh. Bubbles realized that it was okay to be different and have other friends. From then on, Bubbles and Sally cooked jelly together and had lots of fun. The end.

Once upon a time, there was a clever cat named Mittens. Mittens loved to play with yarn and chase mice. One day, Mittens met a sly dog named Max. Max asked Mittens where he got the yarn. Mittens replied, "I found it on the floor!" Max was not happy with Mittens' answer. He wanted to take the yarn back, but Mittens didn't want to give it back. They started to argue

Without OPENMP support: achieved tok/s: 82.421120

With OPENMP support: achieved tok/s: 91.855041