Comments (6)
I have a branch that moves more of the processing into native code, I believe it should bring a noticable performance improvement. You can also try 3B models with this version, which should also be much faster. Feel free to try.
Note that the new llama.cpp changes model compatibility, models that used to work with Sherpa probably don't work any more until conversion.
Pull request: #12
apk available: https://github.com/dsd/sherpa/releases/tag/2.2.1-dsd2
from sherpa.
I have a branch that moves more of the processing into native code, I believe it should bring a noticable performance improvement. You can also try 3B models with this version, which should also be much faster. Feel free to try. Note that the new llama.cpp changes model compatibility, models that used to work with Sherpa probably don't work any more until conversion. Pull request: #12 apk available: https://github.com/dsd/sherpa/releases/tag/2.2.1-dsd2
Hi dsd, it works with the apk you provided , but I failed to run it from your forked source.
and when I run on my Mac , it shows "Library not loaded: @rpath/libllama.dylib"
from sherpa.
It's my first time developing Android apps but feel free to share info about the failure to run from source and I will let you know if I have any ideas.
I did not do any work to retain Mac compatibility but I think this is what needs to be done: #12 (comment)
from sherpa.
I have a branch that moves more of the processing into native code, I believe it should bring a noticable performance improvement. You can also try 3B models with this version, which should also be much faster. Feel free to try. Note that the new llama.cpp changes model compatibility, models that used to work with Sherpa probably don't work any more until conversion. Pull request: #12 apk available: https://github.com/dsd/sherpa/releases/tag/2.2.1-dsd2
Is this app using both CPU and GPU of smartphones?
Also, is there any chance to make it run with less RAM like 4gb?
from sherpa.
llama.cpp is used as the backend, so you would need to check if llama.cpp supports your GPU, and if it is usable on 4GB RAM with the model you are interested in.
from sherpa.
llama.cpp is used as the backend, so you would need to check if llama.cpp supports your GPU, and if it is usable on 4GB RAM with the model you are interested in.
Does it support any mobile gpu like mali or adreno?
from sherpa.
Related Issues (20)
- Model file too old HOT 2
- It crashes HOT 7
- How does this work?!?!?!? HOT 2
- what is the folder of 'download folder'? HOT 1
- Add support for k-quant latest models HOT 2
- How to run it on Mac? HOT 1
- Model file not working HOT 3
- libllama.so is 64-bit instead of 32-bit error when trying to load the model on my Samsung Device HOT 2
- Building libllama.so file for Android HOT 2
- Is this app support Llama 2? HOT 1
- Support GGML quantitized models
- [Feature Request] Add ability to edit pre-prompt (and save them)
- Does it support GGUF (instead of GGML)? HOT 3
- App crash on mtk 1080+ 8GB HOT 1
- Add support for MobileLLaMA/MobileVLM models
- Does this support GGUF models? HOT 2
- Support for GGMLv3? HOT 1
- Share link to `ggml-model.bin` HOT 3
- Support for Linux (Debian-class or other) ?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sherpa.