Comments (5)
We have some preliminary support for MOSS on the main branch, and you can compile and run them manually (by specifying --model moss-moon-003-sft
. Currently, they are still using default schedules (that said, very slow). I'll tune the kernel performance by the end of this week, and we will make them available in the next release.
@LeshengJin is working on ChatGLM support.
from mlc-llm.
@yzh119 thanks, hoping for it. Does there any estimates on when will next release happen?
from mlc-llm.
We can make a release as soon as we finish tuning the Dolly and MOSS performance.
from mlc-llm.
how about Chinese-Alpaca-Plus-7B?
is it possible run on mlc-llm?
from mlc-llm.
We have some preliminary support for MOSS on the main branch, and you can compile and run them manually (by specifying
--model moss-moon-003-sft
. Currently, they are still using default schedules (that said, very slow). I'll tune the kernel performance by the end of this week, and we will make them available in the next release.@LeshengJin is working on ChatGLM support.
Hi @LeshengJin, how was the progress going on ChatGLM?
from mlc-llm.
Related Issues (20)
- [Question] Error occurs when I modify Llava to use GPT2's tokenizer HOT 1
- [Question] How to run MLC-LLM as a local chat server on Android HOT 3
- [Bug] FP8 quantization accuracy loss with TinyLlama-1.1B-Chat-v1.0
- [Question]
- [Question] MLCEngine instance must be a singleton? HOT 2
- Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time. HOT 4
- [Question] batchsize of prefill step
- [Bug] InternalError: Check failed: (offset + needed_size <= this->buffer.size) is false: storage allocation failure, attempted to allocate 513024 at offset 0 in region that is 163840bytes HOT 1
- [Question] How to use cpp in project HOT 1
- [Question] How to use function calling in MLCChat Android app? HOT 2
- [Question] Trying to use vulkan version 1.1 library on Qualcomm chip leads to tvm runtime error HOT 4
- [Bug] can run mlc_llm chat on T4 with the model convert with A100 HOT 3
- [Bug] org.apache.tvm.Base$TVMError Check failed: token_id < static_cast<int>(token_table_.size()) HOT 10
- [Question] Can you programmatically clear the kv cache? HOT 2
- [Question] It takes too much time for the first token to be returned after a reqeust issued. HOT 3
- [Question] The Android SDK is too large HOT 4
- [Bug] current available model_libs in dist/lib/libmodel_android.a: [] HOT 4
- [Bug] Threadgroup memory size exceeds the maximum threadgroup memory allowed HOT 4
- [Question] CMake build error, missing variable: _CMAKE_CUDA_WHOLE_FLAG not set HOT 2
- [Bug] "Build Android APP from Source" step2: "mlc_llm package" returns error ""path/to/mlc-llm/android/android/mlc4j" does not exist" HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mlc-llm.