Comments (1)
avitial
commented on June 30, 2024
@Edward-Lin can you share more details about the hardware you are using to run your test? What kind of performance were you expecting to get on this platform with such model? If possible please try the latest OpenVINO 2024.2 version and make sure you have the latest graphics driver as well, and see if that helps improve things on your side.
Performance may vary and depends on the platform used, I suggest to look at Performance Benchmarks to get perf insights for a selection of Large Language Models running on an Intel® Core™ Ultra 7-165H based system. Hope this helps.
from openvino.genai.
Related Issues (20)
- Convert failed using Salesforce/codegen25-7b-multi HOT 2
- Convert failed of baichuan-inc/Baichuan2-13B-Chat HOT 1
- [Improvement] Wrong output size of google/flan-t5-xl HOT 7
- Convert.py breaks for microsoft/trocr-base-printed HOT 3
- Migrate to optimum-cli from llm_bench usage
- Requesting help to understand how TTFT is calculated. Is there any documentation? HOT 3
- MPT-30B met benchmarking issue HOT 1
- ChatGLM3 output token size is not generated as expected HOT 6
- stable diffusion 1.5 generates noise image with "--num"
- Causal LM text generation example errors out HOT 5
- [Good First Issue]: Project won't run because it can't load openvino_tokenizers.dll - even though it exists! HOT 5
- FullyConnected nodes use slow reference kernel on ARM HOT 18
- some regression for benchmark on stable diffusion v1.5 HOT 3
- the quality of generated image is low by benchmark on stable diffusion v1.5 HOT 4
- chatglm3 fails with jsonl input HOT 7
- chatglm output results are repeating with basic prompts HOT 8
- Typo in README.md?
- [GPU] force exit
- `chat_sample.exe` doesn't work well when using `Llama-2-7b-chat-hf`
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openvino.genai.