Comments (1)
Hello, your image did not upload.
Can you specify your model? What version of Triton you are using etc? Aka the original bug report template here:
Description
A clear and concise description of what the bug is.
Triton Information
What version of Triton are you using?
Are you using the Triton container or did you build it yourself?
To Reproduce
Steps to reproduce the behavior.
Describe the models (framework, inputs, outputs), ideally include the model configuration file (if using an ensemble include the model configuration file for that as well).
Expected behavior
A clear and concise description of what you expected to happen.
from server.
Related Issues (20)
- Support histogram custom metric in Python backend HOT 2
- Backend support for .keras files?
- triton-inference-server cannot be started HOT 1
- Incorrect asset tritonserver2.35.0-jetpack5.1.2-update-2.tgz HOT 1
- How does Triton implement one instance to handle multiple requests simultaneously? HOT 1
- ONNX backend with TensorRT optimizer sometimes fails to start HOT 1
- Any example of triton-vllm with c++ client?
- Tritonserver for FIL backend not starting HOT 1
- Why is my model in ensemble receiving out-of-order input HOT 2
- Add TT-Metalium as a backend
- unexpected datatype TYPE_INT64 for inference input ,expecting TYPE_INT32 HOT 1
- triton malloc fail HOT 7
- Peaks in instantaneous traffic lead to high TP99 inference latency. HOT 2
- Low QPS with momentary traffic surges cause significant increases in inference TP99 latency.
- Single docker layer is too large
- Memory over 100% with decoupled dali video model
- When the request is large, the Triton server has a very high TTFT.
- Uneven QPS leads to low throughput and high latency as well as low GPU utilization HOT 3
- Triton Server 24.05 can't detect CUDA drivers if host system has installed Nvidia driver 555.85
- Building and developing with libtritonserver.so
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from server.