Comments (1)
It looks like your model requires a custom tensorflow operation. That operation is implemented by wrapping a python function. See https://www.tensorflow.org/api_docs/python/tf/py_func.
Currently TRTIS does not support custom tensorflow operations. Only operations that are included in the tensorflow build are included. Can you implement the custom functionality using standard TF operations?
from server.
Related Issues (20)
- How to access the triton inference server from outside of the k8s cluster through ingressroutes? HOT 1
- the queue and time spent processing in the ensemble model is humungous HOT 4
- Can implement the inference process of server interrupt after client interrupt request? HOT 1
- Implementing whole RAG pipeline HOT 3
- Configurable Parallel Model Loading (Python backend) HOT 1
- python backend's environment HOT 1
- Python Backend: How can i add a new labels for all default MetricFamily? HOT 1
- `unable to create cuda shared memory handle` when using multiprocessing to send multiple requests HOT 5
- Can we include commonly used data pre-processing library in triton server docker image? HOT 2
- Multi instance a model in GPU does not increase the throughput in Triton. HOT 1
- Interaction of timeouts, ensemble scheduler and oldest sequence scheduler causes state leakage HOT 15
- error running simple example HOT 14
- How to extract model states stored in Triton (Implicit State Management) HOT 3
- Copy GPU Tensor to CPU on Python backend without using pytorch HOT 2
- conda-pack failing: Failed to initialize Python stub for auto-complete HOT 4
- Dynamic batching that supports static batch size with padding HOT 10
- UNAVAILABLE: Internal: FileNotFoundError: [Errno2] No usable temporary directory found in ['/tp', '/var/tmp','/usr/tmp', '/tmp/python_Lsk/3'] env_6Lp HOT 1
- How does share memory speed up inference? HOT 6
- Memorystore Redis IAM AUTH
- Response caching GPU tensors HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from server.