Comments (1)
Thanks for raising this @jithinsarath.
- SkyServe uses Uvicorn at the moment for the controller / load balancer servers. It has not shown up as bottlenecks as of yet due to the compute-heavy nature of GenAI models. Do you foresee for your use cases this may become a bottleneck?
- We'd love to get community's help on this! Encrypting the requests/responses will be great to see in SkyServe.
- Yes, it should be possible. There are two prototypes of this based on Nginx, from @cblmemo:
- #3251 user-side: user simply sets up the auth in YAML; during
curl
, passcurl --location-trusted .. -H "Authorization: Bearer static_secret_token" ..
- #3250 server-side: changes SkyServe to support nginx auth by default
We'd like to know more about your requirements to figure out the best ways forward :) Happy to follow up here or on Slack.
from skypilot.
Related Issues (20)
- [core] `sky launch` fails on k8s with `bash: /opt/conda/bin/ray: No such file or directory`
- [Tests] GCP Image tests failed on latest master HOT 4
- [Core] Support image id when using docker as runtime environment HOT 2
- Central coordination for multiple skypilot cli users HOT 1
- Examples: add examples for Triton, TensorRT-LLM HOT 1
- [k8s example] Add example for putting `~/.sky` on a persistent volume HOT 1
- [Storage] Investigate `rclone mount` with VFS caching HOT 1
- `file_mount` with `mode: COPY` slower than expected on Google Cloud Storage
- [Serve] Service update scales to zero unexpectedly? HOT 1
- [Core][Controller] Respect region/zone settings in controller resources when creating controller
- [Tests] Add smoke tests for AI gallery
- AWS Serving models insufficient permissions of skypilot role HOT 2
- [Controller] Supporting multiple controllers when ClusterOwnerIdentity changes HOT 1
- [k8s] Ingress paths for exposing ports need to be namespaced
- [tests] Allow custom ~/.sky/config.yaml for tests HOT 1
- RunPod skypilot does not allow stopping instances HOT 2
- Runpod cluster created with wrong number of accelerators HOT 2
- Spot instances not supported for runpod HOT 1
- [cudo] Unable to setup credentials on cudo HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from skypilot.