Comments (3)
Would also be good to remove it from server logs as well
from lorax.
Hmm, not able to repro. This is the error I get when using a bad token:
Traceback (most recent call last):
File "/data/lorax/clients/python/lorax/client.py", line 342, in generate_stream
response = StreamResponse(**json_payload)
File "/opt/conda/lib/python3.10/site-packages/pydantic/main.py", line 176, in __init__
self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for StreamResponse
token
Field required [type=missing, input_value={'error': 'Request failed...ror_type': 'generation'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.7/v/missing
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/data/lorax/test_speculate.py", line 54, in <module>
for resp in client.generate_stream(
File "/data/lorax/clients/python/lorax/client.py", line 345, in generate_stream
raise parse_error(resp.status_code, json_payload)
lorax.errors.GenerationError: Request failed during generation: Server error: 401 Client Error. (Request ID: Root=1-664797a7-3da69d07260da6a91fe42e67;c6bd25da-df19-48cf-bb68-770eb0a94ad7)
Repository Not Found for url: https://huggingface.co/api/models/predibase/test-private-lora.
Please make sure you specified the correct `repo_id` and `repo_type`.
If you are trying to access a private or gated repo, make sure you are authenticated.
Invalid username or password.
from lorax.
It's true that the token will show up in logs if provided in the request body, but not if it's inserted into the headers.
from lorax.
Related Issues (20)
- AutoTokenzier.from_pretrains needs setting with `trust_remote_code` inside `load_module_map` HOT 2
- [QUESTION] How to change HuggingFace model download Path in Lorax When deployed to Kubernetes through HelmChart HOT 1
- Bug Report: lorax-launcher failed with --source "s3" for model_id "mistralai/Mistral-7B-Instruct-v0.2" HOT 1
- Improve warmup checking for max new tokens when using speculative decoding
- Support inference on INF2 instance
- Reject unknown fields from API requests
- When caching adapters, cache the adapter ID + the API token pair HOT 4
- Add HTTP status codes to docs HOT 1
- Quantized KV Cache
- `make install` insufficient for running llama3-8B-Instruct HOT 4
- Fail to run Phi-3 HOT 9
- Quickstart example not working HOT 3
- AssertionError when using model "google/gemma-2b" with multi-gpus
- can't run lorax with docker. HOT 1
- Why are qlora (4bit) and lora (16bit) adapter file sizes the same?
- Fail to load special token in phi-3
- Add Support for AutoModelForSequenceClassification Models
- can't start my local llama3 model server with docker
- Important: In latest main, the server can not serve more than 1 user HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lorax.