Comments (1)
Thank you for your inquiry! In our initial efforts, we focused on benchmarking the speed of various open-source Speculative Decoding methods under the same GPU hardware and testing environment. We did not perform additional work to search for the optimal parameters for each specific method; instead, we used the default settings recommended in their respective repositories.
The Spec-Bench platform is designed to avoid speedup variance introduced by differing GPU hardware and software environments (torch & cuda version, etc). Regarding the specific hyper-parameters you mentioned, we believe the best way is to use the optimal hyper-parameters of each method to compare their performance. However, their optimal hyper-parameters may vary with different devices (as Lookahead mentioned). We encourage users to explore and determine the most suitable parameters for their specific setup (the default parameters work well in most scenarios).
from spec-bench.
Related Issues (11)
- Add Hydra HOT 2
- PaSS methodology. HOT 2
- REST methodology verification process HOT 2
- A100的自回归tokens/s仅为40.24是否太慢了? HOT 2
- decoding tokens are not equal for different methods HOT 2
- accuracy of next-token and next-next-token HOT 1
- Error in sps inference HOT 2
- Add IBM speculator models?
- [Feature Request] Add "Recurrent Drafter"
- Can this project support medusa v2?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spec-bench.