Min P generation parameter about text-generation-inference

[avacaondata]

How have you implemented this locally? I'm seeking to do the same , would appreciate some guidance :) @LawrenceGrigoryan

[LawrenceGrigoryan]

@avacaondata
so basically you need to add MinPLogitsWarper to /server/text_generation_server/utils/logits_process.py and the min_p parameter to NextTokenChooser class in /server/text_generation_server/utils/tokens.py

Then go over a ton of files and add the min_p parameter wherever needed. Here I would suggest you searching for one of the existing generation params like top_p and adding the missing lines of code for the new parameter min_p

After you done that, just build the image with the default Dockerfile and use it as you usually do

If you have any further questions, feel free to ask :)