I would like the --tensor-parallel-size setting of the vllm backend to be added to yaml. about localai

[localai-bot]

Hello Taikono-Himazin,

Thank you for taking the time to submit an enhancement request to the LocalAI project.

Based on your description, it seems that adding the --tensor-parallel-size setting to the vllm backend configuration through a YAML file would indeed be a useful enhancement. This would allow users to leverage multiple GPUs for inference, which is an essential capability for handling larger workloads.

I will pass your suggestion along to the project maintainers. They will review it and decide whether or not to implement it. You may not receive a direct reply, but rest assured, your request has been noted and will be considered.

Remember, LocalAI is a community-driven project, so if you would like to contribute to its development, please feel free to join the community and helps.

Thank you once again for your feedback, and we look forward to seeing more contributions from you!

---

I am a bot, an experiment of @mudler.