KServe wrapper default configuration is faulty about serve

[agunapal]

Hi @sgaist Thanks for the details!
We do have this nightly test https://github.com/pytorch/serve/actions/runs/7999859721

If you can add a test to this, then it would easy to understand and expedite your PR

[sgaist]

Hi @agunapal,

Thanks for the pointer !

I ran the tests and while checking how they were going on, I saw that the PROTOCOL_VERSION environment variable was set properly by the KServe controller (that was the next thing I wanted to fix). Looking at the KServe sources, the fix dates back to September last year.

Seeing that, I verified what I have on my current machine and surprisingly, it's an older version, v0.9.0 to be exact, of the KServe controller that has been deployed. I don't know why that version as I have followed their documentation to deploy the whole system.

This means that the current version of KServe does the right thing when creating the deployment from InferenceService hence my patch would not be strictly required in that context. However I still think that it serves its purpose of making the wrapper's behaviour in line with the base class implementation.

While writing the test, I saw some things that could be improved such as the configuration default handling. I am wondering whether such improvements should be part of a follow up patch or integrated in this one ?

[agunapal]

Hi @sgaist The nightly tests that we run are on nightly images. Hence, you don't see the issue I guess. We will be be making a new release soon.

[agunapal]

Hi @sgaist We welcome any improvements to the code. You can send a PR for one of the issues you want to fix, include a test and we can take it from there

[sgaist]

Good, then I'll open a ticket about the improvements I have in mind for the wrapper on top of what I have done in this one with a follow up patch.