Unable to load onnx models into Triton about clearml-serving

[thepycoder]

Hi @amirhmk!

The model extentions are hardcoded based on the framework type of the model in ClearML.
You have 2 options:

• Override the default filename as in the example here, you'll see the config override in the CLI commands
• Or you can manually specify the model framework when you upload it, set framework argument to 'onnx' and it should properly rename it!

EDIT: I just saw that your framework in clearml is properly set. That's weird and probably a bug on our end. Please check if the first fix works for you, we'll investigate the second :)

Hope this helps! :)

[amirhmk]

Hi @thepycoder, option 1 worked, thanks! Adding default_model_filename: "model.bin" to config.pbtxt and passing it as a file was fine. Though regarding your second point, did onnx models use to have the .bin extention that this is setup like this?

Also as a small aside, I found out that if one passes a file (path) for --aux-config, it'll completely copy paste the passed in config file, and still add the input/output shapes/types/names to the bottom of the final config file - resulting in errors. I thought it would make sense to only pass the config file, and not have to even pass all the shapes one-by-one in the command. Wdyt?

[thepycoder]

option 1 worked, thanks!

Excellent!

Though regarding your second point, did onnx models use to have the .bin extention that this is setup like this?

Possibly yes, this might be a remnant of earlier ONNX days, I'll have to check.

I thought it would make sense to only pass the config file, and not have to even pass all the shapes one-by-one in the command. Wdyt?

Yeah, I think that makes sense, I'll discuss it with the devs :) Thanks a lot for the suggestion!

[ainoam]

I thought it would make sense to only pass the config file, and not have to even pass all the shapes one-by-one in the command. Wdyt?

@amirhmk Aren't these already optional CLI parameters? i.e. you don't have to specify them if their already specified in the config file.

[amirhmk]

@ainoam so most of those are, but the input(or output) shapes/types are mandatory.

[ainoam]

@amirhmk Can you give an example of your CLI invocation, and mark which parameters seem to be unnecessarily mandatory (i.e. also specified in your aux file)

[IlyaMescheryakov1402]

@ainoam #62