cambricon / easydk Goto Github PK

Hi Cambricon team,

Where can i find the OSS code such like bootloader/kernel/libraries applications etc. on your github site?

Thanks,

In "samples/classification_runner.hpp", the variable model_ is used to initial class edk::EasyInfer infer_. After that, model_ is used to get the input shape. But in "src/easyinfer/easy_infer.cpp", the EasyInfer::Init() function treat the input parameter std::shared_ptr<ModelLoader> model with std::move like d_ptr_->model_ = std::move(model). Perhaps, it might cause error in debugging "samples/classification_runner.hpp", because model_ is an empty pointer.

Codes from "samples/classification_runner.hpp":

ClassificationRunner::ClassificationRunner(const std::string& model_path, const std::string& func_name,
                                           const std::string& label_path, const std::string& data_path, bool show,
                                           bool save_video)
    : StreamRunner(data_path), show_(show), save_video_(save_video) {
  // load offline model
  model_ = std::make_shared<edk::ModelLoader>(model_path.c_str(), func_name.c_str());

  // prepare mlu memory operator and memory
  mem_op_.SetModel(model_);

  // init easy_infer
  infer_.Init(model_, 0);

  // create mlu resize and convert operator
  auto& in_shape = model_->InputShape(0);

请问是否考虑过将cambricon作为backend集成进nvidia-triton-server。目前paddle-paddle， pytorch， tensorrt作为一种backend集成入了triton。如果从通用度或者生态考虑，是否有机会集成入triton？另外，目前nvidia的做法是Deepstream+triton，其中triton的的地位与infer_server类似，也是可以作为Deepstream的一个module。然而从我使用者的角度看，更希望的是server内部包含Deepstream（infer_server 包含 cnstream），类似于triton中的ensamble模型（emsamble不够灵活），以cnstream增加pipeline构建灵活度，以infer_server解决服务调度等问题，最终以infer_server作为出口，支持rpc/http调用。