Failed to export model to ONNX about efficientsam

[yformer]

@ArielleBF, @ximitiejiang, @chenin-wang, @alanzhai219, @fPecc, @kaka-lin, @lxfater, @mchaniotakis, EfficientSAM onnx files are available at Hugging Face Space. The export script and running example are provided. Feel free to give it a try.

[kaka-lin]

@ArielleBF, @ximitiejiang, @chenin-wang, @alanzhai219, @fPecc, @kaka-lin, @lxfater, @mchaniotakis, EfficientSAM onnx files are available at Hugging Face Space. The export script and running example are provided. Feel free to give it a try.

Thanks yformer ~

I also create TensorFlow2.x version and coverted to tflite model
If anyone need to use tflite to inference please check here EfficientSAM-tf2-demo.

Thanks!!!

[yacineMTB]

Thanks yformer <3
FYI for other readers; the export tooling is on this repository

EfficientSAM/export_to_onnx.py

Lines 63 to 78 in c9408a7

	def export_onnx_esam_encoder(model, output):
	onnx_model = onnx_models.OnnxEfficientSamEncoder(model=model)
	dynamic_axes = {
	"batched_images": {0: "batch", 2: "height", 3: "width"},
	}
	dummy_inputs = {
	"batched_images": torch.randn(1, 3, 1080, 1920, dtype=torch.float),
	}
	output_names = ["image_embeddings"]
	export_onnx(
	onnx_model=onnx_model,
	output=output,
	dynamic_axes=dynamic_axes,
	dummy_inputs=dummy_inputs,
	output_names=output_names,
	)

[ximitiejiang]

same question.

[yformer]

@ArielleBF @ximitiejiang can you share more information on the export? I will try to export one and share it.

[ArielleBF]

@ArielleBF @ximitiejiang can you share more information on the export? I will try to export one and share it.
When I try to export the image_encoder model using torch.onnx.export, a bug occurred.It is possible that the bug is caused by incompatibility between the jit format file and onnx during the export.

[ximitiejiang]

hello, i tried to export onnx with below codes which are similiar with SAM

def export_onnx():
    model = torch.jit.load('efficientsam_s_gpu.jit')
    output = "efficientsam_s_gpu.onnx"
    output_names = ["masks_predictions", "iou_predictions"]
    dummy_inputs = {
        "images": torch.randn((1, 3, 1024, 1024), dtype=torch.float),
        "point_coords": torch.randint(low=0, high=1024, size=(1, 1, 5, 2), dtype=torch.float),
        "point_labels": torch.randint(low=0, high=4, size=(1, 1, 5), dtype=torch.float)
    }
    dynamic_axes = {
        # "images": {2: "img_height", 3: "img_width"},
        "point_coords": {2: "num_points"},
        "point_labels": {2: "num_points"},
    }
    opset = 14
    with open(output, "wb") as f:
        print(f"Exporting onnx model to {output}...")
        torch.onnx.export(
            model,
            tuple(dummy_inputs.values()),
            f,
            export_params=True,
            verbose=False,
            opset_version=opset,
            do_constant_folding=True,
            input_names=list(dummy_inputs.keys()),
            output_names=output_names,
            dynamic_axes=dynamic_axes,
        )

and i got error as below:
torch.onnx.symbolic_registry.UnsupportedOperatorError: Exporting the operator ::tile to ONNX opset version 14 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub.

[yformer]

@ArielleBF @ximitiejiang thanks for sharing the information. We will be trying to export one.

[chenin-wang]

same question.

[alanzhai219]

same question.

[fPecc]

Same problem here trying to export 'efficientsam_ti_gpu.jit' to ONNX using PyTorch 2.0.1 and opset_version set to 18.

[kaka-lin]

Same problem

[lxfater]

Same problem

[mchaniotakis]

The problem originates from torch.tile here:

        # Tile the image embedding for all queries.
        image_embeddings_tiled = torch.tile(
            image_embeddings[:, None, :, :, :], [1, max_num_queries, 1, 1, 1]
        ).view(
            batch_size * max_num_queries,
            image_embed_dim_c,
            image_embed_dim_h,
            image_embed_dim_w,
        )

This could probably be solved by replacing torch.tile() with tensor.repeat() or using a symbolic for onnx to patch it