onnx / onnx-mlir

Representation and Reference Lowering of ONNX Models in MLIR Compiler Infrastructure

Open Neural Network Exchange (ONNX) is an open ecosystem that empowers AI developers to choose the right tools as their project evolves. ONNX provides an open source format for AI models, both deep learning and traditional ML. It defines an extensible computation graph model, as well as definitions of built-in operators and standard data types. Currently we focus on the capabilities needed for inferencing (scoring).

ONNX is widely supported and can be found in many frameworks, tools, and hardware. Enabling interoperability between different frameworks and streamlining the path from research to production helps increase the speed of innovation in the AI community. We invite the community to join us and further evolve ONNX.

Use ONNX

• Documentation of ONNX Python Package
• Tutorials for creating ONNX models
• Pre-trained ONNX models

Learn about the ONNX spec

• Overview
• ONNX intermediate representation spec
• Versioning principles of the spec
• Operators documentation
• Operators documentation (latest release)
• Python API Overview

Programming utilities for working with ONNX Graphs

• Shape and Type Inference
• Graph Optimization
• Opset Version Conversion

Contribute

ONNX is a community project and the open governance model is described here. We encourage you to join the effort and contribute feedback, ideas, and code. You can participate in the Special Interest Groups and Working Groups to shape the future of ONNX.

Check out our contribution guide to get started.

If you think some operator should be added to ONNX specification, please read this document.

Community meetings

The schedules of the regular meetings of the Steering Committee, the working groups and the SIGs can be found here

Community Meetups are held at least once a year. Content from previous community meetups are at:

• 2020.04.09 https://wiki.lfaidata.foundation/display/DL/LF+AI+Day+-ONNX+Community+Virtual+Meetup+-+Silicon+Valley+-+April+9
• 2020.10.14 https://wiki.lfaidata.foundation/display/DL/LF+AI+Day+-+ONNX+Community+Workshop+-+October+14
• 2021.03.24 https://wiki.lfaidata.foundation/pages/viewpage.action?pageId=35160391
• 2021.10.21 https://wiki.lfaidata.foundation/pages/viewpage.action?pageId=46989689
• 2022.06.24 https://wiki.lfaidata.foundation/display/DL/ONNX+Community+Day+-+June+24
• 2023.06.28 https://wiki.lfaidata.foundation/display/DL/ONNX+Community+Day+2023+-+June+28

Discuss

We encourage you to open Issues, or use Slack (If you have not joined yet, please use this link to join the group) for more real-time discussion.

Follow Us

Stay up to date with the latest ONNX news. [Facebook] [Twitter]

Roadmap

A roadmap process takes place every year. More details can be found here

Installation

Official Python packages

ONNX released packages are published in PyPi.

pip install onnx  # or pip install onnx[reference] for optional reference implementation dependencies

ONNX weekly packages are published in PyPI to enable experimentation and early testing.

vcpkg packages

onnx is in the maintenance list of vcpkg, you can easily use vcpkg to build and install it.

git clone https://github.com/microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.bat # For powershell
./bootstrap-vcpkg.sh # For bash
./vcpkg install onnx

Conda packages

A binary build of ONNX is available from Conda, in conda-forge:

conda install -c conda-forge onnx

Build ONNX from Source

Before building from source uninstall any existing versions of onnx pip uninstall onnx.

c++17 or higher C++ compiler version is required to build ONNX from source. Still, users can specify their own CMAKE_CXX_STANDARD version for building ONNX.

If you don't have protobuf installed, ONNX will internally download and build protobuf for ONNX build.

Or, you can manually install protobuf C/C++ libraries and tools with specified version before proceeding forward. Then depending on how you installed protobuf, you need to set environment variable CMAKE_ARGS to "-DONNX_USE_PROTOBUF_SHARED_LIBS=ON" or "-DONNX_USE_PROTOBUF_SHARED_LIBS=OFF". For example, you may need to run the following command:

Linux:

export CMAKE_ARGS="-DONNX_USE_PROTOBUF_SHARED_LIBS=ON"

Windows:

set CMAKE_ARGS="-DONNX_USE_PROTOBUF_SHARED_LIBS=ON"

The ON/OFF depends on what kind of protobuf library you have. Shared libraries are files ending with *.dll/*.so/*.dylib. Static libraries are files ending with *.a/*.lib. This option depends on how you get your protobuf library and how it was built. And it is default OFF. You don't need to run the commands above if you'd prefer to use a static protobuf library.

Windows

If you are building ONNX from source, it is recommended that you also build Protobuf locally as a static library. The version distributed with conda-forge is a DLL, but ONNX expects it to be a static library. Building protobuf locally also lets you control the version of protobuf. The tested and recommended version is 3.21.12.

The instructions in this README assume you are using Visual Studio. It is recommended that you run all the commands from a shell started from "x64 Native Tools Command Prompt for VS 2019" and keep the build system generator for cmake (e.g., cmake -G "Visual Studio 16 2019") consistent while building protobuf as well as ONNX.

You can get protobuf by running the following commands:

git clone https://github.com/protocolbuffers/protobuf.git
cd protobuf
git checkout v21.12
cd cmake
cmake -G "Visual Studio 16 2019" -A x64 -DCMAKE_INSTALL_PREFIX=<protobuf_install_dir> -Dprotobuf_MSVC_STATIC_RUNTIME=OFF -Dprotobuf_BUILD_SHARED_LIBS=OFF -Dprotobuf_BUILD_TESTS=OFF -Dprotobuf_BUILD_EXAMPLES=OFF .
msbuild protobuf.sln /m /p:Configuration=Release
msbuild INSTALL.vcxproj /p:Configuration=Release

Then it will be built as a static library and installed to <protobuf_install_dir>. Please add the bin directory(which contains protoc.exe) to your PATH.

set CMAKE_PREFIX_PATH=<protobuf_install_dir>;%CMAKE_PREFIX_PATH%

Please note: if your protobuf_install_dir contains spaces, do not add quotation marks around it.

Alternative: if you don't want to change your PATH, you can set ONNX_PROTOC_EXECUTABLE instead.

set CMAKE_ARGS=-DONNX_PROTOC_EXECUTABLE=<full_path_to_protoc.exe>

Then you can build ONNX as:

git clone https://github.com/onnx/onnx.git
cd onnx
git submodule update --init --recursive
# prefer lite proto
set CMAKE_ARGS=-DONNX_USE_LITE_PROTO=ON
pip install -e . -v

Linux

First, you need to install protobuf. The minimum Protobuf compiler (protoc) version required by ONNX is 3.6.1. Please note that old protoc versions might not work with CMAKE_ARGS=-DONNX_USE_LITE_PROTO=ON.

Ubuntu 20.04 (and newer) users may choose to install protobuf via

apt-get install python3-pip python3-dev libprotobuf-dev protobuf-compiler

In this case, it is required to add -DONNX_USE_PROTOBUF_SHARED_LIBS=ON to CMAKE_ARGS in the ONNX build step.

A more general way is to build and install it from source. See the instructions below for more details.

Installing Protobuf from source

Debian/Ubuntu:

  git clone https://github.com/protocolbuffers/protobuf.git
  cd protobuf
  git checkout v21.12
  git submodule update --init --recursive
  mkdir build_source && cd build_source
  cmake ../cmake -Dprotobuf_BUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_INSTALL_SYSCONFDIR=/etc -DCMAKE_POSITION_INDEPENDENT_CODE=ON -Dprotobuf_BUILD_TESTS=OFF -DCMAKE_BUILD_TYPE=Release
  make -j$(nproc)
  make install

CentOS/RHEL/Fedora:

  git clone https://github.com/protocolbuffers/protobuf.git
  cd protobuf
  git checkout v21.12
  git submodule update --init --recursive
  mkdir build_source && cd build_source
  cmake ../cmake  -DCMAKE_INSTALL_LIBDIR=lib64 -Dprotobuf_BUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_INSTALL_SYSCONFDIR=/etc -DCMAKE_POSITION_INDEPENDENT_CODE=ON -Dprotobuf_BUILD_TESTS=OFF -DCMAKE_BUILD_TYPE=Release
  make -j$(nproc)
  make install

Here "-DCMAKE_POSITION_INDEPENDENT_CODE=ON" is crucial. By default static libraries are built without "-fPIC" flag, they are not position independent code. But shared libraries must be position independent code. Python C/C++ extensions(like ONNX) are shared libraries. So if a static library was not built with "-fPIC", it can't be linked to such a shared library.

Once build is successful, update PATH to include protobuf paths.

Then you can build ONNX as:

git clone https://github.com/onnx/onnx.git
cd onnx
git submodule update --init --recursive
# Optional: prefer lite proto
export CMAKE_ARGS=-DONNX_USE_LITE_PROTO=ON
pip install -e . -v

Mac

export NUM_CORES=`sysctl -n hw.ncpu`
brew update
brew install autoconf && brew install automake
wget https://github.com/protocolbuffers/protobuf/releases/download/v21.12/protobuf-cpp-3.21.12.tar.gz
tar -xvf protobuf-cpp-3.21.12.tar.gz
cd protobuf-3.21.12
mkdir build_source && cd build_source
cmake ../cmake -Dprotobuf_BUILD_SHARED_LIBS=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON -Dprotobuf_BUILD_TESTS=OFF -DCMAKE_BUILD_TYPE=Release
make -j${NUM_CORES}
make install

Once build is successful, update PATH to include protobuf paths.

Then you can build ONNX as:

git clone --recursive https://github.com/onnx/onnx.git
cd onnx
# Optional: prefer lite proto
set CMAKE_ARGS=-DONNX_USE_LITE_PROTO=ON
pip install -e . -v

Verify Installation

After installation, run

python -c "import onnx"

to verify it works.

Common Build Options

For full list refer to CMakeLists.txt

Environment variables

• USE_MSVC_STATIC_RUNTIME should be 1 or 0, not ON or OFF. When set to 1 onnx links statically to runtime library. Default: USE_MSVC_STATIC_RUNTIME=0

• DEBUG should be 0 or 1. When set to 1 onnx is built in debug mode. or debug versions of the dependencies, you need to open the CMakeLists file and append a letter d at the end of the package name lines. For example, NAMES protobuf-lite would become NAMES protobuf-lited. Default: Debug=0

CMake variables

• ONNX_USE_PROTOBUF_SHARED_LIBS should be ON or OFF. Default: ONNX_USE_PROTOBUF_SHARED_LIBS=OFF USE_MSVC_STATIC_RUNTIME=0 ONNX_USE_PROTOBUF_SHARED_LIBS determines how onnx links to protobuf libraries.

  • When set to ON - onnx will dynamically link to protobuf shared libs, PROTOBUF_USE_DLLS will be defined as described here, Protobuf_USE_STATIC_LIBS will be set to OFF and USE_MSVC_STATIC_RUNTIME must be 0.
  • When set to OFF - onnx will link statically to protobuf, and Protobuf_USE_STATIC_LIBS will be set to ON (to force the use of the static libraries) and USE_MSVC_STATIC_RUNTIME can be 0 or 1.

• ONNX_USE_LITE_PROTO should be ON or OFF. When set to ON onnx uses lite protobuf instead of full protobuf. Default: ONNX_USE_LITE_PROTO=OFF

• ONNX_WERROR should be ON or OFF. When set to ON warnings are treated as errors. Default: ONNX_WERROR=OFF in local builds, ON in CI and release pipelines.

Common Errors

• Note: the import onnx command does not work from the source checkout directory; in this case you'll see ModuleNotFoundError: No module named 'onnx.onnx_cpp2py_export'. Change into another directory to fix this error.

• If you run into any issues while building Protobuf as a static library, please ensure that shared Protobuf libraries, like libprotobuf, are not installed on your device or in the conda environment. If these shared libraries exist, either remove them to build Protobuf from source as a static library, or skip the Protobuf build from source to use the shared version directly.

• If you run into any issues while building ONNX from source, and your error message reads, Could not find pythonXX.lib, ensure that you have consistent Python versions for common commands, such as python and pip. Clean all existing build files and rebuild ONNX again.

Testing

ONNX uses pytest as test driver. In order to run tests, you will first need to install pytest:

pip install pytest nbval

After installing pytest, use the following command to run tests.

pytest

Development

Check out the contributor guide for instructions.

License

Apache License v2.0

Code of Conduct

ONNX Open Source Code of Conduct

onnx-mlir's Issues

ONNX Operation Status (Update as you work on ONNX Operations)

Please keep up to date.

• As you start working on a new operation, move it from the lower table to the top table,
• write your name under 'person working on it,'
• once you have pushed the functionality in the master branch, mark it as complete.

ONNX to KRNL work.

Update as you push code to the master branch.

ONNX Operation	Person working on it	ONNX Def	Shape Inference	ONNX to KRNL	Optimized	Extended Functionality
Abs		v	v	v
Add		v	v	v		M
And		v	v	v		M
AveragePool	Tung	v	v	v		nP
BatchNorm		v	v	v
Constant		v	v	v
Conv		v	v	v		nP
Cos		v	v	v
Cosh		v	v	v
Div		v	v	v		M
Elu		v	v	v
Exp		v	v	v
Gemm		v	v	v		U
HardSigmoid		v	v	v
Identity		v	v	v
LeakyRelu		v	v	v
Log		v	v	v
MatMul		v	v	v		noM
Max		v	v	v		M V
MaxPool		v	v	v		M noIdx
Min		v	v	v		M V
Mul		v	v	v		noM
Or		v	v	v		M
Pad		v	V	v		const only
Reciprocal		v	v	v
ReduceMax		v	v	v
ReduceMin		v	v	v
ReduceProd		v	v	v
ReduceSum		v	v	v
Relu		v	v	v
Reshape		v	v	v		const only
Selu		v	v	v
Sigmoid		v	v	v
Sign		v	v	v
Sinh		v	v	v
Softmax		v	v	v
Softplus		v	v	v
Softsign		v	v	v
Sqrt		v	v	v
Sub		v	v	v		M
Sum		v	v	v		M V
Tanh		v	v	v
Transpose		v	v	v
Unsqueeze		v	v	v
Xor		v	v	v		M

Extended functionality info (noXXX: additional work is needed).

• M / noM: multi-broadcast imlemented / not implemented
• U / noU: unidir-broadcast implemented / not implemented
• P / noP: padding implemented / not implemented
• V / noV: variadic input implemented / non implemented

Operation not supported yet.

Please pick new operations to work on, and make sure you utilize current code to minimize additional work needed to support new functionality.

Typical sequence of work.

1. Ensure the operation is properly parsed from ONNX and MLIR input.
2. Perform shape inference.
3. Lower to KRNL dialect.

And add literal tests at each step, and end to end tests once completed.

ONNX Operation	Person working on it	ONNX Def	Shape Inference	ONNX to KRNL	Basic Functionality	Extended functionality
ACos
ACosh
ArgMax
ArgMin
ASin
ASinh
ATan
ATanh
BitShift						M
Ceil
Celu
Clip
Compress
Concat						V
ConcatFromSeq
ConstOfShape
ConvInt						P
ConvTrans						P
CumSum
DepthToSpace
DequantizeLin
Det
Dropout
DynQuantizeLin
Einsum						V
Equal						M
Erf
Expand
EyeLike
Flatten
Floor
GRU
GatherElements
GatherND
GlobalAvgPool
GlobalLpPool
GlobalMaxPool
Greater						M
GreaterOrEq						M
HardMax
If						V
InstanceNorm
Inverse
IsInf
IsNaN
Less						M
LessOrEq						M
LSTM
LSTM
LSTM
LogSoftmax
Loop						V
LpNorm
LpPool
MatMulInteger						U
MaxRoiPool
MaxUnpool
Mean						M V
MeanSquareDist						M
MeanVarNorm
Mod						M
Multinomial
Neg
NegLogLikeLoss
NonMaxSupress
NonZero
Not
OneHot
PRelu						U
Power						M
QLinearConv						P
QLinearMatMul						M
QuantizeLinear
RNN
RandomNormal
RandNormalLike
RandUniform
RandUniformLike
Range
ReduceL1
ReduceL2
ReduceLogSum
ReduceLogSumExp
ReduceMean
ReduceSumSquare
Resize
ReverseSequence
RoiAlign
Round
Scan						V
Scatter						deprecated for Scatter Element
ScatterElem
ScatterND
SequenceAt
SeqConstruct						V
SequenceErase
SequenceInsert
SequenceLength
Shape
Shrink
Slice
SoftMaxCross
SpaceToDepth
Split						V
SplitToSequence
Squeeze
StringNorm
Tan
TfldfVect
ThresholdRelu
Tile
TopK
UnfoldToDepth
Unique
Upsample						deprecated
Where						M

[RFC] Opset Version Support

To summarize our discussion on our support for ONNX opset versions, I'm opening up an RFC thread to discuss and track the status of implementation.

Motivation:

ONNX is a standard consisting of the semantic definitions of operations commonly used in ML/AI context. Since the standard (and the underlying field) is rapidly evolving, some definitions go out of date, and to distinguish between new/old definitions, a unique version is given to every definition (including the schema) of an operation.

However, the current state of our infrastructure only supports the latest of many definitions, meaning our support for the ONNX standard is partial. The reason leading up to our partial support is that implementing/maintaining operation lowering support entails expensive endeavor.

Therefore, a proposal to expand our operation lowering support to cover the entirety of ONNX standard must reconcile with the fact that we cannot provide lowering support for all versions of operation semantics/schemas.

Proposed Solution:

• We continue to support the lowering of only the latest version of every operation, we will continue to name these operations without any version prefix/suffix (e.g., onnx.Add).
• We support the representation (thus schema) of all versions of operation, and the older versions of operations will be named with a "-{version}" suffix (e.g., onnx.Add-6).
• For older versions of operations, we convert them, upon encountering, to their corresponding definition/schema of newer versions by invoking pattern-based rewrite rules. Thus, transitively, we will support the lowering of older versions of operations (via a workflow of operation in old schema -Rewrite-> operation in new schema -Lower-> MLIR Dialect/LLVM)

The naming of versioned operations is consistent with that appearing in the ONNX standard (c.f. https://github.com/onnx/onnx/blob/master/docs/Changelog.md#Add-6)

Caveat:

One caveat of such a naming scheme is that existing IR files may go out of date as new versions of operation are introduced. This problem can be mitigated by attaching the current latest onnx opset version as an attribute to modules containing ONNX dialect.

Registration and linking of dialects in onnx-mlir-opt

With a more recent version of MLIR, it is no longer necessary to always link in the MLIR dialects:

onnx-mlir/src/Tool/ONNXMLIROpt/CMakeLists.txt

Line 9 in 46ef799

whole_archive_link_mlir(onnx-mlir-opt ${MLIRWholeArchiveLibs})

Further, there seems to be no need to include InitAllDialects.h, as the function registerAllDialects() isn't called anyway. Are you interested in a PR?

onnx-mlir building error

Hi,
I'm using the llvm github from https://github.com/clang-ykt/llvm-project and build llvm successfully. But build the onnx-mlir with the following errors:
[ 22%] Built target OMShapeInferenceOpInterface
In file included from /docker/liujp/llvm-project/mlir/include/mlir/Transforms/DialectConversion.h:16:0,
from /docker/liujp/onnx-mlir/src/Dialect/Krnl/KrnlHelper.hpp:21,
from /docker/liujp/onnx-mlir/src/Dialect/Krnl/KrnlOps.hpp:20,
from /docker/liujp/onnx-mlir/src/Dialect/Krnl/KrnlHelper.cpp:15:
/docker/liujp/llvm-project/mlir/include/mlir/IR/PatternMatch.h: In instantiation of ‘OpTy mlir::PatternRewriter::create(mlir::Location, Args ...) [with OpTy = mlir::KrnlDefineLoopsOp; Args = {int}]’:
/docker/liujp/onnx-mlir/src/Dialect/Krnl/KrnlHelper.cpp:182:73: required from here
/docker/liujp/llvm-project/mlir/include/mlir/IR/PatternMatch.h:221:16: error: no matching function for call to ‘mlir::KrnlDefineLoopsOp::build(mlir::PatternRewriter&, mlir::OperationState&, int&)’
OpTy::build(*this, state, args...);
^

Could you have a check?

Many thanks!

ljppro

Support unknown dimension in ONNXPadConstantValuePadOp's shape-inference

It looks like ONNXPadConstantValuePadOp's shape inference did not take care of unknown dimensions.

Example program:

func @main_graph(%arg0: tensor<?x?x?x?xf32>) -> tensor<*xf32> {
  %0 = "onnx.PadConstantValuePad"(%arg0) {constant_value = 0.000000e+00 : f32, mode = "constant", pads = [0, 0, 2, 2, 0, 0, 2, 2]} : (tensor<?x?x?x?xf32>) -> tensor<*xf32>
  return %0 : tensor<*xf32>
}

Run with shape-inference pass: onnf-opt --shape-inference test.mlir, output is:

module {
  func @main_graph(%arg0: tensor<?x?x?x?xf32>) -> tensor<?x3x?x3xf32> {
    %0 = "onnx.PadConstantValuePad"(%arg0) {constant_value = 0.000000e+00 : f32, mode = "constant", pads = [0, 0, 2, 2, 0, 0, 2, 2]} : (tensor<?x?x?x?xf32>) -> tensor<?x3x?x3xf32>
    return %0 : tensor<?x3x?x3xf32>
  }
}

The correct output shape should be: <?x?x?x?> instead of <?x3x?x3> because we did not have actual dimension values from the input to infer actual dimension values for the output.

This issue would happen with ONNXPadConstantPadOp also since it uses the same helper function as ONNXPadConstantValuePadOp's for shape inference.

Transform batch norm pattern

Batch norm is defined in ONNX as

Y = (X - mean) * scale / Sqrt(var+eps) + beta

when mean and var (which are input variable in ONNX) are seen as constant, and scale & beta are parameters, so always constant, we can precompute

a = scale / Sqrt(var+eps)
b = - mean * scale / Sqrt(var+eps) + beta

and then the operation reduces to

Y = a * X + b

Not able to build ONNX

This happens when I run the "cmake .." command, I have already built the llvm project (For this I have used the instructions on the MLIR getting started page : )) and set the environment variable to the right path.

and this is the ouput of the CMakeError.log
`Performing C SOURCE FILE Test CMAKE_HAVE_LIBC_PTHREAD failed with the following output:
Change Dir: /home/akhil/Downloads/onnx-mlir/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_c1976/fast && /usr/bin/make -f CMakeFiles/cmTC_c1976.dir/build.make CMakeFiles/cmTC_c1976.dir/build
make[1]: Entering directory '/home/akhil/Downloads/onnx-mlir/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_c1976.dir/src.c.o
/usr/bin/cc -DCMAKE_HAVE_LIBC_PTHREAD -o CMakeFiles/cmTC_c1976.dir/src.c.o -c /home/akhil/Downloads/onnx-mlir/build/CMakeFiles/CMakeTmp/src.c
Linking C executable cmTC_c1976
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_c1976.dir/link.txt --verbose=1
/usr/bin/cc -DCMAKE_HAVE_LIBC_PTHREAD -rdynamic CMakeFiles/cmTC_c1976.dir/src.c.o -o cmTC_c1976
/usr/bin/ld: CMakeFiles/cmTC_c1976.dir/src.c.o: in function main': src.c:(.text+0x46): undefined reference to pthread_create'
/usr/bin/ld: src.c:(.text+0x52): undefined reference to pthread_detach' /usr/bin/ld: src.c:(.text+0x63): undefined reference to pthread_join'
collect2: error: ld returned 1 exit status
make[1]: *** [CMakeFiles/cmTC_c1976.dir/build.make:87: cmTC_c1976] Error 1
make[1]: Leaving directory '/home/akhil/Downloads/onnx-mlir/build/CMakeFiles/CMakeTmp'
make: *** [Makefile:121: cmTC_c1976/fast] Error 2

Source file was:
#include <pthread.h>

void* test_func(void* data)
{
return data;
}

int main(void)
{
pthread_t thread;
pthread_create(&thread, NULL, test_func, NULL);
pthread_detach(thread);
pthread_join(thread, NULL);
pthread_atfork(NULL, NULL, NULL);
pthread_exit(NULL);

return 0;
}

Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /home/akhil/Downloads/onnx-mlir/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_a5dee/fast && /usr/bin/make -f CMakeFiles/cmTC_a5dee.dir/build.make CMakeFiles/cmTC_a5dee.dir/build
make[1]: Entering directory '/home/akhil/Downloads/onnx-mlir/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_a5dee.dir/CheckFunctionExists.c.o
/usr/bin/cc -DCHECK_FUNCTION_EXISTS=pthread_create -o CMakeFiles/cmTC_a5dee.dir/CheckFunctionExists.c.o -c /usr/share/cmake-3.16/Modules/CheckFunctionExists.c
Linking C executable cmTC_a5dee
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_a5dee.dir/link.txt --verbose=1
/usr/bin/cc -DCHECK_FUNCTION_EXISTS=pthread_create -rdynamic CMakeFiles/cmTC_a5dee.dir/CheckFunctionExists.c.o -o cmTC_a5dee -lpthreads
/usr/bin/ld: cannot find -lpthreads
collect2: error: ld returned 1 exit status
make[1]: *** [CMakeFiles/cmTC_a5dee.dir/build.make:87: cmTC_a5dee] Error 1
make[1]: Leaving directory '/home/akhil/Downloads/onnx-mlir/build/CMakeFiles/CMakeTmp'
make: *** [Makefile:121: cmTC_a5dee/fast] Error 2
`

KRNL Lowering for CPUs with Optimizations

Optimizations for GEMM and CONV

GEMM:

• Tiling
• SIMD code gen
• Memory Layouts
• Good instructions mix for Z

CONV:

• Tiling
• SIMD code gen
• Memory Layouts
• Good instructions mix for Z

Build onnx-mlir error: MLIRMlirOptMain NOTFOUND

I bumped into an error while running for build onnx-mlir:
export LLVM_PROJ_SRC=/mypath/llvm-project/
export LLVM_PROJ_BUILD=/mypath/llvm-project/build
mkdir onnx-mlir/build && cd onnx-mlir/build
cmake ..

The error is:

-- Configuring done
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
MLIRMlirOptMain
linked by target "compiler" in directory /mypath/onnx-mlir/src
linked by target "compiler" in directory /mypath/onnx-mlir/src
linked by target "onnf_onnx_decompose" in directory /mypath/onnx-mlir/src
linked by target "onnf_onnx_decompose" in directory /mypath/onnx-mlir/src
linked by target "onnf_shape_inference" in directory /mypath/onnx-mlir/src
linked by target "onnf_shape_inference" in directory /mypath/onnx-mlir/src
linked by target "onnf_lower_frontend" in directory /mypath/onnx-mlir/src
linked by target "onnf_lower_frontend" in directory /mypath/onnx-mlir/src
linked by target "onnf" in directory /mypath/onnx-mlir/src
linked by target "onnf" in directory /mypath/onnx-mlir/src
linked by target "onnf_transform" in directory /mypath/onnx-mlir/src/transform
linked by target "onnf_transform" in directory /mypath/onnx-mlir/src/transform
linked by target "onnf-opt" in directory /mypath/onnx-mlir/src/tool/onnf_opt
linked by target "onnf-opt" in directory /mypath/onnx-mlir/src/tool/onnf_opt
linked by target "builder" in directory /mypath/onnx-mlir/src/builder
linked by target "builder" in directory /mypath/onnx-mlir/src/builder
-- Generating done
CMake Generate step failed. Build files cannot be regenerated correctly.

I have already built llvm successfully and installed gcc, cmake and libprotoc with ZLib.
Why I encountered this error? Did I forget to install or export something?

Thanks!

Add unit test & lit tests for Neg op.

Related to #108.

Add guidance regarding usage of affine.load/store.

zlib link issue with cmake --build . --target onnx-mlir

I am seeing following issue when building onnx-mlir

[ 98%] Built target OMShapeInference
[ 98%] Linking CXX executable ../bin/onnx-mlir
/usr/bin/ld: </path/to/workspace>/llvm-project/build/lib/libLLVMSupport.a(Compression.cpp.o): undefined reference to symbol 'crc32'
/usr/lib/x86_64-linux-gnu/libz.so: error adding symbols: DSO missing from command line

It seems above path to libz.so is returned by cmake command find_package(ZLIB REQUIRED) (in src folder). Any suggestions ? (eg. do I need to update cmake find_package search path, or use static lib ? )

Thanks,
Yetanadur
(Ananth)

Naive cuda support

Will onnx-mlir support using naive cuda kernel to write operator kernels?

Missing mlir-opt options in onnx-mlir-opt

Just noticed today that we are missing some mlir-opt options in onnx-mlir-opt. This issue is a reminder to register some of the missing mlir-opt passes in onnx-mlir-opt.

check-onnx-backend link error

Any idea about the fix?

`
make check-onnx-backend

...
/usr/bin/ld: cannot find -lEmbeddedDataLoader
/usr/bin/ld: cannot find -lcruntime
collect2: error: ld returned 1 exit status
g++ -shared -fPIC temp_model.o /tmp/packed_const-a52592.tmp.o -o temp_model.so -lEmbeddedDataLoader -lcruntime
Command execution failed.
UNREACHABLE executed at /home/chentong/onnx-mlir/src/MainUtils.cpp:94!
Esssssssss/usr/bin/ld: cannot find -lEmbeddedDataLoader
/usr/bin/ld: cannot find -lcruntime
collect2: error: ld returned 1 exit status
...
`

Semantics of ONNXPadConstantValuePad

I wonder what is the current semantics of ONNXPadConstantValuePad.

From a test about ONNXPadConstantValuePad here: https://github.com/onnx/onnx-mlir/blob/master/test/mlir/onnx/onnx_shape_inference.mlir#L279

func @test_PadConstantValuePad_1(%arg0 : tensor<16x13xf32>) -> tensor<*xf32> {
  %0 = "onnx.PadConstantValuePad"(%arg0) {constant_value = 0.000000e+00 : f32, mode = "constant", pads = [0, 2, 0, 0]} : (tensor<16x13xf32>) -> tensor<*xf32>
  "std.return"(%0) : (tensor<*xf32>) -> ()
}
// CHECK-LABEL: test_PadConstantValuePad_1
// CHECK: [[RES:%.+]] = "onnx.PadConstantValuePad"(%arg0) {constant_value = 0.000000e+00 : f32, mode = "constant", pads = [0, 2, 0, 0]} : (tensor<16x13xf32>) -> tensor<18x13xf32>
// CHECK: return [[RES]] : tensor<18x13xf32>

I guess pads = [0, 2, 0, 0] was interpreted as [0, 2] (first and second elements) are pads for the first dimension and [0, 0] (third and fourth elements) are pads for the second dimension.

On the other hand, in Convolution op or MaxPool op, pads = [0, 2, 0, 0] means that [0, 0] (first and third elements) are pads for the first dimension of the input and [2, 0] (second and fourth elements) are pads for the second dimension of the input. More information about pads format for Convolution op is here: https://github.com/onnx/onnx/blob/master/docs/Operators.md#conv

Ops in some ONNX models

Here, we enumerate ops used in some popular models, and summarize their implementing status in onnx-mlir. An op is available once its backend tests have passed.

(PR: means still in pull request, Y: Yes)

Operator	Available in onnx-mlir	MNIST (opset v8)	ResNet50 (opset v8)
Add	Y	Y
AveragePool	Y		Y (No pads; strides: (1, 1))
BatchNormalizationTestMode	Y		Y
ConvNoBias	Y	Y (No bias; pads: (2, 2, 2, 2); strides: (1, 1)	Y (No bias; pads: (0, 0, 0, 0), (1, 1, 1, 1) and (3, 3, 3, 3); strides: (1, 1) and (2, 2))
Gemm	Y		Y
MaxPoolSingleOut	Y	Y (No pads; strides: (2, 2))	Y (Pads: (1, 1, 1, 1); strides: (2, 2))
MatMul	Y	Y
Relu	Y	Y	Y
Reshape	Y	Y	Y
Softmax	Y		Y
Sum	Y		Y

Implement operators in separated functions

Hi,

It seems that when onnx-mlir translate the onnx model into MLIR, it will fuse all operators into a single main_graph function. Can I change this behavior by implement each operator into a separated a function? That is, a function for an operator?

Many thanks!

Shape-inference does not work for more than 16 operations

shape-inference works well if we have less than or equal to 16 operations, but it does not work for more than 16 operations.

E.g. Running onnf-opt --shape-inference test.mlir with the following test.mlir:

module {                                                                                                                                                                
  func @main_graph(%arg0: tensor<5x5x1x32xf32>) -> tensor<*xf32> {
    %0 = "onnx.Identity"(%arg0) : (tensor<5x5x1x32xf32>) -> tensor<*xf32>
    %1 = "onnx.Identity"(%0) : (tensor<*xf32>) -> tensor<*xf32>
    %2 = "onnx.Identity"(%1) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                   
    %3 = "onnx.Identity"(%2) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                 
    %4 = "onnx.Identity"(%3) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                 
    %5 = "onnx.Identity"(%4) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                 
    %6 = "onnx.Identity"(%5) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                 
    %7 = "onnx.Identity"(%6) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                 
    %8 = "onnx.Identity"(%7) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                 
    %9 = "onnx.Identity"(%8) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                 
    %10 = "onnx.Identity"(%9) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                
    %11 = "onnx.Identity"(%10) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                               
    %12 = "onnx.Identity"(%11) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                               
    %13 = "onnx.Identity"(%12) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                               
    %14 = "onnx.Identity"(%13) : (tensor<*xf32>) -> tensor<*xf32>
    %15 = "onnx.Identity"(%14) : (tensor<*xf32>) -> tensor<*xf32>
    return %15 : tensor<*xf32>
  }
}

would result a correct output:

module {
  func @main_graph(%arg0: tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32> {
    %0 = "onnx.Identity"(%arg0) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %1 = "onnx.Identity"(%0) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %2 = "onnx.Identity"(%1) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %3 = "onnx.Identity"(%2) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %4 = "onnx.Identity"(%3) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %5 = "onnx.Identity"(%4) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %6 = "onnx.Identity"(%5) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %7 = "onnx.Identity"(%6) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %8 = "onnx.Identity"(%7) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %9 = "onnx.Identity"(%8) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %10 = "onnx.Identity"(%9) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %11 = "onnx.Identity"(%10) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %12 = "onnx.Identity"(%11) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %13 = "onnx.Identity"(%12) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %14 = "onnx.Identity"(%13) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %15 = "onnx.Identity"(%14) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    return %15 : tensor<5x5x1x32xf32>
  }
}

If we add one more op into test.mlir as follows.

module {                                                                                                                                                                
  func @main_graph(%arg0: tensor<5x5x1x32xf32>) -> tensor<*xf32> {
    %0 = "onnx.Identity"(%arg0) : (tensor<5x5x1x32xf32>) -> tensor<*xf32>
    %1 = "onnx.Identity"(%0) : (tensor<*xf32>) -> tensor<*xf32>
    %2 = "onnx.Identity"(%1) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                   
    %3 = "onnx.Identity"(%2) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                 
    %4 = "onnx.Identity"(%3) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                 
    %5 = "onnx.Identity"(%4) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                 
    %6 = "onnx.Identity"(%5) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                 
    %7 = "onnx.Identity"(%6) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                 
    %8 = "onnx.Identity"(%7) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                 
    %9 = "onnx.Identity"(%8) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                 
    %10 = "onnx.Identity"(%9) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                                
    %11 = "onnx.Identity"(%10) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                               
    %12 = "onnx.Identity"(%11) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                               
    %13 = "onnx.Identity"(%12) : (tensor<*xf32>) -> tensor<*xf32>                                                                                                               
    %14 = "onnx.Identity"(%13) : (tensor<*xf32>) -> tensor<*xf32>
    %15 = "onnx.Identity"(%14) : (tensor<*xf32>) -> tensor<*xf32>
    %16 = "onnx.Identity"(%15) : (tensor<*xf32>) -> tensor<*xf32>
    return %16 : tensor<*xf32>
  }
}

The output of running onnf-opt --shape-inference test.mlir is:

module {
  func @main_graph(%arg0: tensor<5x5x1x32xf32>) -> tensor<*xf32> {
    %0 = "onnx.Identity"(%arg0) : (tensor<5x5x1x32xf32>) -> tensor<5x5x1x32xf32>
    %1 = "onnx.Identity"(%0) : (tensor<5x5x1x32xf32>) -> tensor<*xf32>
    %2 = "onnx.Identity"(%1) : (tensor<*xf32>) -> tensor<*xf32>
    %3 = "onnx.Identity"(%2) : (tensor<*xf32>) -> tensor<*xf32>
    %4 = "onnx.Identity"(%3) : (tensor<*xf32>) -> tensor<*xf32>
    %5 = "onnx.Identity"(%4) : (tensor<*xf32>) -> tensor<*xf32>
    %6 = "onnx.Identity"(%5) : (tensor<*xf32>) -> tensor<*xf32>
    %7 = "onnx.Identity"(%6) : (tensor<*xf32>) -> tensor<*xf32>
    %8 = "onnx.Identity"(%7) : (tensor<*xf32>) -> tensor<*xf32>
    %9 = "onnx.Identity"(%8) : (tensor<*xf32>) -> tensor<*xf32>
    %10 = "onnx.Identity"(%9) : (tensor<*xf32>) -> tensor<*xf32>
    %11 = "onnx.Identity"(%10) : (tensor<*xf32>) -> tensor<*xf32>
    %12 = "onnx.Identity"(%11) : (tensor<*xf32>) -> tensor<*xf32>
    %13 = "onnx.Identity"(%12) : (tensor<*xf32>) -> tensor<*xf32>
    %14 = "onnx.Identity"(%13) : (tensor<*xf32>) -> tensor<*xf32>
    %15 = "onnx.Identity"(%14) : (tensor<*xf32>) -> tensor<*xf32>
    %16 = "onnx.Identity"(%15) : (tensor<*xf32>) -> tensor<*xf32>
    return %16 : tensor<*xf32>
  }
}

Function with no argument fail

When compiling a simple function with no arguments.

module {
  func @main_graph() -> tensor<8x1x1xf32> {
    %6 = "onnx.Constant"() {value = dense<[[[-0.161539719]], [[-0.433835655]], [[0.091641359]], [[-0.0168522168]], [[-0.0650264397]], [[-0.131737873]], [[0.0204175506]], [[-0.121110231]]]> : tensor<8x1x1xf32>} : () -> tensor<8x1x1xf32>
    %7 = "onnx.Constant"() {value = dense<[[[1.0]], [[1.0]], [[2.0]], [[1.0]], [[1.0]], [[1.0]], [[1.0]], [[1.0]]]> : tensor<8x1x1xf32>} : () -> tensor<8x1x1xf32>
    %8 = "onnx.Add"(%6, %7) : (tensor<8x1x1xf32>, tensor<8x1x1xf32>) -> tensor<8x1x1xf32>
    return %8 : tensor<8x1x1xf32>
  }
  "onnx.EntryPoint"() {func = @main_graph, numInputs = 0 : i32, numOutputs = 1 : i32} : () -> ()
}

When emitting the Krnl dialect:

onnx-mlir-opt --shape-inference --canonicalize --lower-frontend tiny.mlir

The following error occurs:

tiny.mlir:8:5: error: type of return operand 0 ('memref<8x1x1xf32>') doesn't match function result type ('tensor<8x1x1xf32>')
    return %8 : tensor<8x1x1xf32>

Pooling memory optimizations

• intro of get-ref
• pooling of get-ref memory into a single alloc
• handling of dynamic memory for individual memory pools
• support bundling of dynamic-shape memory pools
• smart reuse of memory location for get-ref

Status of the support for the ONNX model zoo

This issue is meant as an ongoing discussion about the onnx-mlir coverage of the ONNX model zoo and any other models of interest. Some of the models we have tried and issues found are below.

Supported:
-[x] MNIST
-[x] ResNet

In progress:
-[] ShuffleNet: slight result inconsistency being investigated but all operations are supported

Missing Ops:
-[] DenseNet: missing GlobalAveragePool operation
-[] AlexNet: missing LRN operation
-[] SqueezeNet: missing Dropout operation
-[] CaffeNet: missing LRN operation

Errors:
bertsquad8:

“onnx-mlir: /home/gbercea/patch-compiler/llvm-project/mlir/lib/IR/Value.cpp:20: mlir::Value::Value(mlir::Operation *, unsigned int): Assertion `op->getNumResults() > resultNo && "invalid result number"' failed.”

bidaf:

“onnx-mlir: /home/gbercea/onnf-compiler/onnx-mlir/src/Builder/FrontendDialectTransformer.cpp:84: mlir::Type onnx_mlir::{anonymous}::FrontendGenImpl::convertONNXTypeToMLIRType(onnx::TensorProto_DataType): Assertion `false && "Unsupported data type encountered."' failed.”

Unsupported type: onnx::TensorProto_DataType::TensorProto_DataType_STRING

mnist fails

Hi @chentong319,

Mnist model fails to compile after your commit is merged, could you take a look?
See the model attached.

Constant op now has wrong output type, for instance, try run with -EmitONNXBasic flag on the attached model, and you'll find:

    %7 = "onnx.Constant"() {value = dense<[-1, 9216]> : tensor<2xi64>} : () -> none

Tian.

Model file:
mnist-simple.onnx.zip

Enable affine-loop-fusion

Put this here just a memo on the following issue:

Currently, if we enabled affine-loop-fusion pass, we would get an error saying

/home/tungld/dl/llvm-project/mlir/lib/Dialect/Affine/IR/AffineOps.cpp:482: mlir::AffineMap promoteComposedSymbolsAsDims(mlir::AffineMap, llvm::ArrayRef<mlir::Value>): Assertion `isValidSymbol(sym) && "Expected only valid symbols"' failed.

It works if turning off canonicalized pass OR memory pool pass. Need to investigate the root cause of the issue.

Constant values are all zero on zLinux

When I tested MNIST on zLinux referring to this script #30, the results were not correct. So, I looked into this issue.

This seems to be because Constant values are all zero as shown bellow.
I confirmed typeArray and arrayAttrInitializer are correct by printing them here.
https://github.com/onnx/onnx-mlir/blob/master/src/builder/frontend_dialect_helper.cpp#L145-L148
So, I guess this is not related to endianess of dataset.

@doru1004 , @tungld , Do you have any suggestion? How can I investigate this?

• The result on zLinux

$ onnx-mlir --EmitONNXIR mnist/model.onnx 

module {
  func @main_graph(%arg0: tensor<1x1x28x28xf32>) -> tensor<1x10xf32> {
    %0 = "onnx.Constant"() {value = dense<[[[[0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000
e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00], [0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.00
0000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00], [0.000000e+00, 0.000000e+00, 0.000000e+00, 
0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00], [0.000000e+00, 0.000000e+
00, 0.000
:
:
:
.000000e+00], [0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00], [0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00]]]]> : tensor<16x8x5x5xf32>} : () -> tensor<16x8x5x5xf32>
    %11 = "onnx.PadConstantValuePad"(%9) {constant_value = 0.000000e+00 : f32, mode = "constant", pads = [0, 0, 2, 2, 0, 0, 2, 2]} : (tensor<1x8x14x14xf32>) -> tensor<1x8x18x18xf32>
    %12 = "onnx.ConvNoBias"(%11, %10) {auto_pad = "NOTSET", dilations = [1, 1], group = 1 : i64, kernel_shape = [5, 5], pads = [0, 0, 0, 0], strides = [1, 1]} : (tensor<1x8x18x18xf32>, tensor<16x8x5x5xf32>) -> tensor<1x16x14x14xf32>
    %13 = "onnx.Constant"() {value = dense<[[[0.000000e+00]], [[0.000000e+00]], [[0.000000e+00]], [[0.000000e+00]], [[0.000000e+00]], [[0.000000e+00]], [[0.000000e+00]], [[0.000000e+00]], [[0.000000e+00]], [[0.000000e+00]], [[0.000000e+00]], [[0.000000e+00]], [[0.000000e+00]], [[0.000000e+00]], [[0.000000e+00]], [[0.000000e+00]]]> : tensor<16x1x1xf32>} : () -> tensor<16x1x1xf32>
    %14 = "onnx.Add"(%12, %13) : (tensor<1x16x14x14xf32>, tensor<16x1x1xf32>) -> tensor<1x16x14x14xf32>
    %15 = "onnx.Relu"(%14) : (tensor<1x16x14x14xf32>) -> tensor<1x16x14x14xf32>
    %16 = "onnx.MaxPoolSingleOut"(%15) {auto_pad = "NOTSET", dilations = [1, 1], kernel_shape = [3, 3], pads = [0, 0, 0, 0], strides = [3, 3]} : (tensor<1x16x14x14xf32>) -> tensor<1x16x4x4xf32>
    %17 = "onnx.Constant"() {value = dense<[1, 256]> : tensor<2xi64>} : () -> tensor<2xi64>
    %18 = "onnx.Reshape"(%16, %17) : (tensor<1x16x4x4xf32>, tensor<2xi64>) -> tensor<1x256xf32>
    %19 = "onnx.Constant"() {value = dense<[[0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00]]> : tensor<1x10xf32>} : () -> tensor<1x10xf32>
    %20 = "onnx.Gemm"(%18, %2, %19) {alpha = 1.000000e+00 : f32, beta = 1.000000e+00 : f32, transA = 0 : i64, transB = 0 : i64} : (tensor<1x256xf32>, tensor<256x10xf32>, tensor<1x10xf32>) -> tensor<1x10xf32>
    return %20 : tensor<1x10xf32>
  }
  "onnx.EntryPoint"() {func = @main_graph, numInputs = 1 : i32, numOutputs = 1 : i32} : () -> ()
}

• Expected results (results on Power9 machine):

$ onnx-mlir --EmitONNXIR mnist/model.onnx
module {
  func @main_graph(%arg0: tensor<1x1x28x28xf32>) -> tensor<1x10xf32> {
    %0 = "onnx.Constant"() {value = dense<[[[[0.0916328802, 0.121435903, 0.0853506476, 0.122606859, -0.0987539738, -0.112124488, -0.035732843, -0.0962621123, -0.147887319, 0.0348589383], [-0.0484465249, 0.136892632, 0.1460547, -0.119769312, 0.208040535, -0.227133304, 0.42333591, 0.058677204, -0.167315915, -0.37082842], [-0.19223924, 0.0943201333, 9.369180e-02, -0.0104355607, 0.23059231, -0.452457309, 0.00250081369, -0.0124352779, 0.0317781866, -0.304293126], [0.0180259701, 0.0242814869, 2.873530e-01, 0.170990467, -0.0743945912, 0.0176675227, -0.105357632, -0.0818113684, 0.0598792844, 4.73489607E-4]], [[-0.153508037, 0.120629221, 0.107659891, 0.0944593325, -0.0698504522, -0.0256833807, 0.109131157, -0.0831774398, 0.0825790539, -0.175707862], [-0.168167844, 0.105139866, -0.17431362, -0.0822822303, 0.464749247, 0.0302605852, -0.159338862, 0.199582711, -0.101042688, -0.494540751], [0.329589069, 0.1775
:
:
: 
1, 0.0515007079, -0.140734166, 0.0136560369, -0.113893688]]]]> : tensor<16x8x5x5xf32>} : () -> tensor<16x8x5x5xf32>
    %11 = "onnx.PadConstantValuePad"(%9) {constant_value = 0.000000e+00 : f32, mode = "constant", pads = [0, 0, 2, 2, 0, 0, 2, 2]} : (tensor<1x8x14x14xf32>) -> tensor<1x8x18x18xf32>
    %12 = "onnx.ConvNoBias"(%11, %10) {auto_pad = "NOTSET", dilations = [1, 1], group = 1 : i64, kernel_shape = [5, 5], pads = [0, 0, 0, 0], strides = [1, 1]} : (tensor<1x8x18x18xf32>, tensor<16x8x5x5xf32>) -> tensor<1x16x14x14xf32>
    %13 = "onnx.Constant"() {value = dense<[[[-0.0822488219]], [[-0.108868778]], [[-0.141039595]], [[-0.204869166]], [[-0.17913565]], [[-0.215438381]], [[-0.133805066]], [[-0.195724562]], [[-0.268250644]], [[-0.258212209]], [[-0.0761560649]], [[0.0132841459]], [[-0.00444464432]], [[-0.414740831]], [[-0.17879115]], [[-0.0386558883]]]> : tensor<16x1x1xf32>} : () -> tensor<16x1x1xf32>
    %14 = "onnx.Add"(%12, %13) : (tensor<1x16x14x14xf32>, tensor<16x1x1xf32>) -> tensor<1x16x14x14xf32>
    %15 = "onnx.Relu"(%14) : (tensor<1x16x14x14xf32>) -> tensor<1x16x14x14xf32>
    %16 = "onnx.MaxPoolSingleOut"(%15) {auto_pad = "NOTSET", dilations = [1, 1], kernel_shape = [3, 3], pads = [0, 0, 0, 0], strides = [3, 3]} : (tensor<1x16x14x14xf32>) -> tensor<1x16x4x4xf32>
    %17 = "onnx.Constant"() {value = dense<[1, 256]> : tensor<2xi64>} : () -> tensor<2xi64>
    %18 = "onnx.Reshape"(%16, %17) : (tensor<1x16x4x4xf32>, tensor<2xi64>) -> tensor<1x256xf32>
    %19 = "onnx.Constant"() {value = dense<[[-0.0448560268, 0.00779166119, 0.0681008175, 0.0299937408, -0.126409635, 0.14021875, -0.0552849025, -0.0493838154, 0.0843220502, -0.0545404144]]> : tensor<1x10xf32>} : () -> tensor<1x10xf32>
    %20 = "onnx.Gemm"(%18, %2, %19) {alpha = 1.000000e+00 : f32, beta = 1.000000e+00 : f32, transA = 0 : i64, transB = 0 : i64} : (tensor<1x256xf32>, tensor<256x10xf32>, tensor<1x10xf32>) -> tensor<1x10xf32>
    return %20 : tensor<1x10xf32>
  }
  "onnx.EntryPoint"() {func = @main_graph, numInputs = 1 : i32, numOutputs = 1 : i32} : () -> ()
}

Testing broadcast validity

Quick one: do we report if a broadcast op has incompatible broadcast arguments?

E.g. when processing shape of Add

/===----------------------------------------------------------------------===//
// Add
/// Infer the output shape of the ONNXAddOp. This method is required by the
/// shape inference interface.
LogicalResult ONNXAddOp::inferShapes() {
  if (!getOperand(0).getType().isa<RankedTensorType>() ||
      !getOperand(1).getType().isa<RankedTensorType>())
    return emitError("Input tensor(s) not ranked");
  auto lhsTy = getOperand(0).getType().cast<RankedTensorType>();
  auto rhsTy = getOperand(1).getType().cast<RankedTensorType>();
  getResult().setType(getBroadcastedType(lhsTy, rhsTy));
  return success();
}

The value returned by getBroadcastedType is null when it fails to find a legal broadcasting type. Should we not check and report failure if it does not? @doru1004 @tjingrant

Translation from ONNX type to MLIR type

@tungld @tjingrant
This is related to PR#156.
We need a translation table from types in ONNX to types in MLIR. We can not do one-to-one mapping because some type is not supported by ONNX-MLIR or we may choose to support types in different way.
This translation will be used in Builder to build data for ONNX-MLIR from protbuf, and in gen_onnx-mlir.py for operation Tablegen.
How should we share the code to make sure we have a consistent translation?

Some missing operations for RNNs

These are some missing operations for running RNNs (There would be more missing operations, here we list those we have known so far). Please feel free to add more.

(PR: means still in pull request, Y: Yes, supported)

ONNX operations	Shape inference	Lowering code
Shape	Y	Y
Cast	Y	Y
Squeeze	Y	Y
Tile	Y	Y
Gather	Y	Y
Slice	Y	Y (constant shapes... upcoming llvm changes will add dynamic as well
ConstantOfShape	Y	Y

cruntime does not exist

I got the following error when doing cmake .. with the latest commit:

-- Performing Test HAVE_STD_REGEX -- success
-- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
-- Performing Test HAVE_POSIX_REGEX -- success
-- Performing Test HAVE_STEADY_CLOCK -- success
-- Found PythonInterp: /gsa/jpngsa/home/t/u/tungld/dl/anaconda2/envs/python3.6/bin/python (found version "3.6.9")
-- pybind11 v2.4.dev4
-- INCLUDE_ONNX_ML Dialect TRUE
CMake Error at src/CMakeLists.txt:69 (install):
  install TARGETS given target "cruntime" which does not exist in this
  directory.

Any suggestion to resolve this error?

Constants in ONNX models

This issue is open to keep track the status of the handling of constant model inputs in ONNX-MLIR.

Trained ONNX models contain constant values that store the outcome of the training. As an example of such a model the MNIST model contains the following tensors as constants:

tensor<16x4x4x10xf32>
tensor<2xi64>
tensor<8x1x5x5xf32>
tensor<8x1x1xf32>
tensor<16x8x5x5xf32>
tensor<16x1x1xf32>
tensor<2xi64>
tensor<1x10xf32>

In the current version of ONNX-MLIR constants are emitted as part of the ONNX MLIR code as Constant ONNX dialect operations:

%17 = "onnx.Constant"() {value = dense<[1, 256]> : tensor<2xi64>} : () -> tensor<2xi64>

The constants are then used as regular variables would be and are passed to other ONNX operations.

Constant operations are lowered to the KRNL IR dialect. For the constant operation above the corresponding KRNL code is:

    %c0_6041 = constant 0 : index
    %c1_6042 = constant 1 : index
    %c1_i64 = constant 1 : i64
    affine.store %c1_i64, %3[%c0_6041] : memref<2xi64>
    %c256_i64_6043 = constant 256 : i64
    affine.store %c256_i64_6043, %3[%c1_6042] : memref<2xi64>

Each constant value is defined individually:

   %c1_i64 = constant 1 : i64
   %c256_i64_6043 = constant 256 : i64

as well as every index:

   %c0_6041 = constant 0 : index
   %c1_6042 = constant 1 : index

Each value is then stored in a memref structure:

   affine.store %c1_i64, %3[%c0_6041] : memref<2xi64>
   affine.store %c256_i64_6043, %3[%c1_6042] : memref<2xi64>

These operations are then be mapped to LLVM dialect operations:

    ; insert first value
    %62 = llvm.mlir.constant(1 : i64) : !llvm.i64
    %63 = llvm.extractvalue %59[1] : !llvm<"{ i64*, i64*, i64, [1 x i64], [1 x i64] }">
    %64 = llvm.mlir.constant(0 : index) : !llvm.i64
    %65 = llvm.mlir.constant(1 : index) : !llvm.i64
    %66 = llvm.mul %60, %65 : !llvm.i64
    %67 = llvm.add %64, %66 : !llvm.i64
    %68 = llvm.getelementptr %63[%67] : (!llvm<"i64*">, !llvm.i64) -> !llvm<"i64*">
    llvm.store %62, %68 : !llvm<"i64*">
    ; insert second value
    %69 = llvm.mlir.constant(256 : i64) : !llvm.i64
    %70 = llvm.extractvalue %59[1] : !llvm<"{ i64*, i64*, i64, [1 x i64], [1 x i64] }">
    %71 = llvm.mlir.constant(0 : index) : !llvm.i64
    %72 = llvm.mlir.constant(1 : index) : !llvm.i64
    %73 = llvm.mul %61, %72 : !llvm.i64
    %74 = llvm.add %71, %73 : !llvm.i64
    %75 = llvm.getelementptr %70[%74] : (!llvm<"i64*">, !llvm.i64) -> !llvm<"i64*">
    llvm.store %69, %75 : !llvm<"i64*">

Issues:

1. ONNX IR code may end up containing large arrays of constant values that make the code hard to inspect.
2. In the KRNL dialect, where constants have to be materialized using Standard and Affine operations, constant arrays are constructed one element at a time starting from their individual components. This then propagates to the LLVM dialect resulting in very slow compilation times.

Potential solution for issue 1:

When we don't output intermediate representations of the model in ONNX or KRNL dialects, then we can leave the constant values inline.

Whenever we output the code in ONNX or KRNL we can use placeholders for the constants:

%17 = "onnx.Constant"() {value = dense<CONST_VALUE_1> : tensor<2xi64>} : () -> tensor<2xi64>

To preserve the capability to round-trip, we have to allow for the above code to be read back and still be a complete program. The values of the constant operations can be emitted in a separate file (with the same base name as the original file but a custom extension like .constants) at the time we emit the ONNX dialect code above. When the code is read back in the constant values are going to be read from default .constants file or a user specified one.
This solution shields the user from seeing the constants in the code. At the same time, constants will be inline while optimization passes are run.

Potential solution for issue 2:

• Lower constant operation to a krnl.constant operation which resides in the same module as the function requiring the constant values:

module {
  %shape = "krnl.constant"() {value : dense<[256, 10]> : memref<2xi64>} -> memref<2xi64>
  func @main_graph(%arg0: memref<1x1x28x28xf32>) -> memref<256x10xf32> {
    [... use %shape ...]
  }
  "krnl.entry_point"() {func = @main_graph, numInputs = 1 : i32, numOutputs = 1 : i32} : () -> ()
}

• The next step is to lower to LLVM. First we lower the constant operation to the LLVM equivalent:

llvm.mlir.global internal constant @shape(dense<[256, 10]> : tensor<2xi32>) : !llvm<"<2 x i32>">

This already exists in the LLVM MLIR dialect.

The code usage of %shape should then be lowered to something similar to the following LLVM IR code:

%0 = alloca [2 x i32], align 4
%1 = bitcast [2 x i32]* %0 to i8*
call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 4 %1, i8* align 4 bitcast ([2 x i32]* @shape to i8*), i64 8, i1 false)
[... use %1 ...]

This solution is only a proposal, feedback is welcome.

Dynamic dimension for reshape output not inferred

In the case below:

module {
  func @main_graph(%arg0: tensor<1x784xf32>, %arg1: tensor<4xi32>, %arg2: tensor<5x5x1x32xf32>, %arg3: tensor<32xf32>, %arg4: tensor<5x5x32x64xf32>, %arg5: tensor<64xf32>, %arg6: tensor<3136x1024xf32>, %arg7: tensor<1024xf32>, %arg8: tensor<2xi32>, %arg9: tensor<1024x10xf32>, %arg10: tensor<10xf32>) -> tensor<?x10xf32> {
    %0 = "onnx.Constant"() {sparse_value = [], value = [-1 : i32, 1 : i32, 28 : i32, 28 : i32]} : () -> tensor<4xi32>
    %1 = "onnx.Reshape"(%arg0, %0) : (tensor<1x784xf32>, tensor<4xi32>) -> tensor<?x1x28x28xf32>

The output shape has a dynamic dimension which could be inferred since all other dimensions are known.

Incorrect commit authorship

The last three commits to trunk seem to have been wrongly attributed to me (the person who pushed the merge button) instead of being attributed to @tungld (the author of the PRs).

It looks like this is a consequence of some recent changes to GitHub:
isaacs/github#1303
These changes are in the process of being reverted.
In the meantime we should fix the authorship of these commits to attribute them to Tung.

Undefined datatype for attribute when importing ONNX 1.0 MNIST model

What version of ONNX is supported?

I tried onnx-mlir with the ONNX 1.0 MNIST model from here [1]. It looks like every node has an undefined datatype which seems wrong and crashes the tool. If I use the ONNX 1.2 MNIST model from the same website the tool works.

$ ./onnx-mlir --EmitONNXIR mnist/model.onnx -debug

ImportNode=Constant
'value' undefined
Assertion failed: (false && "datatype for attribute is not implemented"), function convertAttributeProtoToNameValuePair, file /onnx-mlir/src/Builder/FrontendDialectTransformer.cpp, line 227.

[1] https://github.com/onnx/models/tree/master/vision/classification/mnist

Errors during check-onnx-lit

Hi, the first test of check-onnx-lit failed and gives me the following error massage:

FAIL: Open Neural Network Frontend :: onnx/onnx_canonicalization.mlir (1 of 10)
... ...
Testing Time: 0.11s
********************
Failing Tests (1):
    Open Neural Network Frontend :: onnx/onnx_canonicalization.mlir

  Expected Passes    : 9
  Unexpected Failures: 1
test/mlir/CMakeFiles/check-onnx-lit.dir/build.make:57: recipe for target 'test/mlir/CMakeFiles/check-onnx-lit' failed
make[3]: *** [test/mlir/CMakeFiles/check-onnx-lit] Error 1
CMakeFiles/Makefile2:3062: recipe for target 'test/mlir/CMakeFiles/check-onnx-lit.dir/all' failed
make[2]: *** [test/mlir/CMakeFiles/check-onnx-lit.dir/all] Error 2
CMakeFiles/Makefile2:3069: recipe for target 'test/mlir/CMakeFiles/check-onnx-lit.dir/rule' failed
make[1]: *** [test/mlir/CMakeFiles/check-onnx-lit.dir/rule] Error 2
Makefile:1009: recipe for target 'check-onnx-lit' failed
make: *** [check-onnx-lit] Error 2

Meanwhile, when I'm trying to emit MLIR with mobilenet/squeezenet ONNX model from ONNX model zoo, it gives me the following massage:

onnx-mlir: /home/hcye/workspace/onnx-llvm-project/mlir/include/mlir/IR/Types.h:279: U mlir::Type::cast() const [with U = mlir::RankedTensorType]: Assertion `isa<U>()' failed.
Aborted (core dumped)

I'm using the latest ONNX-MLIR and the LLVM repo mentioned in the readme. Thanks!

emitError with no action

@tungld

in the LSTM code, I see a lot of emitError without actions. That is potentially a problem. In general, if it is due to error from user, we should give a message; and if its an error from the implementation, we can emit a message and then assert.

template <>
LstmState allocAndInitializeStates<ONNXLSTMOp, LstmState>(
    ConversionPatternRewriter &rewriter, Location loc, ONNXLSTMOp *op,
    OperandAdaptor<ONNXLSTMOp> operandAdaptor) {
  LstmState state;

  // Insert allocation and deallocation for the results of this operation.
  if (!isNoneType(op->Y())) {
    auto yMemRefType = convertToMemRefType(op->Y().getType());
    if (hasAllConstantDimensions(yMemRefType))
      state.allH = insertAllocAndDealloc(yMemRefType, loc, rewriter,
          checkInsertDealloc(op->getOperation(), 0));
    else
      emitError(loc, "Unsupported dynamic dimensions.");
  } else {
    state.allH = op->Y();
  }

see the emitError. Either the function can return a LogicalResult, or special value such as nullptr. There errors are then is then caught in the caller context. Or you can use an assert and the program will stop

Equivalent to mlir-translate

Is it intended to provide an equivalent to mlir-translate which supports the onnx dialect in addition?

Surfacing the standard `mlir-opt` command line options

The onnx-mlir tool currently surfaces only a handful of command line options, and it would be nice if it would surface the rest of the mlir-opt options, so that the output could be customized.

In particular, I was hoping to use the --convert-std-to-llvm's emit-c-wrappers.

Invoke gen_doc.py

As we edit gen_doc.py, it does not get invoked so it is difficult to double check if changes there are correct.

Would it be possible to invoke it under a custom target in the main makefile?

That would have 2 benefits:

1. Testing that no errors creep into the gen_doc.py
2. Avoid the redundant updates to the files generated by gen_doc.py

Currently, the building of the files is explained but invoke more steps than simply generating a make command.

It is important that we are still in control of the info used by gen_doc.py; we don't want to get a newer version of the specs.

Feel free to comment

The name of ONNF is misleading

ONNF is indicating that ONNX is another ML frontend (like tf, pytorch, etc), which is not true and adding chaos when promoting it to be a shared (common) mlir dialect later (which I think is the one of the goals of this work).

How about naming it as "ONNX dialect" directly? @tjingrant

Questions about graph optimizations

Does the onix-mlir-opt perform all the optimizations specified in https://github.com/microsoft/onnxruntime/blob/master/docs/ONNX_Runtime_Graph_Optimizations.md on the onnx MLIR dialect , and the are the ones specified in the link the only optimisations supported ?

Questions about the Krnl dialect

A couple of questions about the Krnl dialect.

(a) What is the purpose of the "define_loops" and "optimize_loops" instructions? Are there any examples illustrating its usage? It seems to serve as a way of naming loops and to help organize loop-transformations (may be), and it is not clear to me this is the best way to do that. E.g., I find it potentially confusing to think of a "loop" as a "Value"; it is not meant to be a run-time value, it is meant to be a static-concept relevant only while compiling.

(b) I assume "iterate" is meant to be a sequential loop/iteration? It seems useful to capture parallel iterations/loops and reduction loops explicitly in the instruction, instead of having to rely on a compiler-analysis to figure it out.

(c) MLIR's linalg.generic separates the indexing operations on tensors from the computation that is done on those values (rather than relying on a compiler to figure those out). Is there any thought/plan on doing this?

third_party/onnx build problem

I ran into two issues when trying "python setup.py install" and got them fixed. Not sure about the protocol of modifying third_part/onnx. I just post them as an issue now, in case anyone encounters them.

1. purest_runner not supported by pypi. Delete it from required list in setup.py

2. New version of protobuf is causing problem, fixed by #2482 (onnx/onnx#2482) . It is a one-line change in CMakelist.txt

Garbled filename after running onnx-mlir on a model

OS: Ubuntu 18.04 64-bit
Commit id: f5f336d
Steps to reproduce:
I downloaded a model from https://github.com/onnx/models and ran the follow command:
onnx-mlir --EmitONNXIR model.onnx. Only model.mlir was outputted while model.onnx.mlir was missing.

Secondly, after running onnx-mlir --EmitLLVMIR model.onnx, model.onnx.mlir was nowhere to be found. Instead, a file with a garbled filename showed up. The content of the file was fine though.

It appears to be an issue with the way the concatenation is done here :

onnx-mlir/src/MainUtils.cpp

Lines 110 to 111 in 137ce76

	const char * tempFilename = (filename + extension).c_str();
	freopen(tempFilename, "w", stderr);

If I change this to something like below, the filename corruption goes away.

string tempFilename = filename + extension;
freopen(tempFilename.c_str(), "w", stderr);

Let me know what you think. Thanks.

Help on ExecutionSession

Is there any example to run the exeuction session ?

ONNX ML

Ingesting and lowering of the ONNX ML

• adding ONNX ML operators
• adding new types: Strings
• adding new types: Map
• adding new types: Sequences
• hooking to existing lowering

Cover case when default attributes are skipped

In the case of ConvNoBias the auto_pad and group attributes are sometimes omitted in the ONNX model. In this case, ensure that when the shape inference pass is run and the default values are fetched for these attributes, they are also set inside the operation.

Questions About Lowering from the onnx dialect

How exactly does one convert the code in onnx dialect like the one given below to a lower dialect like affine and std ?. Do these conversions already exist ?.
`
func @main_graph(%arg0: tensor<2x3xf32>) -> tensor<2x3xf32> {
%0 = "onnx.Add"(%arg0, %arg0) : (tensor<2x3xf32>, tensor<2x3xf32>) -> tensor<2x3xf32>
return %0 : tensor<2x3xf32>
}
"onnx.EntryPoint"() {func = @main_graph, numInputs = 1 : i32, numOutputs = 1 : i32} : () -> ()

`

Error when trying to convert tf model to onnx using tf2onnx

I have made a simple model in tensorflow it seems to throwing a lot of errors when trying to convert to onnx format. What can i do to fix the infer shape warnings and the key error.

This is the code
`

import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
import numpy as np
import matplotlib.pyplot as plt
import tf2onnx


X_train = np.random.randn(50,1)

#print(X_train)
Y_train = X_train*2 + 25

lr = 0.02
epochs = 1500

X = tf.placeholder(tf.float32,(50,1),name="input")
Y = tf.placeholder(tf.float32,(50,1))

size_input = 1
size_output = 1
size_hidden1 = 2
size_hidden2 = 6
size_hidden3 = 18

Wh1 = tf.Variable(tf.random_normal([size_input,size_hidden1]), name = "Wh1")
bh1 = tf.Variable(tf.random_normal([1,size_hidden1]),name = "bh1")

Wh2 = tf.Variable(tf.random_normal([size_hidden1,size_hidden2]),name = "Wh2")
bh2 = tf.Variable(tf.random_normal([1,size_hidden2]),name = "bh1")

Wh3 = tf.Variable(tf.random_normal([size_hidden2,size_hidden3]),name = "Wh3")
bh3 = tf.Variable(tf.random_normal([1,size_hidden3]),name = "bh3")

Wy = tf.Variable(tf.random_normal([size_hidden3,size_output]),name = "Wy")
by = tf.Variable(tf.random_normal([1,size_hidden3]),name = "by")

a1 = tf.add(tf.matmul(X,Wh1),bh1)
lh1 = tf.nn.relu(a1)

a2 = tf.add(tf.matmul(lh1,Wh2),bh2)
lh2 = tf.nn.relu(a2)

a3 = tf.add(tf.matmul(lh2,Wh3),bh3)
lh3 = tf.nn.relu(a3)

pred = tf.identity(tf.add(tf.matmul(lh3,Wy),by,name = "output"))
#return pred

#pred = tf.identity(model(), name ="output")
loss = tf.reduce_sum(tf.pow((Y-pred),2.0))/(2.0*len(X_train))

opt = tf.train.GradientDescentOptimizer(learning_rate=lr).minimize(loss)

with tf.Session() as sess :
    sess.run(tf.global_variables_initializer())

    for i in range(epochs) :
        for j in range(50) :
            sess.run(opt,feed_dict={X:X_train,Y:Y_train})
        if i%1000 == 0 :
            print("Loss :",loss.eval({X:X_train, Y:Y_train}))

    print("----converting to onnx -------")
    onnx_graph = tf2onnx.tfonnx.process_tf_graph(sess.graph, input_names=["input:0"], output_names=["output:0"])
    #model_proto = onnx_graph.make_model("test")
    #with open("ourmodel.onnx", "wb") as f:
    #    f.write(model_proto.SerializeToString())

The errors

`

Cannot infer shape for add_5: add_5:0
Cannot infer shape for Relu_5: Relu_5:0
Cannot infer shape for MatMul_7: MatMul_7:0
Cannot infer shape for output_1: output_1:0
Cannot infer shape for sub_1: sub_1:0
Cannot infer shape for gradients_1/Sum_1_grad/Reshape: gradients_1/Sum_1_grad/Reshape:0
Cannot infer shape for gradients_1/Sum_1_grad/BroadcastTo: gradients_1/Sum_1_grad/BroadcastTo:0
Cannot infer shape for gradients_1/Pow_1_grad/mul: gradients_1/Pow_1_grad/mul:0
Cannot infer shape for gradients_1/Pow_1_grad/mul_1: gradients_1/Pow_1_grad/mul_1:0
Cannot infer shape for gradients_1/Pow_1_grad/Sum: gradients_1/Pow_1_grad/Sum:0
Cannot infer shape for gradients_1/Pow_1_grad/Reshape: gradients_1/Pow_1_grad/Reshape:0
Cannot infer shape for gradients_1/Pow_1_grad/Greater: gradients_1/Pow_1_grad/Greater:0
Cannot infer shape for gradients_1/Pow_1_grad/ones_like: gradients_1/Pow_1_grad/ones_like:0
Cannot infer shape for gradients_1/Pow_1_grad/Select: gradients_1/Pow_1_grad/Select:0
Cannot infer shape for gradients_1/Pow_1_grad/Log: gradients_1/Pow_1_grad/Log:0
Cannot infer shape for gradients_1/Pow_1_grad/zeros_like: gradients_1/Pow_1_grad/zeros_like:0
Cannot infer shape for gradients_1/Pow_1_grad/Select_1: gradients_1/Pow_1_grad/Select_1:0
Cannot infer shape for gradients_1/Pow_1_grad/mul_2: gradients_1/Pow_1_grad/mul_2:0
Cannot infer shape for gradients_1/Pow_1_grad/mul_3: gradients_1/Pow_1_grad/mul_3:0
Cannot infer shape for gradients_1/Pow_1_grad/Sum_1: gradients_1/Pow_1_grad/Sum_1:0
Cannot infer shape for gradients_1/Pow_1_grad/tuple/control_dependency: gradients_1/Pow_1_grad/tuple/control_dependency:0
Cannot infer shape for gradients_1/sub_1_grad/Sum: gradients_1/sub_1_grad/Sum:0
Cannot infer shape for gradients_1/sub_1_grad/Reshape: gradients_1/sub_1_grad/Reshape:0
Cannot infer shape for gradients_1/sub_1_grad/Neg: gradients_1/sub_1_grad/Neg:0
Cannot infer shape for gradients_1/sub_1_grad/Sum_1: gradients_1/sub_1_grad/Sum_1:0
Cannot infer shape for gradients_1/sub_1_grad/Reshape_1: gradients_1/sub_1_grad/Reshape_1:0
Cannot infer shape for gradients_1/sub_1_grad/tuple/control_dependency: gradients_1/sub_1_grad/tuple/control_dependency:0
.
.
.
.
.
.
.
eyError                                  Traceback (most recent call last)
<ipython-input-27-6d09d7b09280> in <module>
     56     o = tf.identity(sess.run(pred, feed_dict={X:X_train}), name = "outputs")
     57     print("----converting to onnx -------")
---> 58     onnx_graph = tf2onnx.tfonnx.process_tf_graph(sess.graph, input_names=["input:0"], output_names=["outputs:0"])
     59     #model_proto = onnx_graph.make_model("test")
     60     #with open("ourmodel.onnx", "wb") as f:

~/.local/lib/python3.8/site-packages/tf2onnx/tfonnx.py in process_tf_graph(***failed resolving arguments***)
    377         target = constants.DEFAULT_TARGET
    378 
--> 379     onnx_nodes, op_cnt, attr_cnt, output_shapes, dtypes, _ = tensorflow_to_onnx(tf_graph, shape_override)
    380     if not is_subgraph:
    381         # make tf2onnx internal subgraphs from the tensorflow subgraphs

~/.local/lib/python3.8/site-packages/tf2onnx/tf_utils.py in tensorflow_to_onnx(graph, shape_override)
    226     Load tensorflow graph and do a conversion.
    227     """
--> 228     return tflist_to_onnx(graph, shape_override)

~/.local/lib/python3.8/site-packages/tf2onnx/tf_utils.py in tflist_to_onnx(g, shape_override)
    164             if shape is None:
    165                 shape = get_tf_tensor_shape(out)
--> 166             dtypes[out.name] = map_tf_dtype(out.dtype)
    167             output_shapes[out.name] = shape
    168 

~/.local/lib/python3.8/site-packages/tf2onnx/tf_utils.py in map_tf_dtype(dtype)
    114 def map_tf_dtype(dtype):
    115     if dtype:
--> 116         dtype = TF_TO_ONNX_DTYPE[dtype]
    117     return dtype
    118 

KeyError: tf.float32_ref

KRNL Optimization Ops

Tracking Optimization Ops for KRNL IR

• krnl.block
• krnl.permute
• krnl.unroll
• krnl.jam

Enable windows build/install.

Besides Linux build and install, windows build/install should also be supported (MLIR itself supports windows build, btw).