mizux / dotnet-native

Template to build a multi-platforms Native Net 6.0 Nuget package using dotnet cli

Github-CI:

Introduction

| Requirement | Codemap | Dependencies | Build | CI | Appendices | License |

This is an example of how to create a Modern CMake C++/.Net Project.

This project aim to explain how you build a .NetStandard2.0 native (win-x64, linux-x64 and osx-x64) nuget multiple package using .NET Core CLI and the new .csproj format.
e.g. You have a cross platform C++ library (using a CMake based build) and a .NetStandard2.0 wrapper on it thanks to SWIG.
Then you want to provide a cross-platform Nuget package to consume it in a .NetCoreApp3.1 project...

Requirement

You'll need:

• "CMake >= 3.18".
• ".Net Core SDK >= 3.1" to get the dotnet cli.

note: We won't/can't rely on VS 2019 since we want a portable cross-platform dotnet/cli pipeline.

Codemap

The project layout is as follow:

• CMakeLists.txt Top-level for CMake based build.

• cmake Subsidiary CMake files.

  • dotnet.cmake All internall .Net CMake stuff.

• ci Root directory for continuous integration.

• Foo Root directory for Foo library.

  • CMakeLists.txt for Foo.
  • include public folder.
    • foo
      • Foo.hpp
  • src private folder.
    • src/Foo.cpp
  • dotnet
    • CMakeLists.txt for Foo .Net.
    • foo.i SWIG .Net wrapper.

• Bar Root directory for Bar library.

  • CMakeLists.txt for Bar.
  • include public folder.
    • bar
      • Bar.hpp
  • src private folder.
    • src/Bar.cpp
  • dotnet
    • CMakeLists.txt for Bar .Net.
    • bar.i SWIG .Net wrapper.

• FooBar Root directory for FooBar library.

  • CMakeLists.txt for FooBar.
  • include public folder.
    • foobar
      • FooBar.hpp
  • src private folder.
    • src/FooBar.cpp
  • dotnet
    • CMakeLists.txt for FooBar .Net.
    • foobar.i SWIG .Net wrapper.

• dotnet Root directory for .Net template files

  • Mizux.DotnetNative.runtime.csproj.in csproj template for the .Net "native" (i.e. RID dependent) package.
  • Mizux.DotnetNative.csproj.in csproj template for the .Net package.
  • Test.csproj.in csproj template for .Net test project.
  • Example.csproj.in csproj template for .Net example project.

• tests Root directory for tests

  • CMakeLists.txt for DotnetNative.Test .Net.
  • FooTests.cs Code of the Mizux.DotnetNative.FooTests project.

• examples Root directory for examples

  • CMakeLists.txt for DotnetNative.FooApp .Net.
  • FooApp.cs Code of the DotnetNative.FooApp app.

Dependencies

To complexify a little, the CMake project is composed of three libraries (Foo, Bar and FooBar) with the following dependencies:

Foo:
Bar:
FooBar: PUBLIC Foo PRIVATE Bar

Build Process

To Create a native dependent package we will split it in two parts:

• A bunch of Mizux.DotnetNative.runtime.{rid}.nupkg packages for each Runtime Identifier (RId) targeted containing the native libraries.
• A generic package Mizux.DotnetNative.nupkg depending on each runtime packages and containing the managed .Net code.

Actually, You don't need a specific variant of .Net Standard wrapper, simply omit the library extension and .Net magic will pick the correct native library.
ref: https://www.mono-project.com/docs/advanced/pinvoke/#library-names

note: Microsoft.NetCore.App packages follow this layout.

note: While Microsoft use runtime-<rid>.Company.Project for native libraries naming, it is very difficult to get ownership on it, so you should prefer to use Company.Project.runtime-<rid> instead since you can have ownership on Company.* prefix more easily.

We have two use case scenario:

1. Locally, be able to build a Mizux.DotnetNative package which only target the local OS Platform, i.e. building for only one Runtime Identifier (RID).
   note: This is useful since the C++ build is a complex process for Windows, Linux and MacOS. i.e. We don't support cross-compilation for the native library generation.

2. Be able to create a complete cross-platform (ed. platform as multiple rid) Mizux.DotnetNative package.
   i.e. First you generate each native Nuget package (Mizux.DotnetNative.runtime.{rid}.nupkg) on each native architecture, then copy paste these artifacts on one native machine to generate the meta-package Mizux.DotnetNative.

Local Mizux.DotnetNative Package

Let's start with scenario 1: Create a Local only Mizux.DotnetNative.nupkg package targeting one Runtime Identifier (RID).
We would like to build a Mizux.DotnetNative.nupkg package which only depends on one Mizux.DotnetNative.runtime.{rid}.nupkg in order to work locally.

The pipeline for linux-x64 should be as follow: note: The pipeline will be similar for osx-x64 and win-x64 architecture, don't hesitate to look at the CI log.

Building local runtime Mizux.DotnetNative Package

disclaimer: In this git repository, we use CMake and SWIG.
Thus we have the C++ shared library libFoo.so and the SWIG generated .Net wrapper Foo.cs.
note: For a C++ CMake cross-platform project sample, take a look at Mizux/cmake-cpp.
note: For a C++/Swig CMake cross-platform project sample, take a look at Mizux/cmake-swig.

So first let's create the local Mizux.DotnetNative.runtime.{rid}.nupkg nuget package.

Here some dev-note concerning this Mizux.DotnetNative.runtime.{rid}.csproj.

• Once you specify a RuntimeIdentifier then dotnet build or dotnet build -r {rid} will behave identically (save you from typing it).
  • note: it is NOT the case if you use RuntimeIdentifiers (notice the 's')
• It is recommended to add the tag native to the nuget package tags

  <PackageTags>native</PackageTags>

• This package is a runtime package so we don't want to ship an empty assembly file:

  <IncludeBuildOutput>false</IncludeBuildOutput>

• Add the native (i.e. C++) libraries to the nuget package in the repository runtimes/{rid}/native. e.g. for linux-x64:

  <Content Include="*.so">
    <PackagePath>runtimes/linux-x64/native/%(Filename)%(Extension)</PackagePath>
    <Pack>true</Pack>
    <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  </Content>

• Generate the runtime package to a defined directory (i.e. so later in Mizux.DotnetNative package we will be able to locate it)

  <PackageOutputPath>{...}/packages</PackageOutputPath>

Then you can generate the package using:

dotnet pack Mizux.DotnetNative.runtime.{rid}

note: this will automatically trigger the dotnet build.

If everything good the package (located where your PackageOutputPath was defined) should have this layout:

{...}/packages/Mizux.DotnetNative.runtime.{rid}.nupkg:
\- Mizux.DotnetNative.runtime.{rid}.nuspec
\- runtimes
   \- {rid}
      \- native
         \- *.so / *.dylib / *.dll
...

note: {rid} could be linux-x64 and {framework} could be netstandard2.0

tips: since nuget package are zip archive you can use unzip -l <package>.nupkg to study their layout.

Building local Mizux.DotnetNative Package

So now, let's create the local Mizux.DotnetNative.nupkg nuget package which will depend on our previous runtime package.

Here some dev-note concerning this DotnetNative.csproj.

• Add the previous package directory:

  <RestoreSources>{...}/packages;$(RestoreSources)</RestoreSources>

• Add dependency (i.e. PackageReference) on each runtime package(s) availabe:

  <ItemGroup>
    <RuntimeLinux Include="{...}/packages/Mizux.DotnetNative.runtime.linux-x64.*.nupkg"/>
    <RuntimeOsx   Include="{...}/packages/Mizux.DotnetNative.runtime.osx-x64.*.nupkg"/>
    <RuntimeWin   Include="{...}/packages/Mizux.DotnetNative.runtime.win-x64.*.nupkg"/>
    <PackageReference Include="Mizux.DotnetNative.runtime.linux-x64" Version="1.0" Condition="Exists('@(RuntimeLinux)')"/>
    <PackageReference Include="Mizux.DotnetNative.runtime.osx-x64"   Version="1.0" Condition="Exists('@(RuntimeOsx)')"  />
    <PackageReference Include="Mizux.DotnetNative.runtime.win-x64"   Version="1.0" Condition="Exists('@(RuntimeWin)')"  />
  </ItemGroup>

  Thanks to the RestoreSource we can work locally we our just builded package without the need to upload it on nuget.org.

Then you can generate the package using:

dotnet pack Mizux.DotnetNative

If everything good the package (located where your PackageOutputPath was defined) should have this layout:

{...}/packages/Mizux.DotnetNative.nupkg:
\- Mizux.DotnetNative.nuspec
\- lib
   \- {framework}
      \- Mizux.DotnetNative.dll
...

note: {framework} could be netcoreapp3.1 or/and net6.0

Testing local Mizux.DotnetNative.FooApp Package

We can test everything is working by using the Mizux.DotnetNative.FooApp or Mizux.DotnetNative.FooTests project.

First you can build it using:

dotnet build <build_dir>/dotnet/FooApp

note: Since Mizux.DotnetNative.FooApp PackageReference Mizux.DotnetNative and add {...}/packages to the RestoreSource. During the build of DotnetNative.FooApp you can see that Mizux.DotnetNative and Mizux.DotnetNative.runtime.{rid} are automatically installed in the nuget cache.

Then you can run it using:

dotnet run --project <build_dir>/dotnet/FooApp/FooApp.csproj

note: Contrary to dotnet build and dotnet pack you must use --project before the .csproj path (let's call it "dotnet cli command consistency")

You should see:

$ dotnet run --project build/dotnet/FooApp/FooApp.csproj
[1] Enter DotnetNativeApp
[2] Enter Foo::staticFunction(int)
[3] Enter freeFunction(int)
[3] Exit freeFunction(int)
[2] Exit Foo::staticFunction(int)
[1] Exit DotnetNativeApp

Complete Mizux.DotnetNative Package

Let's start with scenario 2: Create a Complete Mizux.DotnetNative.nupkg package targeting multiple Runtime Identifier (RID).
We would like to build a Mizux.DotnetNative.nupkg package which depends on several Mizux.DotnetNative.runtime.{rid}.nupkg.

The pipeline should be as follow:
note: This pipeline should be run on any architecture, provided you have generated the three architecture dependent Mizux.DotnetNative.runtime.{rid}.nupkg nuget packages.

Building All runtime Mizux.DotnetNative Package

Like in the previous scenario, on each targeted OS Platform you can build the coresponding Mizux.DotnetNative.runtime.{rid}.nupkg package.

Simply run on each platform:

cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release

note: replace {rid} by the Runtime Identifier associated to the current OS platform.

Then on one machine used, you copy all other packages in the {...}/packages so when building Mizux.DotnetNative.csproj we can have access to all package...

Building Complete Mizux.DotnetNative Package

This is the same step than in the previous scenario, since we "see" all runtime packages in {...}/packages, the project will depends on each of them.

Once copied all runtime package locally, simply run:

dotnet build <build_dir>/dotnet/Mizux.DotnetNative
dotnet pack <build_dir>/dotnet/Mizux.DotnetNative

Testing Complete Mizux.DotnetNative Package

We can test everything is working by using the Mizux.DotnetNative.FooApp or Mizux.DotnetNative.FooTests project.

First you can build it using:

dotnet build <build_dir>/dotnet/FooApp

note: Since Mizux.DotnetNative.FooApp PackageReference Mizux.DotnetNative and add {...}/packages to the RestoreSource. During the build of Mizux.DotnetNative.FooApp you can see that Mizux.DotnetNative and Mizux.DotnetNative.runtime.{rid} are automatically installed in the nuget cache.

Then you can run it using:

dotnet run --project <build_dir>/dotnet/FooApp/FooApp.csproj

You should see something like this

$ dotnet run --project build/dotnet/FooApp/FooApp.csproj
[1] Enter DotnetNativeApp
[2] Enter Foo::staticFunction(int)
[3] Enter freeFunction(int)
[3] Exit freeFunction(int)
[2] Exit Foo::staticFunction(int)
[1] Exit DotnetNativeApp

Appendices

Few links on the subject...

.Net runtime can deduce library extension so don’t use a platform-specific library name in the DllImport statement. Instead, just use the library name itself, without any prefixes or suffixes, and rely on the runtime to find the appropriate library at runtime.
ref: Mono pinvoke#libraryname

Resources

Project layout

• The Pitchfork Layout

CMake

• CMake Reference Documentation
• https://llvm.org/docs/CMakePrimer.html
• https://cliutils.gitlab.io/modern-cmake/
• https://cgold.readthedocs.io/en/latest/

Target Framework Moniker (TFM)

• .NET TFM list
• .NET Standard implementation support

.Net:Runtime IDentifier (RID)

• .Net Support Policy
• .NET Core RID Catalog
• Creating native packages
• Blog on Nuget Rid Graph

Reference on .csproj format

• Common MSBuild project properties
• MSBuild well-known item metadata
• Additions to the csproj format for .NET Core

Issues

Some issue related to this process

• PackageReference only support TargetFramework condition
• Nuget needs to support dependencies specific to target runtime #1660
• Improve documentation on creating native packages #238
• Guide for packaging C# library using P/Invoke

Misc

Image has been generated using plantuml:

plantuml -Tsvg docs/{file}.dot

So you can find the dot source files in docs.

License

Apache 2. See the LICENSE file for details.

Disclaimer

This is not an official Google product, it is just code that happens to be owned by Google.

dotnet-native's Issues

Adapting this template for a submodule'd library

I'm trying to write bindings for a C/C++ library which I don't control. I've included that library, as is, as a submodule in this repo.

I'm trying to adapt this template so that the bindings can be a sibling of the library folder, rather than a child (so it works with a submodule). That is:

this template:
├─ Foo/
│  ├─ CMakeLists.txt
│  ├─ dotnet/
│  │  ├─ CMakeLists.txt

mine:
├─ Foo/
│  ├─ CMakeLists.txt
├─ Foo.dotnet/
│  ├─ CMakeLists.txt

Right now, it's failing because the swig-generated wrapper doesn't have access to the header file.

[build] C:\whisper.dotnet\build\dotnet\NickDarvey.Whisper\Whisper\whisperCSHARP_wrap.cxx(766,10): fatal  error C1083: Cannot open include file: 'whisper.cpp/whisper.h': No such file or directory [C:\whisper.dotnet\build\whisper.cpp.dotnet\dotnet_whisper.vcxproj]

When I build this (mizux/dotnet-native) repo with Visual Studio 2022 kit selected it generates a build\Foo\dotnet\dotnet_Foo.vcxproj which has:

1. The include folder (containing the header) as an AdditionalIncludeDirectories

   <AdditionalIncludeDirectories>C:\dotnet-native\Foo\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
   

2. The includes folder as an argument to swig.exe

   [...]swig.exe -csharp -IC:/dotnet-native -IC:/dotnet-native/dotnet -IC:/dotnet-native/Foo/include -namespace Mizux.DotnetNative.Foo -dllimport mizux-dotnetnative-native -outdir C:/dotnet-native/build/dotnet/Mizux.DotnetNative/Foo -c++ -o C:/dotnet-native/build/dotnet/Mizux.DotnetNative/Foo/fooCSHARP_wrap.cxx C:/dotnet-native/Foo/dotnet/foo.i
   

When I (attempt) to build my (nickdarvey/whisper.dotnet) it generates a vcxproj which has neither of those. I think that's why I'm getting that error.

Are you able to help me get the location of my header files, ./whisper.cpp, to appear in AdditionalIncludeDirectories?

Native runtime packages compatible with .NET Framework (AnyCPU)?

How can I produce runtime packages compatible with .NET Framework?

I tried to change the framework identified netcoreapp3.1 to netstandard2.0 or net461 but I failed to produce working examples.

The reason why I fail to run the examples is that there are no native DLLs copied to the binary space for the example project:

Analysis of the differences:

• Working with netcoreapp3.1: The folder \dotnet\examples\Example\bin\Debug\netcoreapp3.1 contains a subfolder runtimes\win-x64\native with the native DLLs.
• Not Working with standard2.0: The folder \dotnet\examples\Example\bin\Debug\netstandard2.0 contains no native DLLS and neither a subfolder runtimes subfolder

The nuspec files for the wrapper and native package are the same except for the targetFramework identifier.

Are runtime packages restricted to .NET core and .NET 5.0+ ?

Can't run dotnet using Xcode generator

When using the -G "Xcode" generator, it seems we can't run the samples from CTest.

It seems dotnet didn't succeed to find all its dependencies

A fatal error was encountered. The library 'libhostpolicy.dylib' required to execute the application was not found in '...'.

ref: https://github.com/Mizux/dotnet-native/runs/670363781?check_suite_focus=true

note: When using the default make generator, everything work flawlessly
see: https://github.com/Mizux/dotnet-native/runs/670363803?check_suite_focus=true

XCode `dotnet build` failure

When trying to run dotnet build all output .dll are renamed

e.g. dotnet package use AssemblyName:

dotnet-native/dotnet/Mizux.DotnetNative.csproj.in

Line 6 in 7ac1c8f

<AssemblyName>@DOTNET_PROJECT@</AssemblyName>

which is

dotnet-native/cmake/dotnet.cmake

Line 23 in 7ac1c8f

set(DOTNET_PACKAGE Mizux.DotnetNative)

so with any generator but XCode we got dotnet cli generating the command with arg /out:obj/x64/Release/net6.0/Mizux.DotnetNative.dll but Xcode export the env TARGETNAME=dotnet_package leading to the dotnet internal build command replaced by /out:obj/x64/Release/net6.0/dotnet_package.dll

This impact build, pack, test and run command AFAIK

Cannot PackageReference Mizux.Foo

I have a C++ library with NetStandard2.0 wrapper; this library has specialized builds for Windows, Linux and MacOS. I can generate the .Net wrapper and shared library which is specific for each platform only on this platform (i.e. no cross compilation).

note: You can find this example at: https://github.com/Mizux/dotnet

Native OS library: Mizux.Foo.*-x64.nupkg

On Linux-x64:

Native.cpp -> Native.so -------+
Native.cpp -> Native.linux.cs -+-> Foo.linux-x64.csproj -> Mizux.Foo.linux-x64.npkg

src: https://github.com/Mizux/dotnet/blob/master/src/Foo.linux-x64/Foo.linux-x64.csproj

On osx-x64:

Native.cpp -> Native.dylib --+
Native.cpp -> Native.osx.cs -+-> Foo.osx-x64.csproj -> Mizux.Foo.osx-x64.npkg

src: https://github.com/Mizux/dotnet/blob/master/src/Foo.osx-x64/Foo.osx-x64.csproj

On win-x64:

Native.cpp -> Native.dll ----+
Native.cpp -> Native.win.cs -+-> Foo.win-x64.csproj -> Mizux.Foo.win-x64.npkg

src: https://github.com/Mizux/dotnet/blob/master/src/Foo.win-x64/Foo.win-x64.csproj

So I can generate native package one each OS for this OS.

Question 1: Does my nuget package layout correct ?

unzip -l package/Mizux.Foo.linux-x64.1.0.0.nupkg
Archive:  package/Mizux.Foo.linux-x64.1.0.0.nupkg
  Length      Date    Time    Name
---------  ---------- -----   ----
      509  2018-08-14 15:47   _rels/.rels
      498  2018-08-14 15:47   Mizux.Foo.linux-x64.nuspec
     3584  2018-08-14 13:33   runtimes/linux-x64/lib/netstandard2.0/Mizux.Foo.linux-x64.dll # .CS which load the native.so -> should be in the same repository
        0  2018-08-09 12:04   runtimes/linux-x64/lib/netstandard2.0/native.so
      520  2018-08-14 15:47   [Content_Types].xml
      633  2018-08-14 15:47   package/services/metadata/core-properties/d5f8836b446941b1a8eb822b3c4e009c.psmdcp
---------                     -------
     5744                     6 files

So here I put files in runtimes/linux-x64/lib/netstandard2.0 but when building with: dotnet build --runtime win-x64 src/Foo.linux-x64 file are placed in [bin/Debug/]netstandard2.0/linux-x64/* So do I need to place files in lib/linux-x64/netstandard2.0 instead ?

Meta-Package: Mizux.Foo.nupkg

Then I want to create a meta-package Mizux.Foo containing the three dependencies as PackageReference (supposing I copy past all generated in a local package folder).

package/Mizux.Foo.linux-x64.nupkg -+
package/Mizux.Foo.osx-x64.nupkg ---+-(PackageReference)> Foo.csproj -> Mizux.Foo.nupkg
package/Mizux.Foo.win-x64.nupkg ---+

src: https://github.com/Mizux/dotnet/blob/master/src/Foo/Foo.csproj

I manage to create the nupkg with the .csproj and Mizux.Foo.nuspec inside the nuget package contains:

<dependencies>
  <group targetFramework=".NETStandard2.0">
    <dependency id="Mizux.Foo.linux-x64" version="1.0.0" exclude="Build,Analyzers" />
    <dependency id="Mizux.Foo.osx-x64" version="1.0.0" exclude="Build,Analyzers" />
    <dependency id="Mizux.Foo.win-x64" version="1.0.0" exclude="Build,Analyzers" />
  </group>
</dependencies>

FooApp using Foo.csproj

So far so good ???
I manage to build a FooApp if I use ProjectReference on Foo.csproj and condition on RuntimeIdentifier i.e.

dotnet build --runtime linux-x64 src/FooApp
dotnet src/FooApp/bin/Debug/netcoreapp2.1/linux-x64/Mizux.FooApp.dll

src: https://github.com/Mizux/dotnet/blob/master/src/FooApp/FooApp.csproj

This works but in fact it create a degenerated project where Foo.osx-x64 and Foo.win-x64 are ignored (which is cool when testing locally but not when building the meta-package...)

side note: dotnet run --runtime linux-x64 --project src/FooApp won't work since RuntimeIdentifier is ignored, so it try to rebuild as pure .Net project and look in bin/debug/netstandard2.0...

Example using Mizux.Foo.nupkg

So far so good ???
So now I now I want to create a Example which PackageReference Mizux.Foo package...

dotnet build -r linux-x64 example/Example
Program.cs(8,7): error CS0103: The name 'Foo' does not exist in the current context [/usr/local/google/home/corentinl/work/dotnet/example/Example/Example.csproj]
Build FAILED.

src: https://github.com/Mizux/dotnet/blob/master/example/Example/Example.csproj

Question 2: What's wrong here why example don't see the runtimes dir ?

Question 3: It's seems Foo doesn't expose Foo.*-x64 i.e. transitivity not working, why ?

How to change name of library

Hi Mizux

I tried to generalize your setup a bit fill out the missing parts of the .csproj.in packages. Something that I cannot get my head around are:

I removed the NuGet repository to avoid uploading to nuget.org, such that RestoreSources only contains a local package dir. Do I need to upload stuff or can I use a local directory?

I can build everything but not execute the C# tests. It insists on naming stuff Mizux and DotNetNative but only runtime.

.NET48

Have you tried building it using .NET48. At least on Windows, I get numerous errors about incompatibility on the given platform. The framework is for sure installed. We are still stuck on .NET48