Feature: C# bindings and Semantic Kernel integration about usearch

[alexbarev]

@dranger003, okay! I greatly appreciate your contributions -- your approach has given me insights into some new techniques and boosted development. I'll take a closer look at the code and refactor it a bit then.

[ashvardanian]

A couple of recommendations from @dluc:

1. I would encourage adopting the default C# code style (settings can be found in the SK repo, e.g. see https://github.com/microsoft/semantic-kernel/blob/main/.editorconfig)
2. if possible please make the library .NET Standard 2.0 compatible; that will increase adoption, e.g. across some old apps that haven't migrated to Standard 2.1 or .NET6+

[dluc]

Looking forward to this, in particular the disk based index feature. Let me know if I can help! :-)

[dranger003]

Hi there, I've taken an initial take on some C# bindings, feel free to use any of it if you think this can be useful.
https://github.com/dranger003/usearch-dotnet

[ashvardanian]

Thank you, @dranger003! We will continue from there and will release a NuGet package on GitHub.

[alexbarev]

Hey @dranger003 and @ashvardanian,

I've pushed my version of the USearchNetWrapper. I'd appreciate any feedback or suggestions on how we can integrate our contributions.

Here's a brief overview of what I've done:

• Introduced USearchIndex.cs as the public API.
• Created NativeMethods.cs to wrap calls to dynamic libraries.

Moving forward, here are the tasks I've identified:

• Enhance testing for the Add, Get and Remove functions in USearchIndex
• Test the initialization of USearchIndex with function pointers for a custom metric
• Implement the Search method in USearch
• Think about overloading the Add function to support float16, float8, and byte vectors
• Set up automated testing for all target operating systems
• Publish a NuGet package.

Looking forward to your thoughts!

[dranger003]

Hi @AleksandrKent, great work on interop code! I only focused on the low level P/Invoke calls, which have been pulled. Maybe you can sync up and adjust your wrapper? There are few more things I think I need to clean in the interop layer:

• Although nuint is the right type for size_t it is not widely used at a higher level. I have used int for now but I think it should be changed to ulong throughout
• I used the same testing code as C99 but I think it's missing a number of test cases, and having some benchmarks would be nice
• As you pointed out, there is currently no support other than f32/f64

[dranger003]

@ashvardanian Yes, I sent another pull request to add f16 support.

[ashvardanian]

@AleksandrKent has already made some great progress on the Python side . C# next.

@dluc, for compatibility and simplicity, we currently put metadata into Parquet files on disk, but integrating UStore down the road would make more sense.

[ashvardanian]

Oh, wow! Great to have you with us @dranger003! We are also halfway through with those, probably makes sense to sync up and merge the progress. @AleksandrKent, how is your fork doing?

[ashvardanian]

@dranger003 would you like to merge your progress into the main repo under the csharp/ directory?

[ashvardanian]

@dranger003 and @AleksandrKent, is there support for f16 datatypes in C#? Assuming this may be used with Unity when developing mobile games, one can expect most Arm v8+ CPUs to have half-precision support.

[alexbarev]

Hi @dranger003 and @ashvardanian,

I've got a few thoughts.

We're using dynamic libraries for C# bindings. What if we added these to the NuGet package? There's guidance in Microsoft's docs. This might lead us to move libusearch build scripts from .csproj to GitHub Actions.

My recent work on C# bindings isn't aligning with our feature branch. I thought of creating a separate C# interface for lib, but maybe the current C lib interface is good for NuGet. @dranger003, can you wrap up your work as a NuGet package?

When we start Semantic Kernel integration, I might be more useful. Should we open a new issue for it once NuGet is ready?

Keen to hear your thoughts.

[alexbarev]

Hi @dranger003 and @ashvardanian,

I've been reviewing the recent progress on the C# bindings, and I have some thoughts and suggestions to discuss:

1. Type Casting in usearch_add: Instead of casting to object and then to the proper float type, would it be more efficient to create separate methods for each type and utilize overloading? Generics with if-else statements might not be the best practice for our case.

2. Error Handling in usearch_get: The error returned from usearch_dimensions is lost (see here). To address this and improve separation of concern:
   2.1) We could separate native calls with the DLLImport attribute into a NativeMethods class, and wrappers into another class, as I started in my code. This would create a 1-1 match with C lib functions and handle errors with Marshal.PtrToStringAnsi(error) on the public API side.
   2.2) This approach would also eliminate the need for using DllImportAttribute.EntryPoint, aligning with Microsoft's guidelines on 1-1 namings.
   2.3) We might consider making the Interop class internal.

3. Platform Detection: I suggest using a static constructor instead of preprocessor directives for platform detection. This would provide a clear exception, as I encountered errors on Ubuntu 22.04 when trying to build with the dotnet CLI.

4. Calling Convention: Should we explicitly specify the calling convention as cdecl with DllImportAttribute.CallingConvention for every call to native function? This could be an issue if the target OS uses stdcall while the C lib uses cdecl by default.

5. Building on Ubuntu 22.04: I had trouble running the code on Ubuntu 22.04 via the dotnet CLI. Perhaps we could add a script with a few lines using the dotnet CLI to ensure compatibility. It is also might be helpful to add bash and cmd scripts for manual testing at the local environment with the dotnet CLI.

6. Testing Framework: I recommend rewriting the tests to align with xUnit, for clarity.
   6.1) Additionally, @ashvardanian suggested auto-generating vectors instead of reading data from files, as they are becoming obsolete.

7. .gitignore File: I believe the .gitignore should remain local to your development environment, or we could add specific paths like csharp/bin or csharp/obj to the core repo's .gitignore.

8. Purpose of "*.vcxproj" Files: Are these files necessary for our project? If they are a result of using MSBuild, would it be better to stick with dotnet only, as it is more common to use dotnet for development across different platforms?

9. Using TODOs: some parts of the code are not tested, and it's unclear whether they will work as currently implemented, such as passing the delegate usearch_metric_t to usearch_init. It might be prudent to leave a TODO comment in the code to highlight this concern.

I'm eager to hear your thoughts on these points, and I'm ready to assist with any refactoring based on our considerations.

[ashvardanian]

@AleksandrKent, those suggestions sound great! @dranger003 and @dluc, any objections?

[dranger003]

@AleksandrKent These are great recommendations! I will let you and your team handle the code from here as this is representing more time investment than I currently have available. And I am planning to continue monitoring the evolution of the code, thanks for all the insights!