mimalloc is not publishing the libraries about vcpkg

[dannypike]

Please stop asking me for more information and then telling me that the extra information is not relevant to this ticket. I know it's not relevant to vcpkg; that's why I left it out of the original post and said that it is not relevant, in my original post.

I only added the extra information because you seemed to be a bit confused about why I was having a problem with this port. All of these extra messages between us have been a result of my attempts to explain why I wanted access to the libraries that are not published by this port, in response to your questions.

I have never claimed that the fundamental cause of the problem is either this port, cmake or vcpkg. I am only saying that I cannot solve this particular well-known issue with cmake using this port, because the port does not "publish" the files that I need.

By "publish", I mean that it copies them to my drive but it does not copy them to the location that it sets in the MIMALLOC_* variables.

Please would you fix the port (or find someone to fix it) so that the files get published into the location where the find_package() variables says that it has put them?

That will be the end of my problem with this port. Everything else that we have been discussing has nothing to do with this ticket; it is only because you keep asking me for more details.

Thanks :)

[dannypike]

Ah, so it is a problem with my language and not with the information?

I apologise if you have been upset by the way that I talk. It is not my intention to be offensive or to upset anyone. I always try to give a short ticket that avoids any distracting information, but sometimes misunderstandings happen.

[dannypike]

I don't think the position relative to project makes a difference. I would look at the order of sources (the suggested mi_version()) and link libraries (mimalloc) added to the executables.

Thank you. I put it there to remind myself that it is a "sensitive" part of the makefile, so I don't break the build when I come back to it later to add another library or do something that might cause this problem to resurface by accidentally inserting another library between it and the source file list, etc.

The fact that it's in a "surprising" place will make me stop and think, rather than be on auto-pilot in that case. I work on dozens of different projects and I forget stuff :)

[dg0yt]

Partial file list from vcpkg CI:

mimalloc:x64-windows:/bin/mimalloc-redirect.dll
mimalloc:x64-windows:/bin/mimalloc.dll
mimalloc:x64-windows:/bin/mimalloc.pdb
...
mimalloc:x64-windows:/lib/mimalloc.lib

[dannypike]

Sorry, I don't know what you mean.

Are you agreeing with me, disagreeing with me, or talking to someone else?

[dannypike]

UPDATE: This is what I see if I dump the variables from find_package(mimalloc):

1> [CMake] -- MIMALLOC_CMAKE_DIR=D:/work/github/vcpkg/installed/x64-windows/share
1> [CMake] -- MIMALLOC_INCLUDE_DIR=D:/work/github/vcpkg/installed/x64-windows/share/mimalloc
1> [CMake] -- MIMALLOC_LIBRARY_DIR=D:/work/github/vcpkg/installed/x64-windows/share
1> [CMake] -- MIMALLOC_OBJECT_DIR=D:/work/github/vcpkg/installed/x64-windows/share/mimalloc
1> [CMake] -- MIMALLOC_TARGET_DIR=D:/work/github/vcpkg/installed/x64-windows/share
1> [CMake] -- MIMALLOC_VERSION_DIR=mimalloc

and:

1> [CMake] -- mimalloc-redirect_DIR=mimalloc-redirect_DIR-NOTFOUND
1> [CMake] -- mimalloc_CONFIG=D:/work/github/vcpkg/installed/x64-windows/share/mimalloc/mimalloc-config.cmake
1> [CMake] -- mimalloc_CONSIDERED_CONFIGS=D:/work/github/vcpkg/installed/x64-windows/share/mimalloc/mimalloc-config.cmake
1> [CMake] -- mimalloc_CONSIDERED_VERSIONS=2.1
1> [CMake] -- mimalloc_DIR=D:/work/github/vcpkg/installed/x64-windows/share/mimalloc
1> [CMake] -- mimalloc_FOUND=1
1> [CMake] -- mimalloc_VERSION=2.1
1> [CMake] -- mimalloc_VERSION_COUNT=2
1> [CMake] -- mimalloc_VERSION_MAJOR=2
1> [CMake] -- mimalloc_VERSION_MINOR=1
1> [CMake] -- mimalloc_VERSION_PATCH=0
1> [CMake] -- mimalloc_VERSION_TWEAK=0

(note the NOTFOUND for the redirect one)

and this is what I see when I look for the two library files in my vcpkg folder:

D:\work\github\vcpkg>dir /s /b mimalloc.lib
D:\work\github\vcpkg\buildtrees\mimalloc\x64-windows-rel\mimalloc.lib
D:\work\github\vcpkg\installed\x64-windows\lib\mimalloc.lib
D:\work\github\vcpkg\packages\mimalloc_x64-windows\lib\mimalloc.lib

D:\work\github\vcpkg>dir /s /b mimalloc-redirect.lib
D:\work\github\vcpkg\buildtrees\mimalloc\src\v2.1.2-c0150f463b.clean\bin\mimalloc-redirect.lib

The two files are not where the primary CMake variable MIMALLOC_LIBRARY_DIR says they would be and the mimalloc-redirect.lib file appears to be in a temporary folder; it was not copied to the same "installed" folder as mimalloc.lib.

[dg0yt]

The title says "mimalloc is not publishing the libraries". The message says "the files are not there ... logs not relevant".
I listed the libraries which are installed, and I would take it as an indication that I should disagree.
There is no import lib corresponding to mimalloc-redirect.dll, but I don't know if it needs one.

The variables seem to indicate that cmake config mfixup doesn't handle all relocations which it should handle. This would be a serious port bug, however very different from the original post.

[dannypike]

OK, it sounds like some ambiguity in our common language. When I said it doesn't "publish the libraries", I meant that they are not where it says that they are.

My current project has several external dependencies, one of those dependencies apparently depends on mimalloc. It links fine, but I hit this problem when I try to run the application:

mimalloc-redirect: error: mimalloc-redirect.dll seems to be initialized after ucrtbase.dll

It's a classic problem with CMake - there's no 100% reliable way to specify the link order with CMake. The best you can do is to specify the import libraries explicitly and right after the add_executable and before anything else. Mostly, that technique works (it's not guaranteed) but, in this case, it looked like it was working, exccept that I couldn't find the mimalloc.lib and mimalloc-redirect.lib in the location that was set by find_package().

Do we need it assigned to someone?

[dg0yt]

mimalloc-redirect: error: mimalloc-redirect.dll seems to be initialized after ucrtbase.dll

This doesn't seem to be specific vcpkg problem:
microsoft/mimalloc#668

Now you bring up another problem:

It's a classic problem with CMake - there's no 100% reliable way to specify the link order with CMake.

In general, there is a reliable way to define the linking order of libs: Create separate (imported) targets for two libs of interest, and define a dependency. Then the independent library will be after the dependent library. (This must be done for static libs when not using MSVC.) But I see the difficulty in doing this for implicit link libraries like ucrtbase.

I still wonder if it is necessary to use a link lib for minalloc-redirect.dll. I'm under the impression that it would be loaded dynamically (i.e. like a plugin). And there is a problem with plugins: Except for a few packages, vcpkgs cannot deploy plugins like it deploys linked DLLs, to the directory of your executables.

[dg0yt]

IMO the extra information in the answers is valuable.
But I don't feel comfortable with how they are presented to a community member which shows interest in getting the problem solved.

[dg0yt]

I created a test port, and I fixed a number of quirks. But most of them are not relevant for MSVC: #37499
Here I would like to add a few focused points:

1. I would like to know why do you want to use the CMake variables. I would assume they are more or less obsolete, given the imported targets.
2. The official tests don't show the mimalloc-redirect error message. And I don't see them using a corresponding import lib.
   However, I do see that they implement this recommendation:

   To ensure the mimalloc-override.dll is loaded at run-time it is easiest to insert some call to the mimalloc API in the main function, like mi_version()

3. I notice that vcpkg doesn't deploy the DLLs as usual for MSVC.

Disclaimer: My MSVC testing is limited to vcpkg CI.

[dannypike]

With your numbering:

1. I want to include the library early in the CMakeLists.txt. I find that the linker sometimes reorders the order in which it links things. This behaviour can change with the same project as I add new source files, etc. So I want to encourage it to put mimalloc before ucrtbase and the way that I do that is to specify it right at the top of the CMakeLists file (just after the project statement.

2. I am also in the habit of putting this code above the main(), for the same reason:

	///////////////////////////////////////////////////////////////////////
	// Force the linker to import mimalloc before anything else, so that it
	// can be used as the default allocator
	bool _forceMimalloc = false;
	void forceMimalloc() {
		_forceMimalloc = mi_option_is_enabled(mi_option_eager_commit);
	}

After being in the habit of adding both of these hacks, I find that the warning goes away.

PS: I have just tried commenting out the redirect library and it is still running OK. So, maybe that's not needed, as you said. Or maybe I haven't triggered the linker into breaking the order, yet.

[dg0yt]

Thanks for the response. Maybe we could add the mi_version hint to the usage note printed at the command line.

1. So I want to encourage it to put mimalloc before ucrtbase and the way that I do that is to specify it right at the top of the CMakeLists file (just after the project statement.

I don't think the position relative to project makes a difference. I would look at the order of sources (the suggested mi_version()) and link libraries (mimalloc) added to the executables.