packages.config does not respect allowedVersions attribute about docs.microsoft.com-nuget

[zivkan]

@frederick-brier In the schema section of the docs, it says for allowedVersions:

It does not affect what package is installed during an install or restore operation.

Bringing the a concept similar to floating versions to packages.config is not possible because when you install a package with packages.config, it modifies the project file to contain something like this: <HintPath>../../packages/MyProj.PkgId.1.0.0/lib/net472/MyProj.PkgId.dll</HintPath>.

If NuGet restored version 1.0.5, then that path will be wrong, and the project's not going to build. You need to use the Package Manager UI or Package Manager Console to install upgrade the package.

This is one of the things that is improved/fixed with PackageReference, but there are limitations in which project types can migrate from packages.config to PackageReference

[frederick-brier]

Hi Andy, Several points. First the doc on the following pages is pretty clear about supporting "Constraining upgrade versions" in a packages.config file: https://learn.microsoft.com/en-us/nuget/consume-packages/reinstalling-and-updating-packages#constraining-upgrade-versions https://learn.microsoft.com/en-us/nuget/reference/packages-config Could you please explain how I might be misinterpreting this documentation? Second, when using CMake with the Ninja generator, there is no project file with <PackageReference> elements or <HintPath> elements. There is only the packages.config. There are lots of articles from Microsoft and others that speak to NuGet being used for C++ and multiple platforms and architectures (Linux, MacOS, WinRT, Android, ARM64). Using NuGet for solely for a .NET framework project built in Visual Studio only on Windows is not Microsoft's vision. vcpkg, a Microsoft package manager is built with CMake, and both are cross-platform. Finally, it is not impossible. While I have had to write a number of glue scripts, after a NuGet restore all the packages are in a designated folder, usually ./packages, although mine are placed in CMAKE_BINARY_DIR, a small CMake function iterates over exploded folders (including dependent packages) and adds the include and libs to the respective CMake search paths. In the case of .NET, the <targetFramework> element would determine the subdirectory. The lack of the allowedVersions attribute forced me today to write another script (lots of time) that calls out to Github Packages and return all the versions for a package name and regenerates a new packages.config with the exact version so that the nuget restore would work with the packages.config. This solution feels more brittle, but it is possible. As architect I chose to go with an all Microsoft/Github tech stack thinking the integration of all the tools would be very smooth and well documented especially for a Windows product. Again, I would like to understand how I misinterpreted the documentation. IMHO, this is a bug. I have lost a great deal of time implementing workarounds. Thank you. Sincerely yours, Frederick N. Brier Senior Software Architect at Alludo From: Andy Zivkovic ***@***.***> Sent: Friday, September 8, 2023 2:11 AM To: NuGet/docs.microsoft.com-nuget ***@***.***> Cc: Frederick Brier ***@***.***>; Mention ***@***.***> Subject: Re: [NuGet/docs.microsoft.com-nuget] packages.config does not respect allowedVersions attribute (Issue #3141) CAUTION: This e-mail is from an external sender, do not click on unexpected links or attachments. @frederick-brier<https://github.com/frederick-brier> In the schema section of the docs, it says for allowedVersions: It does not affect what package is installed during an install or restore operation. Bringing the a concept similar to floating versions to packages.config is not possible because when you install a package with packages.config, it modifies the project file to contain something like this: <HintPath>../../packages/MyProj.PkgId.1.0.0/lib/net472/MyProj.PkgId.dll</HintPath>. If NuGet restored version 1.0.5, then that path will be wrong, and the project's not going to build. You need to use the Package Manager UI or Package Manager Console to install upgrade the package. This is one of the things that is improved/fixed with PackageReference, but there are limitations in which project types can migrate from packages.config to PackageReference<https://learn.microsoft.com/en-us/nuget/consume-packages/migrate-packages-config-to-package-reference> - Reply to this email directly, view it on GitHub<#3141 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BBJ4K5NKSARBHATXCUEPAH3XZLOJRANCNFSM6AAAAAA4PPSG2M>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[zivkan]

When I started, I really didn't think this response would end up as long as it did. Sorry. However, as the saying goes, I don't have time to write a short letter, so I wrote a long one instead.

Could you please explain how I might be misinterpreting this documentation?

The first documentation page you linked, https://learn.microsoft.com/en-us/nuget/consume-packages/reinstalling-and-updating-packages#constraining-upgrade-versions, discusses installing and upgrading packages with Visual Studio's Package Manager UI and Package Manager Console. If you take the "Constraining upgrade versions" section out of the context of the rest of the document, I can see how coming in with the idea that restore should install some version with the rage, then the documentation doesn't dispel that idea.

However, I believe that similar to how the documentation doesn't explicitly say "this doesn't affect restore", it also doesn't say that it is used by restore. The first sentence of the section says "By default, reinstalling or updating a package ...". However, when nuget looks at the packages listed in packages.config and downloads them, this action is called restore. Restore is a different action than re-install or update. Again, understandable misunderstanding, and I wish that the NuGet team used clearer language, and had much clearer documentation, but I joined the team after the product was over 7 years old, so I couldn't prevent historic mistakes.

The second link you provided was the packages.config reference documentation, where I pulled the quote from my previous reply "A range of allowed versions for this package applied during package update (see Constraining upgrade versions. It does not affect what package is installed during an install or restore operation.". So, it explicitly states that this metadata is only used for update, and that it does not affect install or restore. Again, it unfortunately requires the reader to understand the difference between update and restore (and install).

Second, when using CMake with the Ninja generator, there is no project file with elements or elements.

In this case, while it's interesting that you, and the part of the community who are working in the way you describe, have found a way to make NuGet work for you, it's outside of how NuGet was designed to work. All the documentation we have on using and installing packages is for MSBuild projects, primarily .NET projects.

If you create a MSBuild based C++ project, and you use either the Package Manager UI or Package Manager Console to install a package, NuGet will create the packages.config file, but it will also modify the vcxproj file to directly reference any package assets. Unless you created a C++/CLI project, there's an exception to every rule, but in this case it uses PackageReference which fixes many issues learned during the first 5 years of nuget's packages.config existance.

Anyway, for C++ MSBuild projects with packages.config, the vcxproj changes is usually only msbuild props/targets files which will result in the vcxproj having something like <Import Project="..\packages\packageid.1.2.3\build\packageid.props" Condition="Exists(...)" />. Here we can again see the package version in the path, which needs to match the version that nuget restore during CI pipelines.

If the CMake or Ninja generator community would like improved tooling, we don't have much documentation on the NuGet Client SDK, but we do have samples for getting package metadata (which includes the list of all versions the package source has, to the highest in a range can be selected), [how to download a package](https://learn.microsoft.com/en-us/nuget/reference/nuget-client-sdk#download-a-package]. Unfortunately we don't have a sample for how to exact a nupkg, but if anyone using the NuGet Client SDK has questions, I can tell them in the NuGet.Packaging package, there's a class named PackageExtractor.

Unfortunately is CMake/Ninja generator want to start using NuGet as an API, then while it provides more flexibility, it also has more responsibility since someone will need to maintain the app/integration, so very understandable if nobody wants to take that responsibility. But in that case, people using the tooling are limited to existing features.

However, I'd like to reiterate that how CMake and Ninja generator is using NuGet is outside the design for how NuGet was intended to be used. NuGet has always been a tool that worked with MSBuild projects.

IMHO, this is a bug.

At the bottom of each documentation page are two links, one to report an issue with the documentation page, which you used to create this issue. The other is for product issues, which would have taken you to https://github.com/NuGet/Home. If you wish to report a product bug, or feature suggestion, then please do so in the Home repo. This issue, reporting a documentation bug, I can see how there's room for improvement, and hopefully someone will update the docs to make it clearer, so other people don't have the same confusion you did. But I wanted to help by explaining that this is not how NuGet works, rather than ignore this issue until the documentation is improved.

While your suggestion, nuget restore, finding some suitable version within the allowedRange makes sense in the context for how you wish to use it, NuGet's primary use-case, with MSBuild projects (mostly .NET, and to a very small extent vcxproj C++ projects) it would unfortunately be harmful. If nuget restored a package other than the exact version specified, then build would fail for these MSBuild projects, since they contain imports and hint paths pointing to the package directory, rather than an error saying the package version couldn't be found. The error that the package version couldn't be found will make it easier and quicker for developers to investigate, rather than more obscure build errors, probably about APIs that can't be found, or in the case of packages like SourceLink, the error would be completely silent, and the developer would need to notice that their pdbs are missing source link metadata. Therefore, I'm not optimistic that NuGet's new issue triage team would accept a feature or design change request to allow what you're asking for. It would benefit a much smaller number of customers than the number of customers who would cause more difficult to investigate failures.

Anyway, I have a lot of other work I need to do, so I'm not going to spend time editing this comment to be shorter. I just wanted to help by helping understand how nuget works, rather than ignore the issue until documentation is changed to explain that that's not how nuget works.

[frederick-brier]

Hi Andy, Thank you for your detailed response. I have already read it twice but am still trying to understand how implementing the handling of allowedVersions would negatively impact the majority of .NET users. I will keep reading until I do. This is not a request for additional information. You have already spent enough time. I have created scripts that allow my CI build to work. I have forked both the NuGet.Client repo as well as the CMake repo, created stories for enhancing both with a reference to the 8 file patch that added flexible constraints to <Package References>. Perhaps I will get around to it at some point. Again, thank you for your time. Fred From: Andy Zivkovic ***@***.***> Sent: Friday, September 8, 2023 6:44 AM To: NuGet/docs.microsoft.com-nuget ***@***.***> Cc: Frederick Brier ***@***.***>; Mention ***@***.***> Subject: Re: [NuGet/docs.microsoft.com-nuget] packages.config does not respect allowedVersions attribute (Issue #3141) CAUTION: This e-mail is from an external sender, do not click on unexpected links or attachments. When I started, I really didn't think this response would end up as long as it did. Sorry. However, as the saying goes, I don't have time to write a short letter, so I wrote a long one instead. Could you please explain how I might be misinterpreting this documentation? The first documentation page you linked, https://learn.microsoft.com/en-us/nuget/consume-packages/reinstalling-and-updating-packages#constraining-upgrade-versions, discusses installing and upgrading packages with Visual Studio's Package Manager UI and Package Manager Console. If you take the "Constraining upgrade versions" section out of the context of the rest of the document, I can see how coming in with the idea that restore should install some version with the rage, then the documentation doesn't dispel that idea. However, I believe that similar to how the documentation doesn't explicitly say "this doesn't affect restore", it also doesn't say that it is used by restore. The first sentence of the section says "By default, reinstalling or updating a package ...". However, when nuget looks at the packages listed in packages.config and downloads them, this action is called restore. Restore is a different action than re-install or update. Again, understandable misunderstanding, and I wish that the NuGet team used clearer language, and had much clearer documentation, but I joined the team after the product was over 7 years old, so I couldn't prevent historic mistakes. The second link you provided was the packages.config reference documentation, where I pulled the quote from my previous reply "A range of allowed versions for this package applied during package update (see Constraining upgrade versions<https://learn.microsoft.com/en-us/nuget/consume-packages/reinstalling-and-updating-packages#constraining-upgrade-versions>. It does not affect what package is installed during an install or restore operation.". So, it explicitly states that this metadata is only used for update, and that it does not affect install or restore. Again, it unfortunately requires the reader to understand the difference between update and restore (and install). Second, when using CMake with the Ninja generator, there is no project file with elements or elements. In this case, while it's interesting that you, and the part of the community who are working in the way you describe, have found a way to make NuGet work for you, it's outside of how NuGet was designed to work. All the documentation we have on using and installing packages is for MSBuild projects, primarily .NET projects. If you create a MSBuild based C++ project, and you use either the Package Manager UI or Package Manager Console to install a package, NuGet will create the packages.config file, but it will also modify the vcxproj file to directly reference any package assets. Unless you created a C++/CLI project, there's an exception to every rule, but in this case it uses PackageReference which fixes many issues learned during the first 5 years of nuget's packages.config existance. Anyway, for C++ MSBuild projects with packages.config, the vcxproj changes is usually only msbuild props/targets files which will result in the vcxproj having something like <Import Project="..\packages\packageid.1.2.3\build\packageid.props" Condition="Exists(...)" />. Here we can again see the package version in the path, which needs to match the version that nuget restore during CI pipelines. If the CMake or Ninja generator community would like improved tooling, we don't have much documentation on the NuGet Client SDK, but we do have samples for getting package metadata<https://learn.microsoft.com/en-us/nuget/reference/nuget-client-sdk#get-package-metadata> (which includes the list of all versions the package source has, to the highest in a range can be selected), [how to download a package](https://learn.microsoft.com/en-us/nuget/reference/nuget-client-sdk#download-a-package]<https://learn.microsoft.com/en-us/nuget/reference/nuget-client-sdk#download-a-package%5D>. Unfortunately we don't have a sample for how to exact a nupkg, but if anyone using the NuGet Client SDK has questions, I can tell them in the NuGet.Packaging package, there's a class named PackageExtractor. Unfortunately is CMake/Ninja generator want to start using NuGet as an API, then while it provides more flexibility, it also has more responsibility since someone will need to maintain the app/integration, so very understandable if nobody wants to take that responsibility. But in that case, people using the tooling are limited to existing features. However, I'd like to reiterate that how CMake and Ninja generator is using NuGet is outside the design for how NuGet was intended to be used. NuGet has always been a tool that worked with MSBuild projects. IMHO, this is a bug. At the bottom of each documentation page are two links, one to report an issue with the documentation page, which you used to create this issue. The other is for product issues, which would have taken you to https://github.com/NuGet/Home. If you wish to report a product bug, or feature suggestion, then please do so in the Home repo. This issue, reporting a documentation bug, I can see how there's room for improvement, and hopefully someone will update the docs to make it clearer, so other people don't have the same confusion you did. But I wanted to help by explaining that this is not how NuGet works, rather than ignore this issue until the documentation is improved. While your suggestion, nuget restore, finding some suitable version within the allowedRange makes sense in the context for how you wish to use it, NuGet's primary use-case, with MSBuild projects (mostly .NET, and to a very small extent vcxproj C++ projects) it would unfortunately be harmful. If nuget restored a package other than the exact version specified, then build would fail for these MSBuild projects, since they contain imports and hint paths pointing to the package directory, rather than an error saying the package version couldn't be found. The error that the package version couldn't be found will make it easier and quicker for developers to investigate, rather than more obscure build errors, probably about APIs that can't be found, or in the case of packages like SourceLink, the error would be completely silent, and the developer would need to notice that their pdbs are missing source link metadata. Therefore, I'm not optimistic that NuGet's new issue triage team would accept a feature or design change request to allow what you're asking for. It would benefit a much smaller number of customers than the number of customers who would cause more difficult to investigate failures. Anyway, I have a lot of other work I need to do, so I'm not going to spend time editing this comment to be shorter. I just wanted to help by helping understand how nuget works, rather than ignore the issue until documentation is changed to explain that that's not how nuget works. - Reply to this email directly, view it on GitHub<#3141 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BBJ4K5N6OIJVB5IBTPD4PXDXZMOJ3ANCNFSM6AAAAAA4PPSG2M>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>