Basic Authentication - Response status code does not indicate success: 401 (Unauthorized) about artifacts-credprovider

[JohnSchmeichel]

@JohnSchmeichel should i still create an issue? we just added ENV USE_NET6_ARTIFACTS_CREDENTIAL_PROVIDER false so it fallbacks to older version and works for us.

If you think that it's good to create a defect - i can create it, but it's a question what should be default version - older .NET so it won't require any changes for older project, or new one and then older project need to force to use older version. And if the answer is newer should be default, then i guess we don't need a ticket

Glad it's working for you (will still look at the script error though). Newer should be the default, the .NET Core 3.1 is out of support and we'll be removing it from the code base in the coming weeks.

[JohnSchmeichel]

@sergey-litvinov-work I think your issue is different as it's failing to successfully install the credential provider, can you open a new issue on this?

[JohnSchmeichel]

It's possible, ultimately that's what is used with PATs. From your experiments it seems clear the credential provider is working as expected if your username and password are correctly returned from the tool (the last log), so the issue is likely on the nuget client side. The first log where you're getting multiple attempts that's nuget client trying the credentials, getting 401, then requesting a new one (to which the same credential is returned). You can see that IsRetry is initially false, but then flips to True as the credentials aren't working.

Since this doesn't seem like a credential provider issue, have you hooked up fiddler or wireshark to capture the outgoing call from nuget client to the private repository? That would indicate if the same credentials are being used on those calls, or highlight the different between using the credential provider to pass credentials vs nuget passing credentials.

[chrisdecker1201]

@JohnSchmeichel Thank you for your quick response.

I tried to read something from the Wireshark output, but as everything is encrypted, and I'm not that familiar with Wireshark it was a dead end for me.

As you mentioned nuget as client, I tried directly using nuget to restore, but with the same response:

nuget restore proget_experimental.csproj -Verbosity detailed

    [CredentialProvider.115133]Running in plug-in mode
    [CredentialProvider.115133]Command-line v1.0.2+c2bc059db245a21f09d4e9afa92bba813c567488: "C:\Users\<USER>\.nuget\plugins\netfx\CredentialProvider.Microsoft\CredentialProvider.Microsoft.exe" -Plugin
    [CredentialProvider.115133]Handling 'Request' 'Initialize'. Time elapsed in ms: 6 - Payload: {"ClientVersion":"6.5.0","Culture":"en-US","RequestTimeout":"00:00:30"}
    [CredentialProvider.115133]Sending response: 'Request' 'Initialize'. Time elapsed in ms: 7
    [CredentialProvider.115133]Time elapsed in milliseconds after sending response 'Request' 'Initialize': 10
    [CredentialProvider.115133]Handling 'Request' 'GetOperationClaims'. Time elapsed in ms: 0 - Payload: {}
    [CredentialProvider.115133]Sending response: 'Request' 'GetOperationClaims'. Time elapsed in ms: 10
    [CredentialProvider.115134]Time elapsed in milliseconds after sending response 'Request' 'GetOperationClaims': 37
    [CredentialProvider.115134]Handling 'Request' 'SetLogLevel'. Time elapsed in ms: 1 - Payload: {"LogLevel":"Debug"}
    [CredentialProvider]Sending response: 'Request' 'SetLogLevel'. Time elapsed in ms: 3
    [CredentialProvider]Time elapsed in milliseconds after sending response 'Request' 'SetLogLevel': 10
    [CredentialProvider]Handling 'Request' 'GetAuthenticationCredentials'. Time elapsed in ms: 4 - Payload: {"Uri":"https://internalproget.server.com/nuget/Experimental/v3/index.json","IsRetry":false,"IsNonInteractive":false,"CanShowDialog":true}
    [CredentialProvider]Creating a progress reporter with interval: 00:00:02
    [CredentialProvider]Handling auth request, Uri: https://internalproget.server.com/nuget/Experimental/v3/index.json, IsRetry: False, IsNonInteractive: False, CanShowDialog: True
    [CredentialProvider]URI: https://internalproget.server.com/nuget/Experimental/v3/index.json
    [CredentialProvider]Using NuGetCredentialProvider.CredentialProviders.VstsBuildTaskServiceEndpoint.VstsBuildTaskServiceEndpointCredentialProvider to try to get credentials for https://internalproget.server.com/nuget/Experimental/v3/index.json.
    [CredentialProvider]VstsBuildTaskServiceEndpointCredentialProvider - IsRetry: False
    [CredentialProvider]VstsBuildTaskServiceEndpointCredentialProvider - Parsing json
    [CredentialProvider]VstsBuildTaskServiceEndpointCredentialProvider - Found credentials for endpoint https://internalproget.server.com/nuget/Experimental/v3/index.json
    [CredentialProvider]Sending response: 'Request' 'GetAuthenticationCredentials'. Time elapsed in ms: 92
    [CredentialProvider]Time elapsed in milliseconds after sending response 'Request' 'GetAuthenticationCredentials': 104
    [CredentialProvider]Handling 'Request' 'GetAuthenticationCredentials'. Time elapsed in ms: 0 - Payload: {"Uri":"https://internalproget.server.com/nuget/Experimental/v3/index.json","IsRetry":true,"IsNonInteractive":false,"CanShowDialog":true}
    [CredentialProvider]Creating a progress reporter with interval: 00:00:02
    [CredentialProvider]Handling auth request, Uri: https://internalproget.server.com/nuget/Experimental/v3/index.json, IsRetry: True, IsNonInteractive: False, CanShowDialog: True
    [CredentialProvider]URI: https://internalproget.server.com/nuget/Experimental/v3/index.json
    [CredentialProvider]Using NuGetCredentialProvider.CredentialProviders.VstsBuildTaskServiceEndpoint.VstsBuildTaskServiceEndpointCredentialProvider to try to get credentials for https://internalproget.server.com/nuget/Experimental/v3/index.json.
    [CredentialProvider]VstsBuildTaskServiceEndpointCredentialProvider - IsRetry: True
    [CredentialProvider]VstsBuildTaskServiceEndpointCredentialProvider - Found credentials for endpoint https://internalproget.server.com/nuget/Experimental/v3/index.json
    [CredentialProvider]Sending response: 'Request' 'GetAuthenticationCredentials'. Time elapsed in ms: 0
    [CredentialProvider]Time elapsed in milliseconds after sending response 'Request' 'GetAuthenticationCredentials': 0
    [CredentialProvider]Handling 'Request' 'GetAuthenticationCredentials'. Time elapsed in ms: 0 - Payload: {"Uri":"https://internalproget.server.com/nuget/Experimental/v3/index.json","IsRetry":true,"IsNonInteractive":false,"CanShowDialog":true}
    [CredentialProvider]Creating a progress reporter with interval: 00:00:02
    [CredentialProvider]Handling auth request, Uri: https://internalproget.server.com/nuget/Experimental/v3/index.json, IsRetry: True, IsNonInteractive: False, CanShowDialog: True
    [CredentialProvider]URI: https://internalproget.server.com/nuget/Experimental/v3/index.json
    [CredentialProvider]Using NuGetCredentialProvider.CredentialProviders.VstsBuildTaskServiceEndpoint.VstsBuildTaskServiceEndpointCredentialProvider to try to get credentials for https://internalproget.server.com/nuget/Experimental/v3/index.json.
    [CredentialProvider]VstsBuildTaskServiceEndpointCredentialProvider - IsRetry: True
    [CredentialProvider]VstsBuildTaskServiceEndpointCredentialProvider - Found credentials for endpoint https://internalproget.server.com/nuget/Experimental/v3/index.json
    [CredentialProvider]Sending response: 'Request' 'GetAuthenticationCredentials'. Time elapsed in ms: 0
    [CredentialProvider]Time elapsed in milliseconds after sending response 'Request' 'GetAuthenticationCredentials': 1
    [CredentialProvider]Handling 'Request' 'GetAuthenticationCredentials'. Time elapsed in ms: 0 - Payload: {"Uri":"https://internalproget.server.com/nuget/Experimental/v3/index.json","IsRetry":true,"IsNonInteractive":false,"CanShowDialog":true}
    [CredentialProvider]Creating a progress reporter with interval: 00:00:02
    [CredentialProvider]Handling auth request, Uri: https://internalproget.server.com/nuget/Experimental/v3/index.json, IsRetry: True, IsNonInteractive: False, CanShowDialog: True
    [CredentialProvider]URI: https://internalproget.server.com/nuget/Experimental/v3/index.json
    [CredentialProvider]Using NuGetCredentialProvider.CredentialProviders.VstsBuildTaskServiceEndpoint.VstsBuildTaskServiceEndpointCredentialProvider to try to get credentials for https://internalproget.server.com/nuget/Experimental/v3/index.json.
    [CredentialProvider]VstsBuildTaskServiceEndpointCredentialProvider - IsRetry: True
    [CredentialProvider]VstsBuildTaskServiceEndpointCredentialProvider - Found credentials for endpoint https://internalproget.server.com/nuget/Experimental/v3/index.json
    [CredentialProvider]Sending response: 'Request' 'GetAuthenticationCredentials'. Time elapsed in ms: 0
    [CredentialProvider]Time elapsed in milliseconds after sending response 'Request' 'GetAuthenticationCredentials': 1

What still works is nuget without CredentialProvider, when I've to insert the credentials manually.

I'll try to get access now to our internal server and hope I can find something in the logs there.

Is there a possibility to somehow see the difference between with and without CredentialProvider without Wireshark?

Additionally, I tried to build the CredentialProvider on my own, but getting of course the following error. Can I somehow ignore this?

The plugin at 'C:\Users\<USER>\artifacts-credprovider\CredentialProvider.Microsoft\bin\Debug\net461\CredentialProvider.Microsoft.exe' did not have a valid embedded signature.

[sergey-litvinov-work]

i'm not sure it's fully related but looks like we also have similar issue. we use .NET Core 3.1 in linux under docker for following image mcr.microsoft.com/dotnet/sdk:3.1-alpine. and we have a step there that installs the latest installcredprovider.sh like this and it started to throw a sh error

Step 9/20 : RUN wget -qO- https://raw.githubusercontent.com/Microsoft/artifacts-credprovider/master/helpers/installcredprovider.sh | sh
 ---> Running in d77b2f6b60bb
sh: v0.*: unknown operand
INFO: Creating the nuget plugin directory (i.e. /root/.nuget/plugins). 
Downloading from https://github.com/Microsoft/artifacts-credprovider/releases/latest/download/Microsoft.Net6.NuGet.CredentialProvider.tar.gz
INFO: credential provider netcore plugin extracted to /root/.nuget/
Removing intermediate container d77b2f6b60bb
 ---> ae45770ae9d6

The last time we used it was March 16 and it worked fine and output was

Step 9/20 : RUN wget -qO- https://raw.githubusercontent.com/Microsoft/artifacts-credprovider/master/helpers/installcredprovider.sh | sh
 ---> Running in 26cc8735d3a6
INFO: Creating the nuget plugin directory (i.e. /root/.nuget/plugins). 
Downloading from https://github.com/Microsoft/artifacts-credprovider/releases/latest/download/Microsoft.NuGet.CredentialProvider.tar.gz
INFO: credential provider netcore plugin extracted to /root/.nuget/
Removing intermediate container 26cc8735d3a6
 ---> edfe802ca579

and then it fails on restore step

Step 13/20 : RUN dotnet build --source "${ARTIFACTS_ENDPOINT}" SomeUnit.Tests.csproj
 ---> Running in c70b4c6e232c
Microsoft (R) Build Engine version 16.7.3+2f374e28e for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
It was not possible to find any compatible framework version
The framework 'Microsoft.NETCore.App', version '6.0.0' was not found.
  - The following frameworks were found:
      3.1.32 at [/usr/share/dotnet/shared/Microsoft.NETCore.App]

You can resolve the problem by installing the specified framework and/or SDK.

The specified framework can be found at:
  - https://aka.ms/dotnet-core-applaunch?framework=Microsoft.NETCore.App&framework_version=6.0.0&arch=x64&rid=alpine.3.16-x64
/usr/share/dotnet/sdk/3.1.426/NuGet.targets(128,5): error : Problem starting the plugin '/root/.nuget/plugins/netcore/CredentialProvider.Microsoft/CredentialProvider.Microsoft.dll'. Broken pipe [/src/Some.Tests/SomeUnit.Tests.csproj]
/usr/share/dotnet/sdk/3.1.426/NuGet.targets(128,5): error : Unable to load the service index for source https://some.pkgs.visualstudio.com/_packaging/some-Nuget/nuget/v3/index.json. [/src/SomeUnit.Tests/SomeUnit.Tests.csproj]
/usr/share/dotnet/sdk/3.1.426/NuGet.targets(128,5): error :   Response status code does not indicate success: 401 (Unauthorized). [/src/SomeUnit.Tests/SomeUnit.Tests.csproj]

Build FAILED.

[JohnSchmeichel]

@chrisdecker1201 try using fiddler if you can, you should be able to see the requests sent from NuGet to the remote server and inspect the headers and payload.

[sergey-litvinov-work]

@JohnSchmeichel should i still create an issue? we just added ENV USE_NET6_ARTIFACTS_CREDENTIAL_PROVIDER false so it fallbacks to older version and works for us.

If you think that it's good to create a defect - i can create it, but it's a question what should be default version - older .NET so it won't require any changes for older project, or new one and then older project need to force to use older version. And if the answer is newer should be default, then i guess we don't need a ticket

[chrisdecker1201]

Thank you for the tip with fiddler. Sadly I'm still confused. I'm not an expert in authentication but the main difference I see is that, when I try to use the credential provider it's using the NTLM and get a 401 response:

No Proxy-Authenticate Header is present.

WWW-Authenticate Header is present: Negotiate

WWW-Authenticate Header is present: NTLM

And when I use direct authentication with

dotnet nuget add source "https://internalproget.server.com/nuget/Experimental/v3/index.json" --name "ProGet Experimental (Basic Auth)" --username "username" --password "password"

it's using Kerberos and get a 200 response:

No Proxy-Authenticate Header is present.

WWW-Authenticate Header (Negotiate) appears to be a Kerberos reply:
...

[chrisdecker1201]

I will try to disable NTLM on the server tomorrow and try again. Maybe that's the solution.

[chrisdecker1201]

I'm not sure anymore if the issue is related to the CredentialProvider or more an issue of the server configuration I have.

I set Network security: Restrict NTLM: Outgoing NTLM traffic to remote servers to Deny all via Group Policy, but after that dotnet restore --verbosity detailed does not output a single CredentialProvider log

  X.509 certificate chain validation will use the default trust store selected by .NET.
  Running non-parallel restore.
  Reading project file C:\Users\<USER>\proget_experimental\proget_experimental.csproj.
  The restore inputs for 'proget_experimental' have changed. Continuing restore.
  Restoring packages for C:\Users\<USER>\proget_experimental\proget_experimental.csproj...
  Restoring packages for .NETCoreApp,Version=v6.0...
C:\Users\<USER>\proget_experimental\proget_experimental.csproj : error NU1301: Unable to load the service index for so
urce https://internalproget.server.com/nuget/Experimental/v3/index.json.
  Checking compatibility of packages on net6.0.
  All packages and projects are compatible with net6.0.
  Committing restore...

With Network security: Restrict NTLM: Outgoing NTLM traffic to remote servers set to Allow all:

  X.509 certificate chain validation will use the default trust store selected by .NET.
  Running non-parallel restore.
  Reading project file C:\Users\<USER>\proget_experimental\proget_experimental.csproj.
  The restore inputs for 'proget_experimental' have changed. Continuing restore.
  Restoring packages for C:\Users\<USER>\proget_experimental\proget_experimental.csproj...
  Restoring packages for .NETCoreApp,Version=v6.0...
  Using C:\Users\<USER>\.nuget\plugins\netcore\CredentialProvider.Microsoft\CredentialProvider.Microsoft.dll as a credential provider plugin.
      [CredentialProvider.070138]Running in plug-in mode
...

[JohnSchmeichel]

I agree that this looks more like an issue with NuGet or server configuration. Some relevant issues from the NuGet side that may help here:

NuGet/Home#5286
NuGet/Home#7841

In particular I suspect the Negotiate is the issue here as it will be used before the Basic credentials are used. You can try to use the -ValidAuthenticationTypes option to restrict to Basic only.

[JohnSchmeichel]

Recommend you follow up with the NuGet team at https://github.com/NuGet/Home if you're still experiencing issues with it authenticating with the external endpoint. Closing out this issue as the Artifacts credential provider is properly returning the provided credentials.

[chrisdecker1201]

@JohnSchmeichel I've get a response NuGet/Home#12546 (comment), but I'm not sure what to do.

If I understand the reponse correct the credentialprovider has an issue. At least I don't know what to do, to fix my issue with this response.

[JohnSchmeichel]

The credential provider only supports Basic, that's hard-coded in and always has been. So likely your external server needs to enable this option then if NuGet will only use Basic authentication from the credential provider if the server indicates that's supported. I thought NuGet will use the credentials in the same manner as if it was given them via nuget.exe sources add but unfortunately that's not the case, the authorization type is filtered.

[chrisdecker1201]

@JohnSchmeichel Wouldn't it be the correct solution, that I can configure the auth type in the credential provider? In my case kerberos.

[JohnSchmeichel]

Then basic auth and this credential provider isn't going to work for you here. Removing the Basic auth filter is not an option, that's required for the tokens the credential provider provides. Would recommend you use the dotnet add source ... version with username + password that you said works for you, or you can look at creating a custom credential provider that supports the semantics your endpoint requires.