v2024.2.0: PowerShell Terminal Fails to Load Due to PSES change to non-Windows-trusted Certificate about vscode-powershell

[JustinGrote]

The Reactive DLL hasn't changed but it may have been being incorrectly signed as Microsoft First Party when it is in fact Microsoft Community, that was my main query to @andyleejordan that may have been "fixed" but now breaks the ability for the extension to be wholly "pre-signed" on windows systems.

This is more of a legal issue than a technical one, so I have to defer to Andy as to whether that DLL can continue to be signed as it was before since it hasn't changed, but future versions may still have the problem anyways. I imagine this may be a change in policy as supply-chain attacks such as Solarwinds, xz, etc. have probably made Microsoft extremely sensitive about these sorts of things.

[cyrkin]

No certificates are trusted on your machine by default. They are signed with updated certificates, you will need to trust those when running AllSigned.

That's not true - there is a list of Trusted root certificates which gets updated by Microsoft and Windows does download them automatically.

That Root certificate ameroot is not an official root certificate - so you have problems with AppLocker.

Issue is not resolved.

Couldn't agree more.

[SebCT]

This issue has been labeled as resolved, please verify the provided fix (or other reason).

Not resolved yet

[JustinGrote]

Also, doing a sigcheck looks like the DLL itself hasn't changed in PSES from 2024.2.0 from 2024.3.2, so was this just a new detection that was a previous mis-signing?

[JustinGrote]

As an aside, I found the reference to System.Reactive which @TylerLeonhardt added 4 years ago
C:\Users\JGrote\Projects\PowerShellEditorServices\test\PowerShellEditorServices.Test.E2E\Processes\ServerProcess.cs

While we could unwind that, It is also referenced in OmniSharp.Extensions.JsonRPC as a transitive dependency

      "OmniSharp.Extensions.JsonRpc": {
        "type": "Transitive",
        "resolved": "0.19.9",
        "contentHash": "utFvrx9OYXhCS5rnfWAVeedJCrucuDLAOrKXjohf/NOjG9FFVbcp+hLqj9Ng+AxoADRD+rSJYHfBOeqGl5zW0A==",
        "dependencies": {
          "MediatR": "8.1.0",
          "Microsoft.Extensions.DependencyInjection": "6.0.1",
          "Microsoft.Extensions.Logging": "6.0.0",
          "Nerdbank.Streams": "2.10.69",
          "Newtonsoft.Json": "13.0.3",
          "OmniSharp.Extensions.JsonRpc.Generators": "0.19.9",
          "System.Collections.Immutable": "5.0.0",
          "System.Reactive": "6.0.0",
          "System.Threading.Channels": "6.0.0"
        }

So we wouldn't be able to remove the assembly due to that entrenched dependency.

[JustinGrote]

Reopening the issue to address System.Reactive signing status.

[JustinGrote]

Maybe it is in fact having a problem with those other DLLs, it just is erroring and exiting first on reactive

[andyleejordan]

That lines up with what I heard internally about AppLocker (that while it still exists, it's mostly been deprecated in favor of WDAC due to better security boundaries, and hence its lack of support for dual-signing). Essentially, with the migration to an internal build/sign/release system called OneBranch, I am required to sign all the libraries I ship. When those are signed with a third-party certificate already, I must counter-sign it with that Microsoft 3rdparty Application Component certificate. I don't really have a way around this, the pipeline fails otherwise.

[SebCT]

Since 2024.2.0 some components (DLLs and PS1/PSM1/PSD1) of the module are signed with a untrusted certificate/CA called "ameroot" - here is the output with sigcheck from Sysinternals, also affects AppLocker if you use application allowlisting:

Signers: Microsoft Azure Code Sign
Cert Status: The revocation status of the certificate or one of the certificates in the certificate chain is unknown., Error 65536 (0x10000), The revocation status of the certificate or one of the certificates in the certificate chain is either offline or stale.
Valid Usage: 1.3.6.1.4.1.311.91.1.1, Code Signing
Cert Issuer: AME CS CA 01
Serial Number: 36 00 00 01 DF 73 81 97 16 BE 32 FD 0D 00 02 00 00 01 DF
Thumbprint: 1226440E939A24EB202C2A517CE13F8326EFDE60
Algorithm: sha256RSA
Valid from: 03:33 20.01.2024
Valid to: 03:33 19.01.2025
AME CS CA 01
Cert Status: The certificate or certificate chain is based on an untrusted root., The revocation status of the certificate or one of the certificates in the certificate chain is unknown., The revocation status of the certificate or one of the certificates in the certificate chain is either offline or stale.
Valid Usage: KDC Auth, Server Auth, Client Auth, Enrollment Agent, 1.3.6.1.4.1.311.21.6, Document Signing, 1.3.6.1.4.1.311.21.6, OCSP Signing, IPSEC IKE Intermediate, DNS Server, EFS Recovery, EFS, Private Key Archival, Smartcard Logon, 1.3.6.1.4.1.311.20.2.3, Code Signing, 1.3.6.1.4.1.311.91.1.1, 1.3.6.1.4.1.311.91.2.1, 1.3.6.1.4.1.311.91.3.1, 1.3.6.1.4.1.311.91.5.1, 1.3.6.1.4.1.311.91.4.1, 1.3.6.1.4.1.311.91.4.2
Cert Issuer: ameroot
Serial Number: 1F 00 00 00 51 EA 8F F6 9C 73 0C A8 3B 00 00 00 00 00 51
Thumbprint: E3981E455AAFF0C851FF1D3C4EF41EE6A4C087F5
Algorithm: sha256RSA
Valid from: 20:44 21.05.2021
Valid to: 20:54 21.05.2026
ameroot
Cert Status: The certificate or certificate chain is based on an untrusted root.
Valid Usage: All
Cert Issuer: ameroot
Serial Number: 25 DA CB 55 C9 C6 77 81 40 9E 56 94 82 DE 4D FE
Thumbprint: 413E8AAC6049924B178BA636CBAF3963CCB963CD
Algorithm: sha256RSA
Valid from: 00:52 25.05.2016
Valid to: 00:57 25.05.2026

Please MS fix this asap and sign this components with a trusted root CA - workaround is now to revert to V2024.0.0 - even the newer prerelease version has the problem with untrusted CA

[andyleejordan]

No certificates are trusted on your machine by default. They are signed with updated certificates, you will need to trust those when running AllSigned.

[andyleejordan]

See more info here: #3741 (comment)

[github-actions]

This issue has been labeled as resolved, please verify the provided fix (or other reason).

[SebCT]

No certificates are trusted on your machine by default. They are signed with updated certificates, you will need to trust those when running AllSigned.

That's not true - there is a list of Trusted root certificates which gets updated by Microsoft and Windows does download them automatically.

That Root certificate ameroot is not an official root certificate - so you have problems with AppLocker.

Issue is not resolved.

[github-actions]

This issue has been labeled as resolved, please verify the provided fix (or other reason).

[github-actions]

This issue has been labeled as resolved, please verify the provided fix (or other reason).

[JustinGrote]

@andyleejordan I can reproduce this on a clean Win11 box, though it's just silently failing for me with allsigned on even with all the verbosity turned on. The signing cert for PSES changed with the latest release, and it's not one that's trusted by Windows by default. Does this have to do with the build changes you made perhaps?

[andyleejordan]

Ahh well that was embarassing. There was a typo in our pipeline that lead to a silent failure to use the correct certificate, so it defaulted to "Microsoft Azure Code Sign" instead of what it was supposed to be set to, "Microsoft Corporation." And that only shows up as a problem in particular circumstances (as you all have found). Hotfix release is on its way with this resolved. Thanks @JustinGrote for pointing that out. #4977

[github-actions]

This issue has been labeled as resolved, please verify the provided fix (or other reason).

[andyleejordan]

Ok v2024.2.1 is out and fixes this. Sorry!

[github-actions]

This issue has been labeled as resolved, please verify the provided fix (or other reason).

[JustinGrote]

Confirmed Fixed in my demo enviroment with AllSigned on.

[github-actions]

This issue has been labeled as resolved, please verify the provided fix (or other reason).

[cyrkin]

Confirmed fixed with v2024.2.1 and AllSigned on.
Big thanks to everyone who helped resolve the issue.

[JustinGrote]

@cyrkin thank you for persisting with additional information beyond our initial recommendation! Otherwise we wouldn't have caught it.

[andyleejordan]

Exactly. There have been recurring issues with AllSigned in the past, so it took some extra digging to figure out what happened here. The specified key was right...the field was mistyped/copied as signing_environment instead of signing_profile and then just silently defaulted to the wrong key. Oh YAML.

[SebCT]

Thanks, working nearly perfect except one DLL: AppLocker is still having a problem with the certificate(s) and counter signing of this file here:

.vscode\extensions\ms-vscode.powershell-2024.2.1\modules\powershelleditorservices\bin\common\System.Reactive.dll

The DLL "System.Reactive.dll" is counter signed from yesterday with a "Microsoft 3rd Party Application Component" certificate, but also still countersigned from the past in 2023 with the signer "Reactive Extensions for .NET (.NET Foundation)", which worked perfect in the previous version 2024.0.0 - could you please investigate this and sign it with only one certificate/counter signing? It's important to allowlist this DLL with publisher rule in AppLocker - thanks in advance.

[andyleejordan]

I noticed that when setting this up actually, and unfortunately the System.Reactive.dll that's being restored from NuGet (version 6.0.0) is not first-party signed by "Microsoft Corporation" but by "Reactive Extensions for .NET (.NET Foundation)" so I have to dual-sign it as "Microsoft 3rd Party Application Component".

[SebCT]

I see, but could the security engineers at Microsoft take a look because of AppLocker Application Allowlisting? Because with this new counter signed DLL a publisher rule doesn't work anymore, with the System.Reactive.dll from V2024.0.0 it works perfect - and all the other DLLs from the new V2024.2.1 work perfect with publisher rules, too. Thanks in advance, would be very happy when this issue could be resolved, security is very important for us ☺️

[JustinGrote]

@andyleejordan is the Reactive DLL a direct dependency or something transitive from csharp-language-server?

[SebCT]

Also, doing a sigcheck looks like the DLL itself hasn't changed in PSES from 2024.2.0 from 2024.3.2, so was this just a new detection that was a previous mis-signing?

As i can confirm, the former version 2024.2.0 and the pre-release version 2024.3.2 have the false and untrusted 'ameroot' certificate.

The DLL's of Hotfix Version 2024.2.1 are signed correctly, except that one DLL "System.Reactive.DLL" - but in version 2024.0.0 the DLL works with publisher rule in AppLocker. So the best would be if the System.Reactive.DLL in 2024.2.1 has the same signing as in 2024.0.0 where it works perfectly - ideally it's the same DLL because of no code change inside the DLL?

[andyleejordan]

You've written it out exactly Justin. When switching to the new signing system, it detected that System.Reactive.dll is Microsoft Community and signed as such, and wouldn't let me sign it as first-party Microsoft Corporation. It was an error that it was previously, and allowed by a system that I no longer have access to. However, all the OmniSharp DLLs etc. are being signed with the same certificate, so I'm not sure why AppLocker is having an issue with this one and not those? We've never published exclusively first-party DLLs because of those dependencies.

[github-actions]

This issue has been labeled as resolved, please verify the provided fix (or other reason).

[SebCT]

If i use the React DLL from V2024.0.0, where only ONE CERTIFICATE is in the DLL, it works perfect - with the new one from V2024.2.1 with TWO CERTIFICATES it doesn't work. If i sign it with my own code signer certificate, which is allowlisted in applocker, it works also perfect - and it has only ONE CERTIFCATE after i signed it.

[andyleejordan]

I have triple checked this, and it's correct as-is, and I am being required to dual-sign it. Please ask your admins to update their AppLocker rules to accept System.Reactive.dll to be signed by Reactive Extensions for .NET (.NET Foundation) and Microsoft 3rd Party Application Component. If I had to hazard a guess, they wrote a wildcard rule for System.*, not expecting a third-party DLL to be prefixed by System. such as this one.

[SebCT]

I have triple checked this, and it's correct as-is, and I am being required to dual-sign it. Please ask your admins to update their AppLocker rules to accept System.Reactive.dll to be signed by Reactive Extensions for .NET (.NET Foundation) and Microsoft 3rd Party Application Component. If I had to hazard a guess, they wrote a wildcard rule for System.*, not expecting a third-party DLL to be prefixed by System. such as this one.

Thanks a lot for your checks and your comment - i'm also in the lucky position to be the admin for AppLocker rules, too :-)

The AppLocker rule is correct and works with V2024.0.0, but the issue, as also mentioned here in the past, is the "dual signing" - AppLocker does only read the first certificate, not the second one if you make publisher rules.

This is a main problem and design issue from the AppLocker mechanism, but this mechanism was designed from Microsoft back in 2007. In WDAC this is not a problem anymore, because WDAC reads all signatures from a PE file and does not rely on the trusted root store from Windows. I can confirm, as i'm also doing WDAC, that it works there without a problem.

Here is the allowlisting publisher rule from AppLocker, it works perfect with V2024.0.0 and "single-signing" - the properties "Binaryname" and "Binaryversion" have a wildcard "*", so it does not rely on the version number of filename, only the Publisher and the productname of the PE File/Header:

In the mean time i engineered further and did find out 2 things, which have a security impact for an enterprise environment with strict allowlisting:

1.) If you make a publisher rule that every publisher is allowed (wildcard for every publisher), it works -> not secure for an enterprise solution.

2.) If you make a publisher rule that every Microsoft signed binary is allowed (wildcard for Microsoft), it also works -> that enables also LOLBIN scenarios, that are also not secure enough for an enterprise environment.

3.) A file hash rule would work, but these are a problem if the DLL will be changed in a future release of the PowerShell Module, so it has an impact in the production environment and does not work for us.

So it is a design problem from AppLocker, maybe Microsoft can repair this issue for Dual signed PE files, but i'm not sure if Microsoft will do enhancements for AppLocker, as WDAC is the future allowlisting mechanism for them.

One final question because i'm curious: what is the reason for dual-signing a file, which hasn't changed from 2024.0.0 to 2024.2.1? Could it be single-signed with a new, fresh certificate?

Thanks for the help :-)

Update: i did resolve it with a "hack" - hope this will help some other AppLocker admins, too :-)

Now you have to manually create 2 publisher rules with 2 different Publishers:

1st rule:

Publisher: O=REACTIVE EXTENSIONS FOR .NET (.NET FOUNDATION), L=REDMOND, S=WASHINGTON, C=US
Productname: SYSTEM.REACTIVE (NETSTANDARD2.0)

2nd rule:

Publisher: O=MICROSOFT CORPORATION, L=REDMOND, S=WASHINGTON, C=US
Productname: SYSTEM.REACTIVE (NETSTANDARD2.0)

Of course you can tighten it with filename and version if someone want. These rules can be made manually in the GUI with custom editing of the Publisher string or with the PowerShell AppLocker module/commandlets with a prepared AppLocker XML rules file and i confirm it works now. The GUI (mmc.exe) cannot read the 2nd certificate, it's still a design flaw from AppLocker, but won't be repaired anymore i guess.

[SebCT]

Thanks for the confirmation concerning depreciation of AppLocker, very useful information 🙏🏻