microsoft / mu_crypto_release

Release platform for Mu CryptoBin

Mu Crypto Release Repository

This project behaves like a FW platform. It consists of scripts, pipelines, and submodules to perform a binary release of EDK2 CryptoPkg drivers to power the BaseCryptOnProtocol/Ppi infrastructure.

These releases will be tagged and tracked in this repo, but published to the Mu Nuget feed.

This repository is part of Project Mu. Please see Project Mu for details https://microsoft.github.io/mu

Repo Scripts

Here is a brief description of the scripts and how they are used.

CommonBuildSettings.py

This script is not to be used directly, but is a configuration module to be consumed by the other scripts in this repo (or used directly as a Stuart settings module). If any submodules or scopes need to be changed, this is the place to do it.

Note that any changes to names might require additional changes in other scripts that look for those names when copying files around.

SingleFlavorBuild.py

This script will run an EDK2 build of a single flavor and a single target of CryptBin. Example: TINY_SHA,DEBUG or STANDARD,RELEASE. This is because a single EDK2 build run requires a single and flavor target and so each run must be set up independently. This script is called multiple times by the MultiFlavorBuild.py script.

Should have good support for -h and print a useful help menu.

MultiFlavorBuild.py

This script organizes the build for an entire release by coordinating multiple calls to SingleFlavorBuild.py. Multiple targets, flavors, and architectures can be passed into this script to be built in sequence. This is the main entry point for the build from the pipeline and is the ideal entry point for any local builds.

Calling this script without any parameters default to all available targets, architectures, and flavors.

Should have good support for -h and print a useful help menu.

AssembleNugetPackage.py

This script will take in a directory of build artifacts and reorganize them into the Nuget package layout.

Should have good support for -h and print a useful help menu.

Building on a Local Dev System

Building locally for test consists of two primary pieces:

• Run the MultiFlavorBuild.py script to build the drivers, depexes, and other release collateral.

• Run the AssembleNugetPackage.py script to organize the release collateral into the format for

  the release package. Just point at the Build directory as the input directory.

The steps are:

1. Make sure that you've updated your pip requirements
2. Run the MultiFlavorBuild.py script with --setup
3. Run the MultiFlavorBuild.py script with --update
4. Run the MultiFlavorBuild.py script with whatever flavors and architectures you would like in your binary pacakge. This is the primary build and may take a while
5. Run the AssembleNugetPackage.py script and point at the Build directory for your local build (completed in step 4). There are extra parameters you can use to configure the package, documented in the help for the script

Releasing a Pipeline Build

TODO: How to support the security patch repos? (This isn't essential if there are no security

patches for CryptoPkg, but it's a necessary piece to figure out for the process.)

Most releases will be performed on a new EDK2 integration cycle (to re-enable the CryptoBin dependencies for the Basecore release branch). As such, most integrations will need all of the following steps.

If you're already on an existing release branch and you've just updated CryptoPkg and want to release a new CryptoBin, you can skip to the Regular Release Steps.

First Steps (for a new integration)

While performing the Basecore integration:

1. Disable the edk2-basecrypto-driver-bin extdep until a new release can be generated for this integration
   • In the file CryptoPkg/Driver/Bin/BaseCryptoDriver_ext_dep.json:
   • Set the scope to "global-disabled"
   • Set the version back to "0.0.0" (for safety so that we don't accidentally mis-version it when re-enabling)

Then, in this repo:

1. Take a look at the .gitmodules file and cross reference all required submodules
   • Make sure that all submodules have a matching release/* branch and that they have all been tagged as *_CIBuild
   • Example: if we're trying to make a new package on the release/202202 branch, we must first check Basecore and Silicon Arm Tiano to make sure that they have 2202_CIBuild tags, proving that they're valid to perform a new release
2. Create the new release branch in this repo
3. Update the version_major_minor parameter in the .azuredevops/pipelines/ReleaseBuild.yml file
4. Update pip-requirements.txt file to match Basecore for this release
5. Update pipeline tools to match Basecore for the new release (right now, this is just the python_version variable)

Regular Release Steps

In this repo:

1. Update to the correct release branches for each submodule in .gitmodules
2. Pull the correct commit for each submodule
3. Determine whether any configuration or PCDs need to change. This configuration is outside the scope of this document. Please refer to the greater documentation around CryptoBin and BCOP
4. Generate the new packaging files. These files are created by a script that lives in Mu Crypto Release
   • Script lives at CryptoBinPkg/Driver/Packaging/generate_cryptodriver.py
   • Running this with no arguments should be an acceptable default. Refer to the script help for information on the possible arguments
   • This script needs to be executed from within a valid Python venv configured for Mu
5. Compare the changes and stage them for PR into Mu Crypto Release
   • Total changes should affect dozens of files in CryptoBinPkg, most of which live in CryptoBinPkg/Driver/Bin directory
   • For most releases, these changes should only be timestamps. If they are anything other than timestamps, make sure you understand why and make sure they are intended
   • IMPORTANT NOTE If any new functions are introduced or any existing crypto family is updated to include new functions (or the prototypes change), you must update the EDKII_CRYPTO_VERSION in CryptoBinPkg/Driver/Packaging/Crypto.template.h
6. Submit your PR to Mu Crypto Release

Once the server is updated for the new release, run the release pipeline on the new branch. The release pipeline is located in the public Project Mu DevOps organization. To release a new version:

1. Go to the release pipeline
2. Run pipeline and select your branch

3. The following parameters are currently available:

   1. If you're confident in this build, you can go ahead and click the "Publish Nuget Package" checkbox
   2. It's possible to swap the VM image and build toolchain to Linux/GCC5
   3. The Major and Minor version is set by default in the pipeline (updated on each release), but can be overridden
   4. The Patch version must be set on each release. This must be manually checked for uniqueness. See here for the currently published versions
   5. The Version Label is optional. For example, a Version Label might be -beta for version X.Y.Z-beta. If you don't want a version label at all, set this to None and the pipeline will ignore it entirely

Once successfully released, tag the commit with the version (e.g. 2022.02.1) and push tag to the server.

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct https://opensource.microsoft.com/codeofconduct/

For more information see the Code of Conduct FAQ https://opensource.microsoft.com/codeofconduct/faq/ or contact [email protected]. with any additional questions or comments.

Contributions

Contributions are always welcome and encouraged! Please open any issues in the Project Mu GitHub tracker and read https://microsoft.github.io/mu/How/contributing/

Copyright & License

Copyright (C) Microsoft Corporation
SPDX-License-Identifier: BSD-2-Clause-Patent

mu_crypto_release's Issues

[Feature]: Add Runtime DXE shared crypto support

Feature Overview

We don't have a direct shared crypto replacement for RuntimeCryptoLib at the moment.

Feature tracks adding a path for Runtime DXE drivers (like VariableRuntimeDxe) to also use shared crypto. This is important since the previous RuntimeCryptLib library instance was remove from CryptoPkg recently alongside the openssl submodule.

Solution Overview

Support shared crypto with Runtime DXE.

Alternatives Considered

No response

Urgency

Medium

Are you going to implement the feature request?

I will implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Feature]: Run BaseCryptLib Unit Test (EFI Shell tests) in this Repo

Feature Overview

The BaseCryptLibUnitTestApp was being run in mu_tiano_platforms using crypto source code from MU_BASECORE/CryptoPkg. With the source backed instance of BaseCryptLib removed, mu_tiano_platforms will solely integrate crypto from the shared crypto binary.

The instances of BaseCryptLib that support dynamic interfaces (i.e. the PPI/Protocol) do not support all of the functions tested by BaseCryptLibUnitTestApp. Also, it would be ideal to test crypto binaries as part of their release flow.

Solution Overview

Run BaseCryptLibUnitTestApp in mu_crypto_release on PRs and releases.

Alternatives Considered

No response

Urgency

Low

Are you going to implement the feature request?

Someone else needs to implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

Test generating new crypto binaries with the repo changes.

With the sweeping repo changes it's important to make sure we're still able to generate new crypto binaries with all the changes. If there are issues with the process fixes will be necessary.

Investigate crypto providers other than openssl.

Our current crypto implementation is based on openssl and has been for years. Unfortunately, we've had some issues continuing to use Openssl for this purpose such as:

1. Binary size bloat
2. Code readability and understanding
3. Lack of ability to modify the codebase

For these reasons we're exploring different sources for our underlying crypto implementation. Any findings/updates will be posted here.

[Feature]: Add Code coverage results to the crypto binary

Feature Overview

With Crypto being such a critical path in firmware it makes sense to add code coverage to the binary. This will provide the benefit of confidence in our binaries functionality.

Solution Overview

Add code coverage testing and results to the binary build pipeline.

Alternatives Considered

No response

Urgency

High

Are you going to implement the feature request?

I will implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Feature]: Standalone MM support

Feature Overview

Currently the Shared crypto binaries produced are only availble for PEI, DXE and SMM. With growing interest in Standalone MM having a binary for it makes sense.

Solution Overview

1. Generate a standalone MM binary for all the different Shared crypto flavors.
2. Separate DXE and SMM crypto into their own fdf files.
3. Include a Standalone MM fdf file.
4. Update the publishing script to also publish the new Standalone MM binaries.

Alternatives Considered

No response

Urgency

Medium

Are you going to implement the feature request?

Someone else needs to implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

Migrate Openssl from 1.1.1 to 3

We're currently using openssl 1.1.1 for our binary generation and it's no longer supported as of September 2023.
We'll need to move to openssl 3.0 and update our crypto implementations to match it.

[Feature]: Move the shared crypto generating driver from MU_BASECORE to this repo.

Feature Overview

Currently the code to generate the shared crypto driver lives in MU_BASECORE. It makes sense to consolidate it with the actual crypto implementations in this repo.

Solution Overview

1. Move the crypto driver content to this repo.
2. Update the published binary to include .map files and symbols files.
3. Add a readme to the binary as well (that includes integration steps)

Alternatives Considered

No response

Urgency

Low

Are you going to implement the feature request?

I will implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Bug]: Issues with build AARCH64 with the VS2022 toolchain

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

Builds results for AARCH64 using the VS2022 toolchain don't meet expectations. Many memory sections instead of the two expected.

Additionally when testing issues are ran into.

Expected Behavior

1. Two memory sections
2. Platforms are able to use the binaries for AARCH64

Steps To Reproduce

1. Build for AARCH64 locally or with the pipeline.
2. Examine the build to see the incorrect number of memory sections.

Build Environment

- OS(s): Windows
- Tool Chain(s): VS2022
- Targets Impacted: AARCH64 crypto reliant platforms

Version Information

Top of release/202302

Urgency

Low

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

Investigate any problems with current crypto binaries.

Check the functionality of our currently published crypto binaries. Confirm if everything behaves as expected or if there are adjustments necessary in terms of the crypto functions supported in our different flavors.

Possible issues: Lack of SHA384 and SHA512 support in the STANDARD flavor binary. CONFIRMED

VariableSmm.c calls VariableWriteServiceInitializeSmm which eventually leads to calling AuthVariableLibInitialize which calls
SHA384 and SHA512 context functions.

[Feature]: Migrate MbedTLS and it's BaseCryptLib implementation into this repo.

Feature Overview

Currently all crypto functionality in project MU is found in the CryptoPkg in MU_BASECORE and is free to reference for anyone consuming it. To reach our goal of having everyone move to our Shared Crypto binary system we're moving the functional crypto implementations into this repo where we'll build the binaries.

Solution Overview

Migrate the MbedTLS submodule and BaseCryptLib implementation of it to this repo.

Alternatives Considered

No response

Urgency

Medium

Are you going to implement the feature request?

I will implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Feature]: evaluate the value of flavors

Feature Overview

Flavors allowed a creator to trim the functionality to match their needs but flavors added significant complexity and inconsistency to the crypto provider. As we look to scale the crypto binary out more broadly the complexity and incompatibility of flavors is not worth the savings.

Solution Overview

remove flavors and provide more consistent api

Alternatives Considered

No response

Urgency

Low

Are you going to implement the feature request?

Someone else needs to implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

AArch64 implementation of CryptoRuntimeDxe

Feature Overview

Hi,
On some specific platforms it is impossible to use a custom trustzone environment or Arm's TFA or Smm. As a result, platforms wishing to use VariableRuntimeDxe need to take a dependency on AuthVariableLib and thus a runtime implementation of BaseCryptLib. While the implementation of a runtime BaseCryptLib provider as well as a CryptoRuntimeDxe was made available recently, CryptoRuntimeDxe is not built for aarch64 and it is impossible for such platforms to continue using VariableRuntimeDxe with AuthVariableServices due to changes that removed BaseCryptLib from the basecore repository.

Solution Overview

Please provide a pre-built binary of CryptoRuntimeDxe for AArch64, solving above scenario as well as inf files pointing towards non existent efi binaries in the current nuget package.

Alternatives Considered

No response

Urgency

Medium

Are you going to implement the feature request?

Someone else needs to implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Documentation]: Make a shared crypto integration guide

Request Description

Create a shared crypto guide to describe how to ingest the SHARED CRYPTO binaries. This needs to include what random libraries are included as well as flags and debug library expectations.

Are you going to make the change?

I will make the change

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Documentation]: Explain how to build OpenSslPkg

Request Description

The build documentation (and scripts like SingleFlavorBuild.py) currently focus on building CryptoBinPkg. The build process for OpensslPkg needs to be documented.

Are you going to make the change?

Someone else needs to make the change

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Feature]: Protocol needs to be backward and forward compatible

Feature Overview

changes to the crypto protocol must not break compatibility and should allow a caller of a different version to still work (with limited functions). The simplest method of achieving this to only allow changes added to the end.

Solution Overview

only add functions to end
put tooling in place to enforce this

Alternatives Considered

No response

Urgency

Medium

Are you going to implement the feature request?

Someone else needs to implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Documentation]: Add crypto flavor matrix

Request Description

The release from this repo includes various crypto flavors. The functionality included in each flavor is key integration information needed for consumers to include the proper binaries in a platform firmware.

Ideally, this would be in a markdown table in the main readme (similar to the crypto family service table here).

Are you going to make the change?

Someone else needs to make the change

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Feature]: Update repo to have a system for integrating with upstream

Feature Overview

Because we moved the different crypto implementations into mu_crypto_release we need to integrate edk2 changes into this repo.

Solution Overview

Have a document that describes the integration process. Additionally it might make sense to clean up the commit history to make integration easier.

Alternatives Considered

No response

Urgency

Low

Are you going to implement the feature request?

I will implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

Test effects of enabling SHA384 and SHA512 on platforms.

Enabling additional crypto algorithms can change the PCR measurements and/or effect measured boot. Check if enabling SHA384 and SHA512 algorithms cause such changes.