Coder Social home page Coder Social logo

microsoft / powershell-featureflags Goto Github PK

View Code? Open in Web Editor NEW
22.0 7.0 12.0 127 KB

PowerShell module containing a Feature Flags implementation based on a local config file.

License: MIT License

PowerShell 97.60% Dockerfile 2.40%
powershell powershell-module windows linux macosx dotnet-core dotnetstandard feature-flags feature-toggles devops

powershell-featureflags's Introduction

Nuget Platforms FeatureFlags

PowerShell Feature Flags

This package contains a simple, low-dependencies implementation of feature flags for PowerShell, which relies on a local configuration file to verify if a given feature should be enabled or not.

The configuration file contains two sections:

  • stages: a section where roll-out stages are defined;
  • features: a section where each feature can be associated to a roll-out stage.

A roll-out stage is defined by a name and an array of conditions that the predicate must match in the order they are presented for the feature associated to the given stage to be enabled.

Stage names and feature names must be non-empty and must consist of non-space characters.

A feature can be assigned an array of stages that it applies to. In addition, it can also accept an environment variable array, and can optionally output an environment configuration file.

For more general information about feature flags, please visit featureflags.io.

Installation

This module is available from the PowerShell Gallery. Therefore, to install it for all users on the machine type the following from an administrator PowerShell prompt:

PS > Install-Module FeatureFlags

To install as an unprivileged user, type the following from any PowerShell prompt:

PS > Install-Module FeatureFlags -Scope CurrentUser

Simple example

Imagine to have a feature flag configuration file called features.json:

{
  "stages": {
    "test": [
      {"allowlist": ["test.*", "dev.*"]}
    ],
    "canary": [
      {"allowlist": ["prod-canary"]}
    ],
    "prod": [
      {"allowlist": ["prod.*"]},
      {"denylist": ["prod-canary"]}
    ]
  },
  "features": {
    "experimental-feature": {
      "stages": ["test"]
    },
    "well-tested-feature": {
      "stages": ["test", "canary", "prod"]
    }
  }
}

This file defines 3 stages: test, canary and prod, and 2 features: experimental-feature and well-tested-feature.

The intent of the configuration is to enable experimental-feature in test only (all predicates starting with test or dev), and to enable well-tested-feature in all stages.

Let's first read the configuration:

$cfg = Get-FeatureFlagConfigFromFile features.json

This step would fail if there is any I/O error (e.g., file doesn't exist), if the file is not valid JSON or if the file does not conform with the feature flags schema.

Let's now test a couple of predicates to verify that the configuration does what we expect:

PS > Test-FeatureFlag -config $cfg -Feature "well-tested-feature" -predicate "test1"
True 
PS > Test-FeatureFlag -config $cfg -Feature "well-tested-feature" -predicate "test2"
True 
PS > Test-FeatureFlag -config $cfg -Feature "well-tested-feature" -predicate "dev1" 
True                                                                                                                                
PS > Test-FeatureFlag -config $cfg -Feature "well-tested-feature" -predicate "prod-canary1"
True 
PS > Test-FeatureFlag -config $cfg -Feature "experimental-feature" -predicate "prod-canary1"
False 
PS > Test-FeatureFlag -config $cfg -Feature "experimental-feature" -predicate "test1"
True 
PS > Test-FeatureFlag -config $cfg -Feature "experimental-feature" -predicate "prod1"
False 

For more complex examples, please look at test cases. More examples will be added in the future (Issue #6).

Life of a feature flag

Feature flags are expected to be in use while a feature is rolled out to production, or in case there is a need to conditionally enable or disable features.

An example lifecycle of a feature flag might be the following:

  1. A new feature is checked in production after testing, in a disabled state;
  2. The feature is enabled for a particular customer;
  3. The feature is enabled for a small set of customers;
  4. The feature is gradually rolled out to increasingly large percentages of customers (e.g., 5%, 10%, 30%, 50%);
  5. The feature is rolled out to all customers (100%)
  6. The test for the feature flag is removed from the code, and the feature flag configuration is removed as well.

Here is how these example stages could be implemented:

  • Stage 1 can be implemented with a denylist condition with value .*.
  • Stages 2 and 3 can be implemented with allowlist conditions.
  • Stages 4 and 5 can be implemented with probability conditions.

Conditions

There are two types of conditions: deterministic (allowlist and denylist, regex-based) and probabilistic (probability, expressed as a number between 0 and 1). Conditions can be repeated if multiple instances are required.

All conditions in each stage must be satisfied, in the order they are listed in the configuration file, for the feature to be considered enabled.

If any condition is not met, evaluation of conditions stops and the feature is considered disabled.

Allow list

The allowlist condition allows to specify a list of regular expressions; if the predicate matches any of the expressions, then the condition is met and the evaluation moves to the next condition, if there is any.

The regular expression is not anchored. This means that a regex of "storage" will match both the predicate "storage" and the predicate "storage1". To prevent unintended matches, it's recommended to always anchor the regex.

So, for example, "^storage$" will only match "storage" and not "storage1".

Deny list

The denylist condition is analogous to the allowlist condition, except that if the predicate matches any of the expressions the condition is considered not met and the evaluation stops.

Probability

The probability condition allows the user to specify a percentage of invocations that will lead to the condition to be met, expressed as a floating point number between 0 and 1.

So, if the user specifies a value of 0.3, roughly 30% of times the condition is checked it will be considered met, while for the remaining 70% of times it will be considered unmet.

The position of the probability condition is very important. Let's look at the following example:

{
    "stages": {
        "allowlist-first": [
            {"allowlist": ["storage.*"]},
            {"probability": 0.1}
        ],
        "probability-first": [
            {"probability": 0.1}
            {"allowlist": ["storage.*"]},
        ]
    }
}

The first stage definition, allowlist-first, will evaluate the probability condition only if the predicate first passes the allowlist.

The second stage definition, probability-first, will instead first evaluate the probability condition, and then apply the allowlist.

Assuming there are predicates that do not match the allowlist, the second stage definition is more restrictive than the first one, leading to fewer positive evaluations of the feature flag.

Cmdlets

This package provides five PowerShell cmdlets:

  • Test-FeatureFlag, which checks if a given feature is enabled by testing a predicate against the given feature flag configuration;
  • Confirm-FeatureFlagConfig, which validates the given feature flag configuration by first validating it against the feature flags JSON schema and then by applying some further validation rules;
  • Get-FeatureFlagConfigFromFile, which parses and validates a feature flag configuration from a given file.
  • Get-EvaluatedFeatureFlags, which can be used to determine the collection of feature flags, from the feature flags config, that apply given the specified predicate.
  • Out-EvaluatedFeaturesFiles, which will write out feature flag files (.json, .ini, env.config) which will indicate which features are enabled, and which environment variables should be set.

A more complex example

NOTE: comments are added in-line in this example, but the JSON format does not allow for comments. Don't add comments to your feature flag configuration file.

{
  // Definition of roll-out stages.
  "stages": {
    // Examples of probabilistic stages.
    "1percent": [
      {"probability": 0.01},
    ],
    "10percent": [
      {"probability": 0.1},
    ],
    "all": [
      {"probability": 1},
    ],
    // Examples of deterministic stages.
    "all-storage": [
      {"allowlist": [".*Storage.*"]},
    ],
    "storage-except-important": [
      {"allowlist": [".*Storage.*"]},
      {"denylist": [".*StorageImportant.*"]},
    ],
    // Example of mixed roll-out stage.
    // This stage will match on predicates containing the word "Storage"
    // but not the word "StorageImportant", and then will consider the feature
    // enabled in 50% of the cases.
    "50-percent-storage-except-StorageImportant": [
      {"allowlist": [".*Storage.*"]},
      {"denylist": ["StorageImportant"]},
      {"probability": 0.5},
    ],
  },
  // Roll out status of different features:
  "features": {
    "msbuild-cache": {
      "stages": ["all-storage"],
      "environmentVariables": [
        { "Use_MsBuildCache": "1" }
      ]
    },
    "experimental-feature": {
      "stages": ["1percent"]
      // Environment Variables are optional
    },
    "well-tested-feature": {
      "stages": ["all"],
      "environmentVariables": [
        { "Use_TestedFeature": "1" }
      ]
    },
  }
}

Why JSON?

The configuration file uses JSON, despite its shortcomings, for the following reasons:

  1. it's supported natively by PowerShell, therefore it makes this package free from dependencies;
  2. it's familiar to most PowerShell developers.

Other formats, such as Protocol Buffers, while being technically superior, have been excluded for the above reasons.

Relationship to similar projects

There are some projects that allow to use Feature Flags in PowerShell:

  • microsoft/featurebits: this package uses a SQL database to store feature flags value. While the features provided by this project are similar, our FeatureFlags package does not need any external dependency to run, as features are stored in a local file.

  • SaaS (Software-as-a-Service) solutions: using an external service for feature flags has its pros and cons. They typically are much easier to manage and offer rich interfaces to manage the flags; however, the specific use case for which this library was born is to enable feature flags for PowerShell code which might not be able to open network connections: this requires the library and the feature flags definition to be co-located with the code (hermeticity).

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

powershell-featureflags's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

powershell-featureflags's Issues

CI broken @ master

https://dev.azure.com/PowerShell-FeatureFlags/PowerShell-FeatureFlags/_build/results?buildId=53&view=logs&j=068a9b68-2bbf-5452-0348-bd699ac0d931&t=ba3706fa-f190-5c4e-3095-1ff5f32c7e56

The error seems to be related to NJsonSchema:

Starting discovery in 1 files.
Microsoft.PowerShell.Commands.WriteErrorException: Error loading JSON schema
at <ScriptBlock>, /home/vsts/work/1/s/FeatureFlags.psm1: line 50
at <ScriptBlock>, /home/vsts/work/1/s/test/FeatureFlags.Tests.ps1: line 9
at <ScriptBlock>, /home/vsts/.local/share/powershell/Modules/Pester/5.0.2/Pester.psm1: line 2724
at Invoke-File, /home/vsts/.local/share/powershell/Modules/Pester/5.0.2/Pester.psm1: line 2733
at Invoke-BlockContainer, /home/vsts/.local/share/powershell/Modules/Pester/5.0.2/Pester.psm1: line 2662
at Discover-Test, /home/vsts/.local/share/powershell/Modules/Pester/5.0.2/Pester.psm1: line 1289
at Invoke-Test, /home/vsts/.local/share/powershell/Modules/Pester/5.0.2/Pester.psm1: line 2195
at Invoke-Pester<End>, /home/vsts/.local/share/powershell/Modules/Pester/5.0.2/Pester.psm1: line 4201
at <ScriptBlock>, /home/vsts/work/1/s/tools/run-tests.ps1: line 10
at <ScriptBlock>, /home/vsts/work/_temp/27c03e40-fcbc-4216-95ae-11ea3e7a5256.ps1: line 2
at <ScriptBlock>, <No file>: line 1

Fix handling of dependencies

Today dependencies are pulled in a very unorthodox way, by just specifying the version number and copying them into the package. This is unclean and needs to be fixed.

Add developer docs

Developer docs

Add a way to identify possible misconfigurations

There are some classes of errors which are not syntactic or semantic errors, but might indicate misconfiguration, and that could be considered warnings.

For example, the presence of stages that are not referenced by any feature could be considered worthy of a warning.

Migrate code to C# or F#

The PowerShell code works, but at this point it might be better to implement the cmdlet in pure C#.

This should buy us some more advantages, as the dotnet tool seems to not support very well the PowerShell sources, treating them more as resources than as source files, and requiring us to write ad-hoc scripts to do the "build" and publishing of the code.

This would also remove the ugly part of the code where we try to find the JSON and JSON Schema DLLs and load assemblies directly from those files.

Unit tests do not work with Pester 5.x

It looks like unit tests don't work with Pester 5.x.

I was able to reproduce locally the failures in #27, and they seem to happen only when unit tests run under a recent Pester build (5.1.0 rather than version 4.10.1 which is the one the test script installs if it finds no Pester).

The test code does not follow the rules for Pester 5.x code (https://github.com/pester/Pester#pester-500).

We may want to investigate porting unit tests to Pester 5.x, or figuring out if there is a way to always run tests with Pester 4.x.

The ubuntu-latest runners for GitHub Actions have Pester 5 installed.

Make the repository public

We should make it public roughly at the same time of uploading it to PSGallery (#2) , so everyone can use it and its source code together.

Probability condition tests are flaky

 Context Probability condition
      [+] Always allows with probability 1 125ms
      [-] Always rejects with probability 0 219ms
        Expected $false, but got $true.
        472:             Test-FeatureFlag "not-launched" "storage-important/master" $config | Should -Be $false
        at <ScriptBlock>, D:\a\1\s\test\FeatureFlags.Tests.ps1: line 472

Those tests sometimes fail.

Include complete usage examples

Ideally, one in PowerShell and one in batch. The examples should be complete and self-contained.

The README today just explains the format of the JSON configuration file, but doesn't really explain how to use the provided cmdlets.

Publish the script on the PowerShell Gallery

Will need

  • a module definition (.psd1)
  • a way to build a package to be updated
  • manual upload of the first version

Upload of new versions might be automated, but I don't think we'll release new versions often enough to warrant automation.

Lack of CmdletBinding prevents ErrorAction from propagating

It seems that not having [CmdletBinding()] for the functions is preventing an explicitly passed -ErrorAction from propagating to child calls. For example, if I call Get-FeatureFlagConfigFromFile -jsonConfigPath $f -ErrorAction SilentlyContinue where $f is a path that does not exist, you will get a halting error; however, if I add [CmdletBinding()] to that function, the halting error moves to the child call Confirm-FeatureFlagConfig - and if I add [CmdletBinding()] to that function, then I get back $null like Get-FeatureFlagConfigFromFile code is written.

To reproduce:
Call $c = Get-FeatureFlagConfigFromFile -jsonConfigPath $f where $f is a path that does not exist
then
Call $c2 = Get-FeatureFlagConfigFromFile -jsonConfigPath $f -ErrorAction SilentlyContinue where $f is a path that does not exist

$c2 should be returned without outputting an error to the console, and be $null
I thought this was working in earlier tests, but trying it now, -ErrorAction is not being passed on.

I'm using version 2.1.0 of FeatureFlags in Powershell 5 with the following $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.19041.2673
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.19041.2673
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Publish to nuget.org

Useful if there are customers that get all dependencies from nuget, instead of depending on the PS Gallery, or if there are people that only look in nuget feeds rather than PS Gallery.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.