soluto / tweek

Tweek - an open source feature manager

[email protected]

What is Tweek?

Tweek is an open source feature management solution for customizing applications and system behavior without deploying new code.

Tweek aims to be a complete open-source alternative to other industry feature/configuration/experiment management solutions such as Facebook's Gatekeeper, LinkedIn's XLNT, Dropbox's Stormcrow and other commercial SaaS solutions.

Features

• Feature toggles, gradual release
• Multi-variant experiments, A/B testing
• Built-in editor with user friendly UI
• Hierarchical configurations
• Advanced targeting of users/groups
• Dependencies between features
• Customizable schema/domain model
• Auditing
• Integrated storage for saving context
• Container friendly
• Integrated OAuth support
• Pluggable storage backend
• Simple RESTful api for fetching configurations
• Scalable

Getting started

The easiest way to start evaluating Tweek is to run it locally on Docker. Make sure you have the latest docker (for windows/mac/etc..) version installed (17-06+).

Running Tweek

• Clone the repo (git clone https://github.com/Soluto/tweek.git).
• [optional] Pull images, run yarn docker-compose pull --parallel (optional for getting started fast with Tweek as it's skip build).
• [optional] Newer versions of docker-compose support parallel build, so you can use yarn docker-compose build --parallel for faster build.

Using Docker Compose

• Run (yarn start) - this might take a few minutes on the first time.

Using Tilt

Tilt is a CLI tool that can be used to create an optimal development environment for multi-container apps such as Tweek. It support automatic rebuilding of images and re-running of containers on files' changes. Additionally, it support more complex live reloading scenarios, like Tweek Editor (React app). Tweek uses Tilt on top of docker-compose for easier and (usually) faster developer experience (compared to Tilt with k8s).

• Install Tilt (https://docs.tilt.dev/install.html)
• tilt up

Using Kubernetes

• Install Skaffold (https://github.com/GoogleContainerTools/skaffold)
• Run skaffold dev --port-forward=false
• Wait for the environment to be stable (this will take about 10 mins on the first time due to building all images, afterward it can take about 2 mins for environment to stabilize)

Troubleshooting

• Run (yarn start --build) to rebuild all images and start Tweek.

Edit your first key

After setting up our environment, we're going to create our first key. Keys in Tweek are the most basic building blocks, and they represent a container for dynamic value that affect feature behaviors. Our first key will be a key that is responsible for the color of a "sign up" button.

• Open http://localhost:8081/login in browser.
• Login
  • User Basic auth (user: admin-app, password: 8v/iUG0vTH4BtVgkSn3Tng==)
  • Can also use OIDC mock server login button for testing OIDC (user: User, password: pwd)
• Go to keys page.
• Click on "Add Key"
• Type my_app/sign_button/color
• Set value type to String
• Add default value "red"
• Save changes
• Click on "Add Rule"
• Set Property to Country (user) and set "=" and "Canada" in the other fields
• In Rule value set the value "blue"
• Click "Save changes"

More on keys and paths

Querying Tweek

Use curl/postman/chrome to fire GET Request:

• http://localhost:8081/api/v2/values/my_app/sign_button/color -> expected to be "red"
• http://localhost:8081/api/v2/values/my_app/sign_button/color?user.Country=canada -> expected to be "blue"
• http://localhost:8081/api/v2/values/my_app/sign_button/_?user.Country=canada -> expected to be {"color":"blue"}

Using the rest api, an application can query Tweek for getting the right set of values for each specific user. More on Tweek Rest api.

Adding context data

Tweek provide UI and rest api for editing context.

• Go to context
• Set Identity Type to User
• Set User id to John
• Click enter
• Set value "Canada" for property Country

After that, we can query Tweek API with:

• http://localhost:8081/api/v2/values/my_app/sign_button/color?user=john -> expected to be "blue"

You can also use the api for updating Tweek context:

• curl -X POST http://localhost:8081/api/v2/context/user/john
  -H 'content-type: application/json'
  -H 'x-client-id: admin-app'
  -H 'x-client-secret: 8v/iUG0vTH4BtVgkSn3Tng=='
  -d '{ "country": "Canada" }'

More on Context.

Gradual feature release

Create a new key in the editor "my_app/sign_button/is_enabled" with value type "boolean" and default value False.
Add a new rule, remove all conditions, set the the rule value to gradual release with 50%. Try querying configuration with different users and you'll have different results.

• http://localhost:8081/api/v2/values/my_app/sign_button/is_enabled?user=barny
• http://localhost:8081/api/v2/values/my_app/sign_button/is_enabled?user=robin
• http://localhost:8081/api/v2/values/my_app/sign_button/is_enabled?user=ted
• http://localhost:8081/api/v2/values/my_app/sign_button/is_enabled?user=lily
• etc...

More on how multi-variant keys work in Tweek. (link)

Deployment to production

FAQ

• Who's using Tweek?
  Tweek is used in large-scale production deployment at Soluto.
• How do I generate ssh keys and a pfx file for use in production?
  There's a script for this purpose in utils/generate_keys.sh
• I found a security vulnerability, should I open an issue about it?
  No. Please send an email to [email protected].

Related projects

Tweek-Clients - Tweek REST clients
Tweek.JPad - Tweek's internal rules engine

Additional Resources

• https://martinfowler.com/articles/feature-toggles.html
• https://en.wikipedia.org/wiki/Feature_toggle

tweek's Issues

Add tests keys for smoke

suggested keys:

@tests
    simple
    nested
        key1
        key2
        key3
    multivarientkeys
        uniform
        bernoulli
        weighted
    matchers
        context
        not
        equal
        in
        comparison
        fixedcontextbasedkey
        keydependency
    rules
        multipleconditions
        multiplerules

Investigate backoffice production error

Backoffice fail to save rules

create tweek client for c#

maybe should have simple client side cache to allow using it in sync code

Only update rules if changes were made to the repo

Split mutator to different lib

Backoffice - allow to add new rules

Allow to add rules in ui

Basic UX for backoffice

fix integration tests

Support for partitions

Partitions are way to organize rules on key.
Instead of having a flat view of rules, rules are organized/grouped by a partition.
A partition can be defined by using one or more properties.

// link to example

migrate to node 6

backoffice CI speed

backoffice build time takes ~10 minutes due to npm i.
let's set a container with npm cache: https://github.com/tswicegood/docker-npm-cache
should speed it quite a lot.

JPad format

Write a specification for JPad format.
Write a schema for validation.

Support for custom compares in matcher

Add CI for project

Move all tests to xunit?

Currently, we use these frameworks for tests:
xunit
nunit 2.x
nunit 3.x

We also use FsCheck and FsUnit libraries for F# tests.
Both of them works properly with nunit 2.x and xunit, but not with nunit 3.x.
Since nunit 2.x will be obsolete and there are many breaking changes between 3.x and 2.x, it might makes sense to drop nunit in favor of xunit.

Context writing endpoint for configuration service

Expose api for writing user data to context.
Suggest format:

POST /context/{identityType}/{identityId}/{key}
value

or

POST /context/{identityType}/{identityId}  
{key1:value1, key2:value2, ...}

Need to consider security (not MVP)

Prove "FF" incremental roll out is working (or not...)

Tweek value distributor currently create different distribution for different users based on the CreationDate of the user vs the experiment start-time.

In that way, by adding a new distribution, old users will get the old distribution values (retaining persistence) while new ones will get the new distribution values.

It was assumed that simply updating the rule instead of adding a new distribution is the same as reshuffle, but it's actually not the same because of the algo. (unless we replace the ruleid as well)

FF uses bernoulliTrial, which means a two variant distribution of [true|false] based on percentage.
It seems that in this case, since rule id, identity id, total bucket size (100) does not change, the users that got true will get the same values on increasing the "true" distribution.

In the case, FF rollout will work properly - meaning old users won't get "reshuffled".
We need to write a simple unit test to prove it, it might be that we already implement FF rollout by accident :)

Reorganize folders

Current folder structure is a bit of mess.

Add unit tests for backoffice

Add safer mechanism for deleting rules

Should be deprecated once we implement proper diff view.

Add Git Configuration to web.config

Currently, it's reading it directly from our code.

Persist/save meta key data

• add meta client
• add new meta repository and mock implementation
• set new couchbase key space
• create couchbase meta repository implementation
• set server api (proxy in serverside to the couchbase)

both client and server should have mock/real meta provider

set ci cd process to tweek backoffice

using travis-ci and docker

Security for backoffice

• SSL
• AD login

refactor tweek-rules structre

Current:

tweek-rules:
rules/{*keypath}.jpad
meta/{*keypath}.json
tags.json

suggested:

tweek-authoring:
dist/keys/{*keypath}.jpad
meta/keys/{*keypath}.json
         tags/{tagname}.json
         identities/{identity}.json (identity+properties)
         types/{type}.json

Support expression in "values"

Allow getting values from "context", can be useful for migration of calculated values.

CI for synchronizing rules

Git Driver Implementation for rules

• Read data directly from git (MVP)
• Basic authoring capability for running tests.
• Setup git environment.

create tweek client for js

should have simple client side cache to allow using it in sync code

Backoffice - save changes

Currently changes are not persisted to rules store.

Add data migration process

• - Batch process against devicestate table?
• - Ongoing - tweek feeder (need to check quality)

meta data features - type specs

type Tag:String

type KeyMetaData{
    key:string;
    displayName:string;
    description?:string;
    type?:string;
    tags:string;
}

type SearchQuery{
    text:string;
    tags:string;
}

type MetaPropertyType{
    name:string;
    type:string | MetaDataType (inline);
    defaultValue?:string;
    description?:string;
}

type MetaDataType{
    name:string;
    baseType:string;
    allowedValues?:string[];
    compare?:string;
    validate?:string;   
}

type MetaRepository{
    //keys
    getKeyMetaData(key:string):Promise<KeyMetaData>;
    setKeyMetaData(key:string, meta:KeyMetaData):Promise;
    searchKeys(SearchQuery):Promise<KeyMetaData[]>;

    //props
    listProperties():Promise<PropertyMetaData[]>;
    setPropertyMetaData(property:string, meta:PropertyMetaData ):Promise;
    deletePropertyMetaData(property:string, meta:PropertyMetaData ):Promise;

    //tags
    listTags():Promise<Tag[]>;
    deleteTag(tag:string):Promise;

    //datatypes
    listMetaDataTypes():Promise<MetaDataType[]>;
    setMetaDataType(dataType:string, meta:PropertyMetaData ):Promise;
    deleteMetaDataType(dataType:string):Promise;
}

Add UI tests for backoffice

Integrate search capability for meta data.

• integrate search engine
• implement first iteration of UI

Add support for exist/not exist prop

Support for checking if key is not in context

//Key does not exists
{
     "MyProp":null
}
//Key exists
{
     "MyProp":{"$not": null}
}

better backoffice server logging

the server logs for tweek backoffice are currently in the docker service logs.
the docker logs are not user-friendly, slow, doesnt show past logs

Update Rules flow

Suggested flow:

• Tweek management service register hook on git repo to be notified on push
• Git hook on update send post to tweek management/admin service
• Tweek management service pulls all the data from
• Tweek management service does some validations/test based on admin settings
• Tweek management service bundle all the rules to single json
• Tweek management service send the rules to configuration service instance.
• Tweek Configuration service instance compile the rules
• If compilation works, configuration services sends 200 back to management
• Configuration service use db to propagate configuration changes to all instances.

Some notes
Tweek management service does not exist yet.
I think it will include in the future (Not all MVP):

• Rules editing front-end
• Rules validation constraints
• Configuration service settings (format/parser/etc...)
• Service Health check
• Other Management (authentication, etc...)
• Management service can use consul or zookeeper to sync and scale

basic layout: Set less infra files

need to add ownerType to jpad when creating Multivariant cases.

Handle custom comparers parsing error

Error on the "target" value breaks the service which make sense, since we can easily prevent it.
Error on the value from the context fail the request.
It might be better to fail only the specific rule and log error, and return the other rules data to the agents.

Make driver loading and settings more modular and less code based

Both in services and integration tests.

reduce save time

each tag adding, jpad or meta change, doing a pull and push, this operation causes a jpads build taking ~1-2 minutes.
as a mvp we will do 1 save for all changes (tags,meta,jpad) instead of save for each

Testing couchbase driver

Couchbase is a document and kv db based on memcached.
It got the following advantages:

• Free and open source
• Extremely fast read throughput (for key-value)
• Persistence
• High availability - with cluster management and data-center replication
• Decentralized - all nodes are identical, no SPOF.
• Flexible data model (document db)
• Indexing and Querying capabilities
• Search support
• Map reduce views
• Nice management tools
• Not as used as Cassandra/Mongodb/Redis, but still used by big players
• Consistency - "Document access in Couchbase is strongly consistent, query access is eventually consistent" sounds good
• Should be easy to deploy on azure/docker cloud
• And there's an Enterprise version - support/qa/etc...

You can check the admin panel on:
http://couchbase-07cc5a45.b5501720.svc.dockerapp.io:8091/
username: tweek
pass:...

While we continue to use our "drivers" abstraction to decouple our app from the db, it might be the best solution to meet all our needs easily.

Most benchmarks are lies, but it should have very fast read performance -
http://www.datastax.com/dev/blog/how-not-to-benchmark-cassandra-a-case-study
http://www.couchbase.com/sites/default/files/uploads/all/Benchmarking-Couchbase-Server-for-Interactive-Applications.pdf

Refactor "rules" terminology to jpad "cases"

Backoffice - deployment

Get address for BlobRulesDriver from the management

Authorization for reading configuration/writing context

JPad Editor

• Basic Viewer.
• Ability to edit rules.
• Commit and publish.

Editor Experience

• View
  • Matcher
    • Properties
      • Simple fields
      • Other keys
    • Operators
      • Binary
      • Conjuction
  • Values
    • SingleVariant
    • MultiVariant
      • Barnouli
      • Weighted
• Edit
  • Cases
    • Delete
    • Add
    • Reorder
  • Matcher
    • Add condition
    • Delete condition
    • Properties
      • Simple fields
      • Other keys
    • Operators
      • Binary
      • Conjuction
  • Values
    • Change types
    • SingleVariant
    • MultiVariant
      • Barnouli
      • Weighted
• Meta Integration

Convert JPad Project to fsharp

Current C# wrapper is really not need and make code more complex with ugly interop tricks.
EntryPoints are pretty much the same as C# api:

type JPadParser(settings:ParserSettings) :
       Parse: (source:string)->IRule

type JPadGenerator :
       AddSingleVariant: (...params)-> generator
       AddMultiVariant: (...params)-> generator
       Build() -> RuleDefinition //(format*string)

key meta data viewer and editor

• view display name and description
• view tags
• edit tags
• edit display name and description

include writing the reducer, add action types