Coder Social home page Coder Social logo

runtime-config / runtime-config Goto Github PK

View Code? Open in Web Editor NEW
9.0 1.0 0.0 184 KB

Server for storing service settings.

License: MIT License

Shell 0.20% Dockerfile 1.95% Makefile 0.33% Python 96.71% Mako 0.81%
feature-flags feature-toggles runtime-config runtime-settings python3 postgres

runtime-config's Introduction

license version coverage tests status

runtime-config

Server for storing service settings. Together with the client, it allows you to change application settings at runtime.

Table of contents:

Overview

This application is responsible for storing settings and providing an API for the client library. If you want to see the api of this service, follow the link after starting the service. Swagger will be available there.

It's very important for understanding who created / changed some settings when a team has the ability to change application settings in real time. Therefore, the database implements logging of all changes to the setting table. You can see them in the setting_history table. The logs contain information about users who made changes and when changes were made.

Usage

At the moment, WEB UI is not implemented, so now you need to edit the description of variables directly in the service database. You can do this in the settings table.

An example of a request to create a variable that will be used in some-service-name service:

INSERT INTO setting(name, value, value_type, service_name)
VALUES ('timeout', '1', 'int', 'some-service-name');

With the help of the client, this variable can be get like this:

source = ConfigServerSrc(host='http://127.0.0.1:8080', service_name='some-service-name')
config = await RuntimeConfig.create(init_settings={}, source=source)
print(config.get('timeout'))

If you want to change the value of a nested variable, then use the following query:

INSERT INTO setting(name, value, value_type, service_name)
VALUES ('db__timeout', '1', 'int', 'some-service-name');

With the help of the client, this variable can be get like this:

source = ConfigServerSrc(host='http://127.0.0.1:8080', service_name='some-service-name')
config = await RuntimeConfig.create(init_settings={'db': {'timeout': 10}}, source=source)
print(config.db['timeout'])

Available variable types:

  • str
  • int
  • bool
  • null
  • json

Deploy

docker-compose

  1. Create a .env file with the necessary environment variables. An example of an .env file can be found in .env.example.

  2. Copy docker-compose.yml.

  3. Start the service and its dependencies.

    docker compose up
    

Other

  1. Define all environment variables necessary for the service to work. You can find them in .env.example.

  2. Select a docker image with the latest version of the application. You can find docker images here.

  3. Deploy postgres.

  4. Apply all migrations to postgres. Run the following alembic upgrade head command inside the docker image.

  5. Run the selected docker image. When starting, you need to pass the serve argument, which means that the web application will be launched to process requests from client libraries.

Build docker image

An example command for building a docker image with an application of version 0.1.0

ver=0.1.0 make build-img

Development

Setting up a local development environment on macOS with Apple Silicone

  1. Install brew.

    /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
    
  2. Add to ~/.zshrc and reopen console.

    # brew
    eval "$(/opt/homebrew/bin/brew shellenv)"
    export HOMEBREW_NO_ANALYTICS=1
    
    # poetry
    export PATH="$HOME/.local/bin:$PATH"
    
    # pyenv
    export PYENV_ROOT="$HOME/.pyenv"
    export PATH="$PYENV_ROOT/bin:$PATH"
    eval "$(pyenv init --path)"
    
    # direnv
    eval "$(direnv hook zsh)"
    
    # Generates flags necessary for building from source C libraries
    export LDFLAGS=""
    export CPPFLAGS=""
    export PKG_CONFIG_PATH=""
    
    pkgs=(curl readline sqlite)
    for pkg in $pkgs; do
        pkg_dir="$HOMEBREW_PREFIX/opt/$pkg"
    
        lib_dir="$pkg_dir/lib"
    
        if [ -d "$lib_dir" ]; then
            export LDFLAGS="$LDFLAGS -L$lib_dir"
        fi
    
        include_dir="$pkg_dir/include"
        if [ -d "$include_dir" ]; then
            export CPPFLAGS="$CPPFLAGS -I$include_dir"
        fi
    
        pkg_config_dir="$lib_dir/pkgconfig"
        if [ -d "$pkg_config_dir" ]; then
            if [ "x$PKG_CONFIG_PATH" = "x" ]; then
                export PKG_CONFIG_PATH="$pkg_config_dir"
            else
                export PKG_CONFIG_PATH="PKG_CONFIG_PATH:$pkg_config_dir"
            fi
        fi
    done
    
  3. Install required software.

    brew install wget direnv postgresql openssl [email protected] xz
    curl https://pyenv.run | bash
    
    brew install --cask docker
    
    curl -sSL https://install.python-poetry.org | python3.9 -
    poetry config virtualenvs.create false
    
    pyenv install 3.10.8
    
  4. Disable background postgres service.

    /opt/homebrew/opt/postgresql/bin/postgres -D /opt/homebrew/var/postgres
    
  5. Create configuration files. After creating the .env file, set the variables in it to the values you need.

    cp ./env.example ./.env
    cp ./envrc.example ./.envrc
    
  6. Reopen console and run direnv allow. After executing this command, a virtual environment for python will be created.

  7. Install requirements.

    poetry install
    
  8. Run tests to make sure the environment has been set up correctly.

    make tests
    

Start service locally

  1. Start postgres

    docker compose -f docker-compose.dev.yml up db
    
  2. Run migrations

    make migrate
    
  3. Start the service

    python ./src/runtime_config/cli.py serve --reload
    

Start service in docker

make up

Work with migrations

The automatic generation of the migration is done by the following command (you need to be careful and check what was generated):

alembic revision --autogenerate -m <description>

Applying all migrations:

alembic upgrade head

Downgrade last migration:

alembic downgrade -1

runtime-config's People

Contributors

aleksey925 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

runtime-config's Issues

Settings are unreachable after deletion

I'm not sure if is that on purpose, or if it's an actual feature but settings are unreachable after deletion. See the example.

First, we create a setting.

curl -X 'POST' \
  'http://localhost:8080/setting/create' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "name": "name",
  "value": "a",
  "value_type": "str",
  "disable": false,
  "service_name": "default"
}'
-->
{
  "id": 3,
  "name": "name",
  "value": "a",
  "value_type": "str",
  "disable": false,
  "service_name": "default",
  "created_by_db_user": "admin",
  "updated_at": "2022-10-17T18:16:10.383388"
}

Second, we update it.

curl -X 'POST' \
  'http://localhost:8080/setting/edit' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "id": 3,
  "name": "name",
  "value": "b",
  "value_type": "str",
  "disable": false,
  "service_name": "default"
}'
-->
{
  "id": 3,
  "name": "name",
  "value": "b",
  "value_type": "str",
  "disable": false,
  "service_name": "default",
  "created_by_db_user": "admin",
  "updated_at": "2022-10-17T18:20:13.212667"
}

Then fetch the setting with the history.

curl -X 'GET' \
  'http://localhost:8080/setting/get/3?include_history=true' \
  -H 'accept: application/json'
-->
{
  "setting": {
    "id": 3,
    "name": "name",
    "value": "b",
    "value_type": "str",
    "disable": false,
    "service_name": "default",
    "created_by_db_user": "admin",
    "updated_at": "2022-10-17T18:22:05.588031"
  },
  "change_history": [
    {
      "id": 3,
      "name": "name",
      "value": "a",
      "value_type": "str",
      "disable": false,
      "service_name": "default",
      "created_by_db_user": "admin",
      "updated_at": "2022-10-17T18:16:10.383388",
      "is_deleted": false,                                                   <--- something interesting ๐Ÿ‘€ 
      "deleted_by_db_user": null
    }
  ]
}

Here we can see the presence of is_deleted field. So, my assumption is that if I'm going to delete this setting, it will apply soft deletion and this is going to be reflected in the history.

curl -X 'GET' \
  'http://localhost:8080/setting/delete/3' \
  -H 'accept: application/json'
--->
{
  "status": "success"
}

Now, let's get the setting once more.

curl -X 'GET' \
  'http://localhost:8080/setting/get/3?include_history=true' \
  -H 'accept: application/json'
-->
{
  "setting": null,
  "change_history": []
}

A-ha! The setting is null, indeed. But, the history is empty. Not just null like we have on a newly created setting, but empty ๐Ÿค”. I suppose there is some issue with the history after deletion.

Empty `change_history` is always presented

On /setting/get/{id}, I can see that change_history is always presented, no matter what is the value of include_history param. It's either [] or the actual values, that feels a bit weird. I might expect this field to be missing or to be null but not an empty array.

curl -X 'GET' \
  'http://localhost:8080/setting/get/1?include_history=false' \
  -H 'accept: application/json'

Expected response:

{
  "setting": {
    "id": 1,
    "name": "data",
    "value": "hello world2",
    "value_type": "str",
    "disable": false,
    "service_name": "default",
    "created_by_db_user": "admin",
    "updated_at": "2022-10-09T11:24:00.738198"
  }
}

Actual response:

{
  "setting": {
    "id": 1,
    "name": "data",
    "value": "hello world2",
    "value_type": "str",
    "disable": false,
    "service_name": "default",
    "created_by_db_user": "admin",
    "updated_at": "2022-10-09T11:24:00.738198"
  },
  "change_history": []
}

Field naming inconsistency

Hey-hey! ๐Ÿ‘‹

I noticed a smaller inconsistency in the field naming strategy for this type of object:

{
  "setting": {
    "id": 2,
    "name": "aboba",
    "value": "false",
    "value_type": "bool",
    "disabled": false, <-------------------------------- THIS
    "service_name": "default",
    "created_by_db_user": "admin",
    "updated_at": "2022-10-22T10:29:29.886533"
  },
  "change_history": [
    {
      "id": 6,
      "name": "aboba",
      "value": "true",
      "value_type": "bool",
      "disabled": false, <-------------------------------- THIS
      "service_name": "default",
      "created_by_db_user": "admin",
      "updated_at": "2022-10-15T09:00:40.752155",
      "is_deleted": false, <-------------------------------- THIS
      "deleted_by_db_user": null
    }
  ]
}

IMO, it should be either is_deleted & is_disabled, or deleted & disabled. I'd suggest following the second option because it's already verbose enough and doesn't conflict with Java's is... getters ๐Ÿ˜‰.

Add HTTP/2 support

In the current implementation, the application returns this as a body of the response if the request was performed through HTTP/2:

No supported WebSocket library detected. Please use 'pip install uvicorn[standard]', or install 'websockets' or 'wsproto' manually.

There is no issue if I specify HTTP/1.1 as a preferred version for my HTTP client.

As I understood, the issue with uvicorn that doesn't support it at all. That is not the case for hypercorn, for instance. Anyways, I'm not a Python developer, so not really familiar with this.

Fix typo in `setting` model

I noticed a smaller typo in setting model. See the example:

{
  "setting": {
    "id": 0,
    "name": "string",
    "value": "string",
    "value_type": "str",
    "disable": true, <-------------------------- THIS
    "service_name": "string",
    "created_by_db_user": "string",
    "updated_at": "2022-10-15T09:05:00.279Z"
  },
  "change_history": [
    {
      "id": 0,
      "name": "string",
      "value": "string",
      "value_type": "str",
      "disable": true, <-------------------------- THIS
      "service_name": "string",
      "created_by_db_user": "string",
      "updated_at": "2022-10-15T09:05:00.279Z",
      "is_deleted": true,
      "deleted_by_db_user": "string"
    }
  ]
}

I guess, it's supposed to be disabled rather than disable?

Thank you in advance! ๐Ÿ‘

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.