Configuration powertool with metosin/malli.
This library provides out-of-the-box:
- Configuration sourcing from command line arguments, environment variables, or any other configuration management tool you want to integrate with;
- Very expressive means to describe the configuration you expect;
- An agnostic way to protect secrets in your config that turn them into opaque, non-printable values, while still being able to decode them and access the original value when need be;
- Seamless and powerful CLI generation based on your config schema, returns a map representing the parsed, decoded arguments and environment variables you are interested in.
The returned map can be used as a config fragment, that you can later merge with the config value provided by any other system. As such it intends to play nicely with configuration tools, so the actual configuration value of your program is a map that is a graceful merge of several overlapping config fragment:
- Default configuration value;
- Environment variables when the program starts up;
- Value from some configuration management system;
- Command line arguments.
The expected shape of your configuration being described as a malli schema so you can parse and decode strings as well as validating any constraints. It's quite powerful.
Break versioning is used, so no breaking changes will be introduced without
incrementing the major version. Some bug fixes may be introduced and I will keep
adding new features as I encounter new needs. As illustrated below, malli-cli
should already cover most of your use cases with simplicity – or open an issue.
utility_name [-a][-b][-c option_argument]
[-d|-e][-f[option_argument]][operand...]The utility in the example is named utility_name. It is followed by options,
option-arguments, and operands. The arguments that consist of
- characters and single letters or digits, such as a, are known as
"options" (or, historically, "flags"). Certain options are followed by an "
option-argument", as shown with [ -c option_argument ]. The arguments
following the last options and option-arguments are named
"operands".
Let's consider this config schema:
(require '[piotr-yuxuan.malli-cli :as malli-cli])
(require '[malli.core :as m])
(def Config
(m/schema
[:map {:closed true, :decode/args-transformer malli-cli/args-transformer}
[:show-config? {:optional true}
[boolean? {:description "Print actual configuration value and exit."
:arg-number 0}]]
[:help {:optional true}
[boolean? {:description "Display usage summary and exit."
:short-option "-h"
:arg-number 0}]]
[:upload-api [string? {:description "Address of target upload-api instance. If not set from the command line, lookup env var $CORP_UPLOAD_API."
:short-option "-a"
;; Cli option will be looked up, then env var, then default.
:env-var "CORP_UPLOAD_API"
:default "http://localhost:8080"}]]
[:log-level [:enum {:description "Non-idempotent -v increases log level, --log-level sets it."
;; Express how to decode a string into an enum value.
:decode/string keyword
:short-option "-v"
:short-option/arg-number 0
:short-option/update-fn malli-cli/non-idempotent-option
:default :error
;; Used in summary to pretty-print the default value to a string.
:default->str name}
:off :fatal :error :warn :info :debug :trace :all]]
[:proxy [:map
[:host string?]
;; malli will parse and return an integer.
[:port pos-int?]]]]))Here is the command summary produced by (malli-cli/summary Config)
for this config:
Short Long option Default Description
--show-config Print actual configuration value and exit.
-h --help Display usage summary and exit.
-a --upload-api "http://localhost:8080" Address of target upload-api instance. If not set from the command line, lookup env var $CORP_UPLOAD_API.
-v --log-level error Non-idempotent -v increases log level, --log-level sets it.
--proxy-host
--proxy-portLet's try to call this program (code details below). You may invoke your Clojure main function with:
lein run \
--help -vvv \
-a "https://localhost:4000"Let's suppose your configuration system provides this value:
{:proxy {:host "https://proxy.example.com"
:port 3128}}and the shell environment variable CORP_UPLOAD_API is set to
https://localhost:7000. Then the resulting configuration passed to your app
will be:
{:help true
:upload-api "https://localhost:4000"
:log-level :debug
:proxy {;; Nested config maps are supported
:host "http://proxy.example.com"
;; malli transform strings into appropriate types
:port 3128}Let's try another time with this command with same provided config and env vars:
lein run \
--log-level=off
--show-configThe program will exit after printing:
{:log-level :off,
:show-config? true,
:upload-api "http://localhost:7000"}From a technical point of view, it leverages malli coercion and decoding capabilities so that you may define the shape of your configuration and default value in one place, then derive a command-line interface from it.
(require '[piotr-yuxuan.malli-cli :as malli-cli])
(require '[malli.core :as m])
(require '[clojure.pprint])
(require '[piotr-yuxuan.malli-cli.utils :refer [deep-merge]])
(defn load-config
[args]
(deep-merge
;; Value retrieved from any configuration system you want
(:value (configure {:key service-name
:env (env)
:version (version)}))
;; Command-line arguments, env-vars, and default values.
(m/decode Config args malli-cli/cli-transformer)))
(defn -main
[& args]
(let [config (load-config args)]
(cond (not (m/validate Config config))
(do (log/error "Invalid configuration value"
(m/explain Config config))
(Thread/sleep 60000) ; Leave some time to retrieve the logs.
(System/exit 1))
(:show-config? config)
(do (clojure.pprint/pprint config)
(System/exit 0))
(:help config)
(do (println (simple-summary Config))
(System/exit 0))
:else
(app/start config))))See tests for minimal working code for each of these examples.
- Long option flag and value
--long-option VALUEmay give
{:long-option "VALUE"}
;; Example schema:
[:map {:decode/args-transformer malli-cli/args-transformer}
[:long-option string?]]- Grouped flag and value with
--long-option=VALUEmay give
{:long-option "VALUE"}
;; Example schema:
[:map {:decode/args-transformer malli-cli/args-transformer}
[:long-option string?]]- Short option names with
-s VALUEmay give
{:some-option "VALUE"}
;; Example schema:
[:map {:decode/args-transformer malli-cli/args-transformer}
[:some-option [string? {:short-option "-s"}]]]- Options that accept a variable number of arguments:
-a -b val0 --c val1 val2
{:a true
:b "val0"
:c ["val1" "val2"]}
;; Example schema:
[:map {:decode/args-transformer malli-cli/args-transformer}
[:a [boolean? {:arg-number 0}]]
[:b string?]
[:c [string? {:arg-number 2}]]]- Non-option arguments are supported directly amongst options, or after a
double-dash so
-a 1 ARG0 -b 2 -- ARG1 ARG2may be equivalent to:
{:a 1
:b 2
:piotr-yuxuan.malli-cli/arguments ["ARG0" "ARG1" "ARG2"]}
;; Example schema:
[:map {:decode/args-transformer malli-cli/args-transformer}
[:a [boolean? {:arg-number 0}]]
[:b string?]]- Grouped short flags like
-halare expanded like, for example:
{:help true
:all true
:list true}
;; Example schema:
[:map {:decode/args-transformer malli-cli/args-transformer}
[:help [boolean? {:short-option "-h" :arg-number 0}]]
[:all [boolean? {:short-option "-a" :arg-number 0}]]
[:list [boolean? {:short-option "-l" :arg-number 0}]]]- Non-idempotent options like
-vvvare supported and may be rendered as:
{:verbosity 3}
;; or, depending on what you want:
{:log-level :debug}
;; Example schemas:
[:map {:decode/args-transformer malli-cli/args-transformer}
[:log-level [:and
keyword?
[:enum {:short-option "-v"
:short-option/arg-number 0
:short-option/update-fn malli-cli/non-idempotent-option
:default :error}
:off :fatal :error :warn :info :debug :trace :all]]]]
[:map {:decode/args-transformer malli-cli/args-transformer}
[:verbosity [int? {:short-option "-v"
:short-option/arg-number 0
:short-option/update-fn (fn [options {:keys [in]} _cli-args]
(update-in options in (fnil inc 0)))
:default 0}]]]- You may use nested maps in your config schema so that
--proxy-host https://example.org/upload --proxy-port 3447is expanded as:
{:proxy {:host "https://example.org/upload"
:port 3447}}
;; Example schema:
[:map {:decode/args-transformer malli-cli/args-transformer}
[:proxy [:map
[:host string?]
[:port pos-int?]]]]- Namespaced keyword are allowed, albeit the command-line option name stays
simple
--upload-parallelism 32may give:
{:upload/parallelism 32}
;; Example schema:
[:map {:decode/args-transformer malli-cli/args-transformer}
[:upload/parallelism pos-int?]]- You can provide your own code to update the result map with some complex
behaviour, like for example
--name Piotr:
{:vanity-name ">> Piotr <<"
:original-name "Piotr"
:first-letter \P}
;; Example schema:
[:map {:decode/args-transformer malli-cli/args-transformer}
[:vanity-name [string? {:long-option "--name"
:update-fn (fn [options {:keys [in]} [username]]
(-> options
(assoc :vanity-name (format ">> %s <<" username))
(assoc :original-name username)
(assoc :first-letter (first username))))}]]
[:original-name string?]
[:first-letter char?]]- Build a simple summary string (see schema Config above):
-h --help nil
-a --upload-api "http://localhost:8080" Address of target upload-api instance.
-v --log-level :error
--proxy-host nil
--proxy-port nil- Error handling with unknown options:
;; Example schema:
[:map {:decode/args-transformer malli-cli/args-transformer}
[:my-option string?]]
;; Example input:
["--unknown-long-option" "--other-option" "VALUE" "-s"}
;; Exemple output:
#:piotr-yuxuan.malli-cli{:unknown-option-errors ({:arg "-s"} {:arg "--other-option"} {:arg "--unknown-long-option"}),
:known-options ("--my-option"),
:arguments ["VALUE"],
:cli-args ["--unknown-long-option" "--other-option" "VALUE" "-s"]}- Environment variable
USERset topiotr-yuxuanmay give:
{:user "piotr-yuxuan"}
;; Example schema:
[:map {:decode/args-transformer malli-cli/args-transformer}
[:user [string? {:env-var "USER"}]]]- Protect secret values and keep them out of logs when configuration is printed.
Secrets are turned into strings by default, but you may provide
custom
secret-fnandplaintext-fn.
(m/encode
[:map
[:ui/service-name string?]
[:database/username {:secret true} string?]
[:database/password {:secret true} string?]]
{:ui/service-name "Hello, world!"
:database/username "Username must stay out of logs."
:database/password "Password must stay out of logs."}
malli-cli/secret-transformer)
=> {:ui/service-name "Hello, world!",
:database/username "***",
:database/password "***"}Note that environment variables behave like default values with lower priority than command-line arguments. Env vars are resolved at decode time, not at schema compile time. This lack of purity is balanced by the environment being constant and set by the JVM at start-up time.
- GNU Program Argument Syntax Conventions
- POSIX.1-2017 Utility argument syntax
- https://github.com/l3nz/cli-matic is a similar, more established project.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent