etc gathers configuration values from multiple sources (cli options, OS
environment variables, files) using a declarative spec file that defines where
these values are to be found and located in a configuration map.
- Raison d'etre
- Defining a spec file
- Reading a spec file
- Gathering configuration values explicitly
- Accessing configuration values
- Printing your configuration values
- Report Misspellings on Environment Variables
- Cabal Flags
- Full Example
etc is a configuration management that:
-
Allows to have a versioned spec of all values your application can accept
-
Provides documentation about Environment Variables used by your application
-
Provides an API for gathering values from multiple sources (files, overwrite files, cli arguments, environment variables) and then composing them into a single configuration map
-
Gives a sane precedence over sources of the configuration value sources
-
Provides inspection utilities to understand why the configuration is the way it is
-
Provides an API that abstracts away the source of configuration values and allows easy casting into record types your application or other libraries understand
You need to use a spec file to define the structure of your application's configuration map; also if an entry value on this configuration map can have multiple input sources (environment variable, configuration files, command line option, etc), you can specify right there what this sources may be. The map can be defined using JSON or YAML; following an example in YAML format:
###
# These paths are going to be read for configuration values merging values
# from all of them, if they have the same keys, entries on (2) will have
# precedence over entries on (1)
etc/filepaths:
- ./resources/config.json # 1
- /etc/my-app/config.json # 2
###
# The program is going to have a Command Line interface
etc/cli:
desc: "Description of the program that reads this configuration spec"
header: "my-app - A program that has declarative configuration input"
# The program is going to have 2 sub-commands
commands:
config:
desc: "Prints configuration summary"
header: ""
run:
desc: "Executes main program"
header: ""
###
# With etc/entries we define the configuration map structure your
# application is going to be reading values from
etc/entries:
credentials:
username:
# Define the spec for ["credentials", "username"]
etc/spec:
type: string
# default value (least precedence)
default: "root"
# if environment variable is defined, put its value in this entry
env: "MY_APP_USERNAME"
# cli input is going to have one option for this value
cli:
input: option
metavar: USERNAME
help: Username of the system
required: false
# option is going to be available only on run sub-command
commands:
- run
# Define the spec for ["credentials", "password"]
password:
etc/spec:
type: string
env: "MY_APP_PASSWORD"
cli:
input: option
metavar: PASSWORD
help: "Password of user"
required: true
commands:
- runThe important keys to notice on the previous example:
-
etc/filepathstells where to look for files to gather the configuration of your app, it could be more than one file because you may want to have a default file for development, and then override it with some configurations for production/integration, The further the filepath is the higher precedence its values are going to have. -
etc/entriesspecifies how your configuration map is going to look like and how your business logic will be accessing it -
etc/specprovide means to define metadata for a configuration value entry, what is its default value, if it can be found via an environment variable, or if it may be specified as an CLI option/argument input.
To read a spec file you need to use the System.Etc.readConfigSpec function, this
function can accept either a JSON or YAML filepath. You can also use the
System.Etc.parseConfigSpec if you already gather the contents of a spec file from a
different source.
NOTE: When using System.Etc.parseConfigSpec or System.Etc.readConfigSpec
and the CLI cabal feature flag is true, unless you use the
System.Etc.resolveCommandCli function, you will have to explicitly declare the
ConfigSpec type parameter.
In order to allow etc to read from YAML files, you will need to use the yaml
cabal flag when installing the library, here are some instructions on how to
pass cabal flags
using
stack and
cabal. We do this so that in case
you want to stick with the JSON format, you don't have to pull dependencies you
don't need.
Even though a spec file defines where the configuration values can be found,
etc won't collect those values unless it is explicitly told to do so. To do
this you must use functions that will resolve these configuration sources.
When defining the spec, you can specify default values on the etc/spec
metadata entry. To get this values from the spec you must call the
System.Etc.resolveDefault with the result from System.Etc.readConfigSpec as an argument.
import qualified System.Etc as Etc
getConfiguration :: IO Etc.Config
getConfiguration = do
spec <- Etc.readConfigSpec "./path/to/spec.yaml"
return (Etc.resolveDefault spec)To get values from configuration files on your filesystem, you must specify an
etc/filepaths entry on the spec file, this will tell etc to merge a list of
configuration values from each path, the latter the filepath, the more
precedence it has on the configuration map.
After this entry is defined in your spec, you must then call the
System.Etc.resolveFiles function with the result of System.Etc.readConfigSpec as a
parameter.
This helps to have a scheme of over-writable configurations on deployed
applications, you can have the first path in the list of etc/filepaths entry
be the config used while developing your app, and once deployed you can have
production configuration values on a well known path (say
/etc/my-app/config.yaml).
import Data.Monoid (mappend)
import qualified System.Etc as Etc
getConfiguration :: IO Etc.Config
getConfiguration = do
spec <- Etc.readConfigSpec "./path/to/spec.yaml"
let
defaultConfig =
Etc.resolveDefault spec
(fileConfig, _fileWarnings) <- Etc.resolveFiles spec
return (fileConfig `mappend` defaultConfig)When an env key is specified in the etc/spec metadata of a configuration
value entry, etc will consider an environment variable with the given name.
After this entry is defined in your spec, you must then call the
System.Etc.resolveEnv function with the result of System.Etc.readConfigSpec as a
parameter.
import Data.Monoid (mappend)
import qualified System.Etc as Etc
getConfiguration :: IO Etc.Config
getConfiguration = do
spec <- Etc.readConfigSpec "./path/to/spec.yaml"
let
defaultConfig =
Etc.resolveDefault spec
(fileConfig, _fileWarnings) <- Etc.resolveFiles spec
envConfig <- Etc.resolveEnv spec
return (fileConfig `mappend` envConfig `mappend` defaultConfig)You can setup a CLI input for your program by using the etc/cli entry at the
root of the spec file, and the cli entry on the etc/spec metadata entries
for configuration values.
When a cli key is specified in the etc/spec metadata of a configuration
value entry, etc will consider inputs from a command line interface for your
application.
The opt/cli entry map must have the following keys:
-
desc: A one line description of what your application does -
header: The header used when getting the information from the auto-generated--helpoption -
commands: A map of sub-commands that this program can have; each entry is the name of the sub-command, and the value is a map with the keydescwith the same purpose as the top-leveldescentry defined above.NOTE: you must use
System.Etc.resolveCommandClifor thecommandsentry to take effect
The cli entry map can have the following keys (input is required):
-
required: specifies if the entry is required on the CLI -
input: how you want to receive the input value, it can either beargumentoroption -
metavar: the name of the input argument on the example/documentation string of the CLI help -
long(only available onoptioninputs): the name of the option in long form (e.g.--name) -
short(only available onoptioninputs): the name of the option in short form (.e.g-n) -
commands: A list of sub-commands that are going to have this option/argument available; make sure the commands listed here are also listed in theetc/clientry of your spec file.
When the commands key is not specified on the etc/cli entry of the spec
file, you must use this resolver.
After the cli entry is defined in your spec, you must then call the
System.Etc.resolvePlainCli function with the result of System.Etc.readConfigSpec as a
parameter.
import Data.Monoid (mappend)
import qualified System.Etc as Etc
getConfiguration :: IO Etc.Config
getConfiguration = do
spec <- Etc.readConfigSpec "./path/to/spec.yaml"
let
defaultConfig =
Etc.resolveDefault spec
(fileConfig, _fileWarnings) <- Etc.resolveFiles spec
envConfig <- Etc.resolveEnv spec
cliConfig <- Etc.resolvePlainCli spec
return (fileConfig
`mappend` cliConfig
`mappend` envConfig
`mappend` defaultConfig)When the commands key is specified on the etc/cli entry of the spec file, you must
use this resolver.
After the cli entry is defined in your spec, you must then call the
System.Etc.resolveCommandCli function with the result of System.Etc.readConfigSpec as a
parameter.
This will return a tuple with the chosen sub-command and the configuration map;
the command Haskell type needs to be an instance of the Aeson.FromJSON,
Aeson.ToJSON and Data.Hashable.Hashable typeclasses for the command to be
parsed/serialized effectively.
import GHC.Generics (Generic)
import Data.Hashable (Hashable)
import qualified Data.Aeson as JSON
import qualified Data.Aeson.Types as JSON (typeMismatch)
import Data.Monoid (mappend)
import qualified System.Etc as Etc
data Cmd
= Config
| Run
deriving (Show, Eq, Generic)
instance Hashable Cmd
instance JSON.FromJSON Cmd where
parseJSON json =
case json of
JSON.String cmdName ->
if cmdName == "config" then
return Config
else if cmdName == "run" then
return Run
else
JSON.typeMismatch ("Cmd (" `mappend` Text.unpack cmdName `mappend` ")") json
_ ->
JSON.typeMismatch "Cmd" json
instance JSON.ToJSON Cmd where
toJSON cmd =
case cmd of
Config ->
JSON.String "config"
Run ->
JSON.String "run"
getConfiguration :: IO (Cmd, Etc.Config)
getConfiguration = do
spec <- Etc.readConfigSpec "./path/to/spec.yaml"
let
defaultConfig =
Etc.resolveDefault spec
envConfig <- Etc.resolveEnv spec
(fileConfig, _fileWarnings) <- Etc.resolveFiles spec
(cmd, cliConfig) <- Etc.resolveCommandCli spec
return ( cmd
, fileConfig
`mappend` cliConfig
`mappend` envConfig
`mappend` defaultConfig)In order to allow etc to generate CLI inputs for your program, you will need
to use the cli cabal flag when installing the library, here are some
instructions on how to pass cabal flags
using
stack and
cabal. We do this so that in case
you are not interested in generating a CLI input for your program, you don't
have to pull dependencies you don't need.
Sometimes, you would like to use the concept of CLI or environment variables, without
actually calling the OS APIs, etc provides pure versions for these resolvers:
-
System.Etc.resolveEnvPure -
System.Etc.resolvePlainCliPure -
System.Etc.resolveCommandCliPure
This work exactly the same as their non-pure counterparts, but receive one extra argument to fetch the required input.
Internally, etc stores every value that it gathers from all sources like a
JSON object (using the Data.Aeson.Value type), this provides a lot of
flexibility around what value you can get from your configuration map, allowing
your to use Aeson typeclasses to cast configuration values to more business
logic data structures.
There are two functions that can be used to get values out from a configuration map:
System.Etc.getConfigValue
Reads values specified on a spec file and casts it to a Haskell type
using the Aeson.FromJSON typeclass
System.Etc.getConfigValueWith
Reads values specified on a spec file and casts it using a custom function that
uses the Aeson parser API; this works great when the data structures of
libraries you use don't support Aeson or the format in your config file is not
quite the same as the already implemented Aeson.FromJSON parser of a type
given by a library.
An example of their usage is given in the full example section
A lot of times you may want to assert where a configuration value is coming
from, or if a particular environment variable was considered effectively by your
program. You an use the System.Etc.printPrettyConfig function to render the
configuration map and the different values/sources that were resolved when
calculating it. This function is really useful for debugging purposes.
Here is the output of one of the example applications:
$ MY_APP_USERNAME=foo etc-command-example run -u bar -p 123
Executing main program
credentials.username
bar [ Cli ]
foo [ Env: MY_APP_USERNAME ]
root [ Default ]
credentials.password
123 [ Cli ]The output displays all the configuration values and their sources, the first
value on the list is the value that System.Etc.getConfigValue returns for that
particular key.
When you define env keys on the etc/entries map of your spec file, we can
infer what are the valid Environment Variables that need to be defined for your
application, knowing this, etc can infer when there is a typo on one of this
environment variables and report this. You need to have the extra cabal flag and
call the System.Etc.reportEnvMisspellingWarnings with the configuration spec as
as an argument.
Here is an example of the output this function prints to stderr when the given
Environment Variables are almost identical to the ones found on the spec file:
$ MY_AP_USERNAME=foo etc-command-example run -u bar -p 123
WARNING: Environment variable `MY_AP_USERNAME' found, perhaps you meant `MY_APP_USERNAME'
To reduce the amount of dependencies this library brings, you can choose the exact bits of functionality you need for your application.
-
yaml: Allows (in addition of JSON) to have spec file and configuration files in YAML format -
cli: Provides the CLI functionality explained in this README -
extra: Provides helper functions for inspecting the resolved configuration as well as providing warning messages for misspelled environment variables
NOTE: This example uses the spec file stated above
import Control.Applicative ((<$>), (<*>))
import Data.Aeson ((.:))
import Data.Hashable (Hashable)
import Data.Monoid (mappend)
import GHC.Generics (Generic)
import qualified Data.Aeson as JSON
import qualified Data.Aeson.Types as JSON (typeMismatch)
import qualified System.Etc as Etc
data Credentials
= Credentials { username :: Text
, password :: Text }
deriving (Show)
data Cmd
= Config
| Run
deriving (Show, Eq, Generic)
instance Hashable Cmd
instance JSON.FromJSON Cmd where
parseJSON json =
case json of
JSON.String cmdName ->
if cmdName == "config" then
return Config
else if cmdName == "run" then
return Run
else
JSON.typeMismatch ("Cmd (" `mappend` Text.unpack cmdName `mappend` ")") json
_ ->
JSON.typeMismatch "Cmd" json
instance JSON.ToJSON Cmd where
toJSON cmd =
case cmd of
Config ->
JSON.String "config"
Run ->
JSON.String "run"
parseCredentials json =
case json of
JSON.Object object ->
Credentials
<$> object .: "user"
<*> object .: "password"
getConfiguration :: IO (Cmd, Etc.Config)
getConfiguration = do
spec <- Etc.readConfigSpec "./path/to/spec.yaml"
Etc.reportEnvMisspellingWarnings spec
let
defaultConfig =
Etc.resolveDefault spec
(fileConfig, _fileWarnings) <- Etc.resolveFiles spec
envConfig <- Etc.resolveEnv spec
(cmd, cliConfig) <- Etc.resolveCommandCli spec
return ( cmd
, fileConfig
`mappend` cliConfig
`mappend` envConfig
`mappend` defaultConfig )
main :: IO ()
main = do
(cmd, config) <- getConfiguration
case cmd of
Config -> do
Etc.printPrettyConfig config
Run -> do
-- Get individual entries (Uses instance of Text type for the Aeson.FromJSON
-- typeclass)
username <- Etc.getConfigValue ["credentials", "username"]
-- Get the values with a supplied JSON parser
creds <- Etc.getConfigValueWith parseCredentials ["credentials"]
print (username :: Text)
print creds
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent