npryce / konfig

Simple config properties API for Kotlin

Konfig - A Type Safe Configuration API for Kotlin

Konfig provides an extensible, type-safe API for configuration properties gathered from multiple sources — built in resources, system properties, property files, environment variables, command-line arguments, etc.

A secondary goal of Konfig is to make configuration "self explanatory”.

Misconfiguration errors are reported with the location and “true name” of the badly configured property. E.g. a program may look up a key defined as Key("http.port", intType). At runtime, it will be parsed from an environment variable named HTTP_PORT. So the error message reports the name of the environment variable, so that the user can easily find and fix the error.

Configuration can be inspected and listed. For example, it can be exposed by HTTP to a network management system to help site reliability engineers understand the current configuration of a running application.

Getting Started

To get started, add com.natpryce:konfig:<version> as a dependency, import com.natpryce.konfig.* and then:

1. Define typed property keys

   val server_port = Key("server.port", intType)
   val server_host = Key("server.host", stringType)

2. Build a Configuration object that loads properties:

   val config = systemProperties() overriding
                EnvironmentVariables() overriding
                ConfigurationProperties.fromFile(File("/etc/myservice.properties")) overriding
                ConfigurationProperties.fromResource("defaults.properties")

3. Define some properties. For example, in defaults.properties:

   server.port=8080
   server.host=0.0.0.0

4. Look up properties by key. They are returned as typed values, not strings, and so can be used directly:

   val server = Server(config[server_port], config[server_host])
   server.start()

Konfig can load properties from:

• Java property files and resources
• Java system properties
• Environment variables
• Hard-coded maps (with convenient syntax)
• Command-line parameters (with long and short option syntax)

Konfig can easily be extended with new property types and sources of configuration data.

Konfig can report where configuration properties are searched for and where they were found.

Naming of Properties

Konfig's Configuration objects expect property names to follow Java property name conventions: dots to represent hierarchy, lower-case identifiers within the hierarchy, hyphens to separate words in those identifiers.

For example: servers.file-storage.s3-api-key, servers.file-storage.s3-bucket.

Each Configuration implementation maps from that naming convention to the convention used by the underlying configuration store. E.g. the EnvironmentVariables implementation maps Java property name convention to the upper-case-and-underscores convention used for Unix environment variables.

Configuration is an interface and Key is a data class. This makes it straight forward to write an implementation of Configuration that translates the names of keys to different naming conventions, if your configuration follows an unusual convention.

Reflectomagic key definition

Konfig has a few ways to reduce boilerplate code when defining configuration keys.

1. You can use Kotlin's delgated property protocol to name keys after the constants that hold them:

   val host by stringType // defines a key named "host"
   val port by intType    // defines a key named "port"
   
   ...
   
   val client = TcpClient(configuration[host], configuration[port])
   

2. You can declare objects that extend PropertyGroup to define hierarchies of property keys that follow the Konfig naming conventions described above:

   object server : PropertyGroup() {
       val base_uri by uriType   // defines a key named "server.base-uri"
       val api_key by stringType // defines a key named "server.api-key"
   }
   
   ...
   
   val client = HttpClient(configuration[server.base_uri], configuration[server.api_key])
   

konfig's Issues

Multi-valued properties

Nice library !

I am trying to use it for a project and I need to have a list of properties of a specific type, eg:

class ServiceConfig : Configuration by systemProperties() overriding
        EnvironmentVariables() overriding fromResource("service.properties") {

    open class Node: PropertyGroup() {
        val name by stringType
        val url by uriType
    }

    object nodes : PropertyGroup() {
        object node1 : Node()
        object node2 : Node()
        // ... (eventually this will keep on growing)
    }
}

Is there some way to do this or is it planned for a future implementation?

Thanks

java.lang.IllegalArgumentException when accessing configuration properties

Hi,

the stacktrace is:

Exception in thread "main" java.lang.IllegalArgumentException: Parameter specified as non-null is null: method kotlin.jvm.JvmClassMappingKt.getKotlinClass, parameter $receiver
    at kotlin.jvm.JvmClassMappingKt.getKotlinClass(JvmClassMapping.kt)
    at com.natpryce.konfig.PropertyGroup.outer(magic.kt:16)
    at com.natpryce.konfig.PropertyGroup.namePrefix(magic.kt:18)
    at com.natpryce.konfig.PropertyGroup.name(magic.kt:17)
    at com.natpryce.konfig.PropertyGroup.key(magic.kt:23)
    at com.natpryce.konfig.MagicKt.getValue(magic.kt:28)

I took a look at the code and there is a silent requirement for the configuration class to be inside another class:

open class PropertyGroup(private val outer: PropertyGroup? = null) : PropertyKeys() {
    private fun outer() = outer ?: javaClass.enclosingClass.kotlin.objectInstance as? PropertyGroup

If the configuration object is a top level class in a separate file, javaClass.enclosingClass is null. This is the case in all the tests, btw, that the config object is inside an other class (eg.: https://github.com/npryce/konfig/blob/master/src/test/kotlin/com/natpryce/konfig/magic_tests.kt#L19) However it's not obvious from the README.md that it is a requirement.

I'm wondering how to fix this - I guess it should not be necessary to require a top level class, however I might be missing something.

Either the documentation should be updated or the code :)

Wrong version tag for 1.6.2.1

The version 1.6.2.1 was published with this version tag: "version=1.6.2.1" instead of "1.6.2.1".
For example to add it to maven file it is required to use:

<dependency>
    <groupId>com.natpryce</groupId>
    <artifactId>konfig</artifactId>
    <version>version=1.6.2.1</version>
</dependency>

instead of

<dependency>
    <groupId>com.natpryce</groupId>
    <artifactId>konfig</artifactId>
    <version>1.6.2.1</version>
</dependency>

Optional values not supported?

Optionals (for args, environment values etc) or defaults don't seem to be supported. Am I right?

I suppose you can get around it by using try blocks but it quickly gets ugly.

How to write new values to config file?

Is there any way to save the changed values to the same file from which the values were loaded, so when the program is run a second time then the new values are loaded?

Invalid version uploaded to MavenCentral

It seems that version=1.6.2.1 was uploaded by mistake. See http://mvnrepository.com/artifact/com.natpryce/konfig

Intrinsic Configuration objects should include source file and line in their location

This would help one find the code that sets up the configuration.

seems to have issue with _

For some reason when I specify a name with _ in it, it converts it to -

object controlScript : PropertyGroup() {
val APPDYNAMICS_AGENT_UNIQUE_HOST_ID by stringType
}

println(config[controlScript.APPDYNAMICS_AGENT_UNIQUE_HOST_ID] )

Exception in thread "main" com.natpryce.konfig.Misconfiguration: controlScript.APPDYNAMICS-AGENT-UNIQUE-HOST-ID property not found; searched:

• controlScript.APPDYNAMICS-AGENT-UNIQUE-HOST-ID in /Users/laipt/Desktop/petels/POC.cicd/cicd/src/main/resources/application-dev.properties

Build project

I have difficulties to build the project.

• gradle is not the latest
• it looks like it requires JDK 8 to build (which I do not want to install)
• Kotlin version is very old

I ported the project to Maven in order to build it. Would you accept migration to Maven ?

LICENSE?

This project doesn't have a LICENSE.txt
It would be awesome to use your project in my code but I can't if its something like GPL.

add `default` field into `Key`

Please add default field into Key:

    val nameKey = Key("name", stringType, "DefaultName")
    val name = config[nameKey] /// if not contains in CONFIG -> ="DefaultName"

If use -h command it will print:

Description of NAME. Default value: DefaultName

And if not setted in configuration - use Deafult Value

instead I do now:

    val connectiontimeoutKey = Key("connection.timeout", intType)
    fun getConnectionTimeout(): Int {
        return if (connectiontimeoutKey in vals) vals[connectiontimeoutKey]
        else DEFAULT_CONNECTION_TIMEOUT
    }

but I need:

    val connectiontimeoutKey = Key("connection.timeout", intType, DEFAULT_CONNECTION_TIMEOUT)
    fun getConnectionTimeout(): Int {
        return vals[connectiontimeoutKey]
    }

Kotlin libraries included as compile dependencies instead of provided

get list of all available keys

Is it possible to be able to get a list of every possible key in a configuration? At the moment it's all gets/contains etc. For background, what we want to do is basically run a test that ensures the keys are consistent/available across all envs, as otherwise we don't know till we start the app if the config files have all the params (we had the issue recently).

Thanks :)

Hoplite

I belive https://github.com/sksamuel/hoplite gives you much of the same functionality

Deploy to jCenter

Hello,

Can you build and deploy this library to jCenter?
Thank you,
Alex

Include `fromInputStream` method for supporting android assets.

Include fromInputStream method for supporting android assets or raw resources.

Is the projected maintained ?

• no commits more than a year
• no issues solved
• pull requests stay open without comments

Is the project alive ?

Load config from optional resource

It would be nice to have a function fromOptionalResource analogous to fromOptionalFile.

Currently I'm doing something like:

private fun ConfigurationProperties.Companion.fromOptionalResource(resourceName: String) =
  if (ClassLoader.getSystemResource(resourceName) != null) fromResource(resourceName)
  else EmptyConfiguration

private fun konfig(resourceName: String): Configuration = systemProperties() overriding
  EnvironmentVariables overriding
  ConfigurationProperties.fromOptionalResource(resourceName)

Document how to use CLI for configuration

I can't seem to find how to use the CLI for configuration, despite the fact the README says it's an available source.

Make the API easier to use from Java

Required keys

Is there any way to make Key "required"?

I want to have some of my cli arguments required and other - optional.

Load from resource from within a servlet

I'm trying to load resources from the classpath in a servlet application. I would expect to be able to use

ConfigurationProperties.fromResource("application.properties")

or maybe

ConfigurationProperties.fromResource(SomeClass::class.java, "application.properties")

But both don't work. The first uses the system classloader, instead of the servlet classloader (which makes sense), but the second variant should work. In the code I see that the class that is passed in is used like this:

relativeToClass.getResource(resourceName)

For some reason this doesn't work. What does work is

relativeToClass.classLoader.getResource(resourceName)

Since I cannot passin the URL returned from this call, I have to resort to

val res = SomeClass::class.java.classLoader.getResource("application.properties")
ConfigurationProperties.fromFile(File(res.file))

Which isn't very elegant. Am I missing something here?

Document how Konfig names configuration properties and translates names,

Konfig uses "-" to separate words within property levels, and "." to separate property levels.

It translates names in the code to that naming convention. And also translates that naming convention to the naming conventions used by different configuration sources (e.g. it translates to upper-case names with underscores to look up environment variables).

fr: ultra-simplified version using delegates

I think that this would be an easy feature given the great foundation in this project, and what Kotlin can do with delegates.

In my class, I'd like to have magic delegated properties that take into account everything (SystemProperties, EnvironmentVariables, command line, and default values). I think it would be possible to do so in a way that a single import is all you need:

import com.natpryce.konfig.byConfig

class MyClass {
  val speed:Double = byConfig(0.0)
  val name:String = byConfig("Larry")
  init {
    println("My name is $name because of ${::name.source}")
  }
}

How do I read config props from resource/file only if it exists and otherwise ignore it?

val config = systemProperties() overriding
        EnvironmentVariables() overriding
        ConfigurationProperties.fromResource("local.properties")

Currently this fails when local.properties doesn't exist. Is there a way to let it ignore that file, if it's not there?

Consider use of `typeOf`

Kotlin 1.3.40 added typeOf to the standard library, which may simplify code. However, it is marked as "experimental". Reading the tea leaves, Kotlin plans to make this feature standard (non-experimental) at some point, so I presume there are some multi-platform issues for the present. My focus is JVM, so I'm satisfied with how typeOf works.

https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.reflect/type-of.html

konfig might be simplified/improved with use of typeOf.

add `default` field into `CommandLineOption`

Please add default field into CommandLineOption:

`
val cmdName = CommandLineOption(Key("name", stringType), "name", "name", "DefaultName", "Description of NAME")

`

And if use -h command it will print:

Description of NAME. Default value: DefaultName

And if not setted in configuration - use Deafult Value

Add option to exclude underscore replacement

Is it possible to add functionality to make this magic.kt test pass? I have some properties with key name '_' underscore.

From:

object a_group : PropertyGroup() {
val a_property by stringType
}

@test
fun underscore_not_replaced_with_hyphen() {
assertThat(a_group.a_property.name, equalTo("a_group.a_property"))
}

magic.kt:23

Create release with License

Users need a release with a license to be able to use this code in a real world project.

Will you please cut a release with the license file?

`Configuration` should have a method to report the location in which a value is found for a Key

This will help debugging, diagnostics etc.

Something like...

fun Configuration.locationOf(key: Key<*>): Location

Support different file formats for config

Thank you for your hard work!

I have a question about supported config file format. Will it be possible to use format other than java properties format (yml etc)?

[Suggestion] support of dynamic configuration

Currently Konfig only support static configuration. For example myproperty.value can be directly named and referenced. But say I want to generate some more dynamic elements such as

server.port=8080
server.host=0.0.0.0
server.modules=a, b
server.modules.a.initMessage=A Init
server.modules.a.stopMessage=A Stop
server.modules.b.initMessage=B Init
server.modules.b.stopMessage=B Stop

To begin with, let's assume we have a data class to support a "Module":

data class Module(val initMessage: String, val stopMessage: String)

We also have a property group:

object server : PropertyGroup() {
    val modules by listType(stringType)
}

val moduleIds : List<String> = config[server.modules]

Right now we can extract a list of strings, containing (a, b), it would be interesting to actually obtain the list of modules we're after or something we can work with.

Because the modules a and b are defined dynamically, we can't pre-bake them in our PropertyGroup

---

Adding a special helper, we can do a bit more:

typealias Extractor<V> = (String, String, Configuration) -> V
fun <V> Configuration.get(keys: Key<List<String>>, extractor: Extractor<V>): Map<String, V> {
    return this[keys]
            .map { it to extractor(keys.name, it, this) }
            .toMap()
}

This allows us to change our code a bit:

val moduleIds : List<String> = config[server.modules]
val mapModuleIdToInitMessage : Map<String, String> = config.get(server.modules){ prefix, id, config ->
  config[Key("$prefix.$id.initMessage", stringType)]
}

We now have a very basic map ("a" to "A Init", "b" to "B Init")

We could do the same thing to obtain the actual module object:

val moduleIds : List<String> = config[server.modules]
val mapModules : Map<String, Module> = config.get(server.modules){ prefix, id, config ->
  Module(config[Key("$prefix.$id.initMessage", stringType)], config[Key("$prefix.$id.stopMessage", stringType)])
}

we now have a map ("a" to ("A Init", "A Stop"), "b" to ("B Init", "B Stop")). We can extract the values if we so desire mapModules.values, etc.

---

While this works, I'm not very fond of having the "stopMessage" and "initMessage" extraction being done without involving the PropertyGroup, to improve that we could add a few helpers in the PropertyGroup:

object server : PropertyGroup() {
    val modules by listType(stringType)

    fun modulesInitMessage(id: String) = key("${server::modules.name}.$id.initMessage", stringType)
    fun modulesStopMessage(id: String) = key("${server::modules.name}.$id.stopMessage", stringType)
}

Now we can use moduleInitMessage("a") to obtain dynamically a key for the initMessage of module a.

Which means that we can slightly alter our call:

val mapModules : Map<String, Module> = config.get(server.modules){ _, id, config ->
  Module(config[server.modulesInitMessage(id)], config[server.modulesStopMessage(id)])
}

This feels much cleaner already.

---

We could argue that because the new get method is on config, we could forgo the entire config passing to the extractor. And potentially enforce a usage pattern by not providing the prefix either.

It could get as simple as:

data class Module(val initMessage: String, val stopMessage: String)

object server : PropertyGroup() {
    val modules by listType(stringType)

    fun modulesInitMessage(id: String) = key("${server::modules.name}.$id.initMessage", stringType)
    fun modulesStopMessage(id: String) = key("${server::modules.name}.$id.stopMessage", stringType)
}

val mapModules : Map<String, Module> = config.get(server.modules){
    Module(config[server.modulesInitMessage(it)], config[server.modulesStopMessage(it)])
}

What do you think? Is it worth supporting this?

Property group with underscore in name doesn't work

import com.natpryce.konfig.*
import org.junit.Test

class KonfigUnderscoreTest {

    object db_server : PropertyGroup() {
        val port by intType
        val host by stringType
    }

    val config = ConfigurationProperties.fromResource("db_server.properties")

    val port = config[db_server.port]
    val host = config[db_server.host]

    @Test
    fun konfig_test(){
        println("$host:$port")
    }

}

com.natpryce.konfig.Misconfiguration: db-server.port property not found; searched:

• db-server.port in resource db_server.properties

It seems to silently convert underscores to dashes somewhere.

Packages and file facades are not yet supported in Kotlin reflection

Using:

fun main(args: Array<String>) {
    val p = object : PropertyGroup(){
        val aa by intType
    }

    val config = ConfigurationProperties.fromResource("test.props")
    println(config[p.aa])
}

And test.props:
aa=3

I get:

Exception in thread "main" java.lang.UnsupportedOperationException: Packages and file facades are not yet supported in Kotlin reflection. Meanwhile please use Java reflection to inspect this class: class ConfigTestKt
    at kotlin.reflect.jvm.internal.KClassImpl.reportUnresolvedClass(KClassImpl.kt:170)
    at kotlin.reflect.jvm.internal.KClassImpl.access$reportUnresolvedClass(KClassImpl.kt:38)
    at kotlin.reflect.jvm.internal.KClassImpl$descriptor_$1.invoke(KClassImpl.kt:46)
    at kotlin.reflect.jvm.internal.KClassImpl$descriptor_$1.invoke(KClassImpl.kt:38)
    at kotlin.reflect.jvm.internal.ReflectProperties$LazySoftVal.invoke(ReflectProperties.java:93)
    at kotlin.reflect.jvm.internal.KClassImpl.getDescriptor(KClassImpl.kt:50)
    at kotlin.reflect.jvm.internal.KClassImpl$objectInstance_$1.invoke(KClassImpl.kt:136)
    at kotlin.reflect.jvm.internal.ReflectProperties$LazyVal.invoke(ReflectProperties.java:63)
    at kotlin.reflect.jvm.internal.KClassImpl.getObjectInstance(KClassImpl.kt:149)
    at com.natpryce.konfig.PropertyGroup.outer(magic.kt:16)
    at com.natpryce.konfig.PropertyGroup.namePrefix(magic.kt:18)
    at com.natpryce.konfig.PropertyGroup.name(magic.kt:17)
    at com.natpryce.konfig.PropertyGroup.key(magic.kt:23)
    at com.natpryce.konfig.MagicKt.getValue(magic.kt:28)
    at ConfigTestKt$main$p$1.getAa(ConfigTest.kt)
    at ConfigTestKt.main(ConfigTest.kt:12)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

This is with Kotlin 1.0.2-release-IJ143-96.