go-chassis / go-archaius

a dynamic configuration framework used in distributed system

中文版README

Go-Chassis is a microservice framework for rapid development of microservices in Go. it focus on helping developer to deliver cloud native application more easily. The idea of logo is, developer can recreate and customize their own "wheel"(a framework) by go chassis to accelerate the delivery of software.

Why use Go chassis

• powerful middleware "handler chain": powerful than "filter" or "interceptor". each handler in chain is able to get the running result of backward handler and your business logic. It is very useful in varies of scenario, for example:

1. a circuit breaker need to check command results
2. track response status and record it, so that prometheus can collect them
3. track critical response result, so that you can audit them
4. distribute tracing, you can complete the end span spec after business logic executed

the commonplace above is helping you decouple common function from business logic. without handler chain. those function will couple with business logic

• go chassis is designed as a protocol-independent framework, any protocol is able to integrate with go chassis and leverage same function like load balancing, circuit breaker,rate limiting, routing management, those function resilient your service

• go chassis makes service observable by bringing open tracing and prometheus to it.

• go chassis is flexible, many different modules can be replaced by other implementation, like registry, metrics, handler chain, config server etc

• With many build-in function like route management, circuit breaker, load balancing, monitoring etc, your don't need to investigate, implement and integrate many solutions yourself.

Features

• Pluggable discovery service: Support Service center, kubernetes. fit both client side discovery and server side discovery pattern, and you can disable service discovery to use end to end communication.
• Pluggable Protocol: You can customize protocol, by default support http and grpc, go chassis define standardized model to makes all request of different protocol leverage same features
• Multiple server management: you can separate API by protocols and ports
• Handler Chain: Able to add your own code during service calling for client and server side
• rich middleware: based on handler chain, supply circuit breaker, rate limiting, monitoring, auth features. see
• Traffic marker Traffic marker module is able to mark requests in both client(consumer) or server(provider) side, with marker, you can govern traffic based on it.
• Traffic management: Able to route to different service based on weight and match rule, it can be used in many scenario, such as canary release
• Security: build in cipher, authentication, RSA related funtions
• Safety and resilience: support fault-tolerant(retry, rate limiting, client-side load-balancing, circuit breaker) to makes your service facing any unpredictable situation.
• Telemetry: Able to expose Prometheus metric API automatically and customize metrics report. Use opentracing-go as standard library.
• Backing services: use backend service as a plugin, so that your app can be easily tested, and swap to another plugin.
• Hot re-configuration: Powered by go-archaius, configurations can be reload in runtime, like load balancing, circuit breaker, rate limiting, developer is also able to develop a service which has hot-reconfiguration feature easily. see
• API first go chassis will automatically generate Open API 2.0 doc and register it to service center. you can manage all the API docs in one place
• Spring Cloud integrate with servicecomb, go chassis can work together with spring cloud.
• Service mesh: you can introduce multi-language to your microservice system. powered by servicecomb-mesher.
• Less dependencies: checkout the go.mod file, it has less dependency on open source project by default, to import more features checkout plugins to see more features

Get started

1.Generate go mod

go mod init

2.Add go chassis

 go get github.com/go-chassis/go-chassis/[email protected]

if you are facing network issue

export GOPROXY=https://goproxy.io

3.Write your first http micro service

Documentations

You can see more documentations in here,

Examples

You can check examples here

NOTICE: Now examples is migrating to here

Communication Protocols

Go-Chassis supports 2 types of communication protocol.

1. http - an approach that leverages the HTTP protocol for communication.
2. gRPC - native grpc protocol, go chassis bring circuit breaker, route management etc to grpc.

Debug suggestion for dlv:

Add -tags debug into go build arguments before debugging, if your go version is go1.10 onward.

example:

go build -tags debug -o server -gcflags "all=-N -l" server.go

Chassis customized debug tag to resolve dlv debug issue:

golang/go#23733

https://github.com/derekparker/delve/issues/865

Other project using go-chassis

• apache/servicecomb-kie: A cloud native distributed configuration management service, go chassis and mesher integrate with it, so that user can manage service configurations by this service.
• apache/servicecomb-mesher: A service mesh able to co-work with go chassis, it is able to run as a API gateway also.
• KubeEdge: Kubernetes Native Edge Computing Framework (project under CNCF) https://kubeedge.io

Known Users

To register your self, go to #592

Contributing

If you're interested in being a contributor and want to get involved in developing, please check CONTRIBUTING and wiki for details.

Join slack

Committer

• ichiro999
• humingcheng

go-archaius's Issues

readme文档拼写错误

This is a dynamic configuration clinet for golang just like archaius for Java

client拼写错误

华为云官网配置中心指导配置type为config_center，实际应该是config-center

Describe the bug
华为云官网https://support.huaweicloud.com/devg-servicestage/ss-devg-0009.html，对于配置中心配置type为 type: config_center，实际应该是config-center

Version of go chassis

To Reproduce
Steps to reproduce the behavior:

Logs

always empty events for first added config

before I start I comment below config

after started I uncomment this config to make rate limiting work, but events is empty

Archaius support file content as a value

in real use case, user don't want to manage configs in a file separately. instead they want to use file content as a config value

replace open-logging with zerolog

to reduce the dependencies and make the logging performance better, I suggest to replace open-logging with zerolog(https://github.com/rs/zerolog)

访问空指针

配置文件修改事件处理函数compareUpdate，可能访问空指针，导致panic。
if confInfo == nil { confInfo.Value = nil continue }
https://github.com/go-chassis/go-archaius/blob/master/source/file/file.go

对于配置文件的配置读取，新增环境变量语法的兼容。

例如下面的例子，使用如果配置文件中的value有${GO||/usr/local/go}这种格式的字符串，就去获取一下环境变量${GO}的值，如果环境变量为空，就返回||与)之间的默认字符串。

demo.yaml :

gopath: ${GO||/usr/local/go}	// return the default value "/usr/local/go/".

模块事件行为优化

1. 目前模块事件只支持模块key类似“AAA"，而不支持模块key类似”AAA.BBB“
2. 触发模块事件调度时，同时会尝试去调度普通事件的回调
3. CustomInit方法没有对running状态的判断和修改，并没有产生完整的Init作用

[]string 字符串切片解析会出错

Add GetConfigsWithSourceNames() api to get all configs and its sources

Is your feature request related to a problem? Please describe.

This feature is handy for debugging, when the user wants to check all config key values along with its source, the current GetConfigs api only returns a map with key and values. User does not know which source the key reads from.

Describe the solution you'd like

Add an api GetConfigsWithSourceNames, modified base on GetConfigs api in archaius.go.

GetConfigsWithSourceNames gives the information about all configurations
each config key, along with its source will be returned
// the returned map will be like:
// map[string]interface{}{
// key string: map[string]interface{"value": value, "sourceName": sourceName}
// }

Describe alternatives you've considered
build a separate map when archaius reads keys from different sources. which is not an efficient solution.

Additional context

mentioned in #131

ssl不支持有bool类型的参数

Describe the bug
A clear and concise description of what the bug is.

Version of go archaius
v1.3.2
To Reproduce
Steps to reproduce the behavior:
ssl:
rest.Provider.verifyPeer: false
rest.Provider.cipherPlugin: go-lib-aes
rest.Provider.cipherSuits: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
rest.Provider.protocol: TLSv1.2
rest.Provider.caFile: xxx
rest.Provider.certFile: xxx
rest.Provider.keyFile: xxx
rest.Provider.certPwdFile: xxx

存在bool类型的，就会出现读取不到这些配置情况，导致证书加载部分异常

代码异常部分
unmarshal.go里面的populateMap函数

		// maybe next map type，此时mapValueType 为string，而 setVal.Type() 为bool，导致函数退出，没有获取完需要的值
		if mapValueType != setVal.Type() {
			return rValue, nil

		}

Logs

希望增加kie project参数配置

希望完善kie配置中心接入

apollo 配置支持解析 yaml json 等

apollo config.yaml 命名空间配置示例

a:
  b:
    c: 1

apollo 支持 yaml json 等格式的命名空间, 目前的做法是直接作为一个配置, 如 config.yaml 命名空间会被读取成 key 为
config.yaml.content, value 为 yaml 内容的一个配置.

希望可以实现解析, 按照 a.b.c 这样的方式直接读取到配置内容

对 kie status进行支持

类似huaweicloud/spring-cloud-huawei@c5f148e

配置热更新回调接口修改成同一个module里的event就通知一次。

场景：

项目中配置的使用，一般都是多个配置项合在一起使用，比如mysql的配置一般都会有账号、密码、端口地址等。如果在项目运行过程中，需要修改该项配置，同时修改了账号、密码和端口。event就会通知同一个模块多次，业务方没有考虑这种情况的时候，就会热更新多次，存在问题，即使订阅多个key，感觉也不能避免回调方回调多次的情况。

建议：

是否可以考虑将Listener接口的event做成切片，这样各个配置源就有更大的操作空间，给予到业务方就会只有一个热更新通知。

type Listener interface {
	Event(event *Event)
}

修改为：

type Listener interface {
	Event(event []*Event)
}

接口地址：

go-archaius/event/event_system.go

Line 53 in f6cc7f3

type Listener interface {

event可以监控配置中心的namespace吗？

go-archaius/core/core.go

Lines 68 to 73 in 96fc4ad

	type Event struct {
	EventSource string
	EventType string
	Key string
	Value interface{}
	}

go-archaius/core/config-manager/configurationmanager.go

Line 396 in 96fc4ad

sourceName, ok := configMgr.ConfigurationMap[event.Key]

从这两处看event的key貌似就是配置信息的key。

event有办法做到监控整个文件吗？比如配置中心中有个namespace叫test，能注册针对这个test的event吗，而不仅仅是针对其中的某个键值对。

UseFileNameAsKeyContentAsValue function import invalid in examples/file/ main.go

FileHandler has been moved to package utils from package filesource

添加一个优先级为5的配置源

目前 archaius 里的配置源，文件配置源优先级是最低的。
我觉得有一个比文件配置源优化级更低配置源是有必要的。
例如，我有这样的场景：go-chassis 的配置项太多了，并且有一些并非业务愿意关心的（如service-center的地址），我想为其中多数的配置项提供默认值，其他人使用大数时候直接使用默认值，感知不到这些配置项；但同时我也保留他自己去配置的能力，当他有需要的时候，他还是可以去了解、去(在文件中)配置出这些值。
目前对个场景我没有什么很好的办法，我可以在我的封装代码里设置对应配置项的环境变量，但是环境变量会覆盖文件，用户的配置能力就会被剥夺掉一些。
如果我有一个配置源，级别低于文件配置源，那我就可以通过它来实现这种设置默认值的工作。
实现路径最简单的就是，允许创建自定义级别的 Memory 配置源。

github.com/go-chassis/[email protected]: invalid pseudo-version: does not match version-control timestamp (expected 20180927015433)

go mod download

go: github.com/go-chassis/[email protected] requires
        github.com/go-chassis/[email protected] requires
        github.com/go-chassis/[email protected]: invalid pseudo-version: does not match version-control timestamp (expected 20180927015433)

how to get config value from the local file?

I define a config file about database setting. follow example code in URL:
https://go-chassis.readthedocs.io/en/latest/user-guides/dynamic-conf.html

	func main() {
	    chassis.RegisterSchema("rest", &schemas.RestFulHello{})
	    chassis.RegisterSchema("rest", &schemas.RestFulMessage{})
	    //start all server you register in server/schemas.
	    if err := chassis.Init(); err != nil {
		lager.Logger.Error("Init failed.", err)
		return
	    }

	    //test code
	    dbCfgFile := fileutil.GetConfDir() + "\\db.yaml"
	    archaius.AddFile(dbCfgFile)
	    dbAddr:=archaius.GetString("dbconf.conn1.addr", "")
	    fmt.Println(dbAddr)

	    chassis.Run()
        }

but I can't get value of dbAddr, it's empty.

Then I check code of https://github.com/go-chassis/go-archaius/blob/master/examples/main.go, it's ok.
but I find code

        ConfigFactory.DeInit() 

in sample code, and this code will clean all config item about FileSource. Is there any problem with this behavior?

Later, I just use object of NewYamlConfigurationSource to get config value, like this:

	//test
	dbCfgFile := fileutil.GetConfDir() + "\\db.yaml"
	fSource := filesource.NewYamlConfigurationSource()
	fSource.AddFileSource(dbCfgFile, 0)
	dbAddr, _:=fSource.GetConfigurationByKey("dbconf.conn1.addr")

	fmt.Println("---------------- DB Addr:", dbAddr)

It's work, and get real value of dbAddr.

实现内存配置项备份特性

Is your feature request related to a problem? Please describe.
no
Describe the solution you'd like
1.框架提供一个备份选项，包括备份方式，备份周期等，当配置时，框架将自动进行备份
2.当开启备份功能时，框架初始化时，先load备份数据，然后在拉取其他的配置source
3.备份功能提供扩展功能，允许客户自行定义备份存放的地方（如OBS，本地，k8s的configmap等）

Describe alternatives you've considered
no

Additional context
no

user story
作为框架的client，希望框架具有配置备份功能，这样当所以远端配置服务中心无法服务时，能够让服务在启动时先使用备份的配置项进行工作，等远端的配置中心恢复后再更新配置项

Other project using go archaius

If you have a open source project which is using go-archaius, or you are bulding eco-system or go archaius source or config server client, hope you can share it with us in this format.

• [your project name](project link): introduction for your project

go const更新导致代码检查不通过

archaius.New should add config center init logic

Ohhh, this is a bigger than bigger bug

go-archaius/source/file/file.go

Line 297 in e9597f0

confInfo.Value = nil

more simple and clean examples, separate them in different go files, not all in one

1. event listener
2. source priority
3. custom source
4. file source add file

custom Event not work

Hi, I Have a custom event listen for config file changes, but it not work, can you help me ?

package main

import (
	"fmt"
	"github.com/go-chassis/go-archaius"
	"github.com/go-chassis/go-archaius/core"
)
type Listener struct {
	Key string
}
func (e *Listener) Event(event *core.Event) {
	//  I want to dynamically get the latest values here.
	fmt.Printf("****************%v",event)
}
func main(){
	archaius.Init(archaius.WithRequiredFiles([]string{
		"./test.yaml",
	}))
	select{}
}

I want to get real-time changed values via Event function , but it didn't do anything, what do I need to do to make it work?

当获取不到配置的时候，指针类型应该赋予nil

go-archaius/source/unmarshal.go

Line 91 in 4b83c04

if rValue.IsNil() {

在使用UnmarshalConfig的时候，如果yaml文件或者任意一个配置源获取不到配置，该项字段应该赋予零值，指针类型赋予nil。

实现自定义配置中心的Client interface 参数疑问

文件名

source/remote/client.go

接口

type Client interface {
	//PullConfigs pull all configs from remote
	PullConfigs(labels ...map[string]string) (map[string]interface{}, error)
	//PullConfig pull one config from remote
	PullConfig(key, contentType string, labels map[string]string) (interface{}, error)
	// PushConfigs push config to cc
	PushConfigs(data map[string]interface{}, labels map[string]string) (map[string]interface{}, error)
	// DeleteConfigsByKeys delete config for cc by keys
	DeleteConfigsByKeys(keys []string, labels map[string]string) (map[string]interface{}, error)
	//Watch get kv change results, you can compare them with local kv cache and refresh local cache
	Watch(f func(map[string]interface{}), errHandler func(err error), labels map[string]string) error
	Options() Options
}

PullConfig

这个参数:contentType是什么意思

---

PS.目前我们的项目正在使用go-chassis来开发一个业务框架，已有的配置中心为apollo，正在做support apollo的工作，想问下目前为什么停止对apollo的开发了？

Set方法存在锁没有释放问题

Describe the bug
Set方法没有释放锁，会导致进程无法运行

Version of go archaius
v1.4.0
To Reproduce

1. 先使用Set方法设置配置项
2. 调用AddRemoteSource时，出现进程无法运行

与servicecomb-kie项目相互依赖

examples/event示例：配置刷新失败

我想结合这个示例看一下Event部分的源码，但是示例跑的似乎有点问题：配置更改后，刷新失败。
运行环境：win10，IDE：Goland

操作：将配置文件中age的值由13改为132
日志如下：
2019/04/15 14:30:50 13
2019/04/15 14:30:55 13
2019/04/15 14:31:00 13
2019/04/15 14:31:01 DEBUG: file event [E:\microservice\go-archaius-0.14.0\examples\event\event.yaml RENAME], operation is %!d(MISSING). reload it.
2019/04/15 14:31:01 WARN: [[E:\microservice\go-archaius-0.14.0\examples\event\event.yaml open E:\microservice\go-archaius-0.14.0\examples\event\event.yaml: The system cannot find the file specified.]] file does not exist so not able to watch further
2019/04/15 14:31:01 DEBUG: file event [E:\microservice\go-archaius-0.14.0\examples\event\event.yaml___jb_old___ WRITE], operation is %!d(MISSING). reload it.
2019/04/15 14:31:01 DEBUG: Event generated events [[]]
2019/04/15 14:31:01 DEBUG: file event [E:\microservice\go-archaius-0.14.0\examples\event\event.yaml___jb_old___ REMOVE], operation is %!d(MISSING). reload it.
2019/04/15 14:31:01 WARN: the file change mode: ["E:\microservice\go-archaius-0.14.0\examples\event\event.yaml___jb_old___": REMOVE], continue
2019/04/15 14:31:05 13
2019/04/15 14:31:10 13

configmap can not be UnmarshalConfig

Describe the bug
A clear and concise description of what the bug is.
use archaius to parse configmap

        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: roboartisan-robooperationmanagement-config
          labels:
            name: roboartisan-robooperationmanagement-config
        data:
            database.ip: 0.0.0.0
            database.port: 5432
            database.user_name: root

the struct database

type Database struct {
	UserName string `json:"user_name"`
	Password string `json:"password"`
	IP       string `json:"ip"`
	Port     int    `json:"port"`
	DBName   string `json:"db_name"`
	SslMode  string `json:"ssl_mode"`
	MaxIdle  int    `json:"max_idle"`
	MaxConn  int    `json:"max_conn"`
}

After archaius.UnmarshalConfig ，string type data can be parse, but int type data fail to parse
Version of go chassis
v1.5.4
To Reproduce
Steps to reproduce the behavior:

Logs
#``

Got nil when using GetConfigurationByKey to get a array from a yaml file

I have a yaml file with this structure:
mq:
endpoints:
- hosts: 127.0.0.1
port: 5672
- hosts: 192.168.1.1
port: 5672

The return value of GetConfigurationByKey("mq.endpoints") is nil

Capability to read config from K8S configmap

A new folder contains the configuration file will be mounted to POD when using K8S configmap. When the configmap changing, the old folder will be removed and a new one will be created. Currently, such scenarios can not be handled coz the file-source can only watch the existing file upgrade from the first loading.
We need a new configmap source to watch the parent directory to load any new files once configmap changes.