soyking / e3ch

etcd v3 client with hierarchy

e3ch

etcd v3 client with hierarchy

There are directory and key-value in etcd v2, which is convenient to manage the key-value store. But etcd v3 only supports flat key-value space (see #633).Though you could use prefix to adjust the new API, it is not easy to manage key-value store or make the structure clearly. e3ch is built for making etcd v3 'look like' a key-value store supporting hierarchy.

Design

e3ch will set a special value for a directory, which can be defined by user. For a key /database/mysql/setting.json, the directory key such as /database and /database/mysql will be set as the special value.

When creating/putting key-value, e3ch will confirm the parent key should be the directory special value. For example, when putting /database/fake-mysql/setting.json, if the value of /database/fake-mysql is not the special value, the action will be rejected.

When listing/deleting a directory, e3ch will get key-value with the prefix. For example, when listing /database, e3ch will get all key-value with prefix /database/. But it will only return the keys under the directory, and the keys in deeper directory will be ignored. So it may cost too long if there are too many keys under the directory, escpecially when listing the root /

Example

package main

import (
	clientv3 "go.etcd.io/etcd/client/v3"
	"github.com/soyking/e3ch"
)

func main() {
	// initial etcd v3 client
	e3Clt, err := clientv3.New(clientv3.Config{Endpoints: []string{"127.0.0.1:2379"}})
	if err != nil {
		panic(err)
	}

	// new e3ch client with namespace(rootKey)
	clt, err := client.New(e3Clt, "my_space")
	if err != nil {
		panic(err)
	}

	// set the rootKey as directory
	err = clt.FormatRootKey()
	if err != nil {
		panic(err)
	}

	clt.CreateDir("/dir1")
	clt.Create("/dir1/key1", "")
	clt.Create("/dir", "")
	clt.Put("/dir1/key1", "value1")
	clt.Get("/dir1/key1")
	clt.List("/dir1")
	clt.Delete("/dir")
}

more examples in *_test.go

Run Test

go test -check.vv

e3ch's Issues

etcd import path errors

Hi,

Hope you are all well !

I think that e3ch needs a rewrite of the import paths ^^

For now, I have the following error:

# github.com/soyking/e3ch
../../.go/pkg/mod/github.com/soyking/[email protected]/get.go:19:32: cannot use resp.Kvs[0] (type *"github.com/coreos/etcd/mvcc/mvccpb".KeyValue) as type *"go.etcd.io/etcd/mvcc/mvccpb".KeyValue in argument to clt.createNode
../../.go/pkg/mod/github.com/soyking/[email protected]/list.go:40:34: cannot use rangeResp.Kvs (type []*"github.com/coreos/etcd/mvcc/mvccpb".KeyValue) as type []*"go.etcd.io/etcd/mvcc/mvccpb".KeyValue in argument to clt.list

Is it possible to sort it out ?

Thanks for any inputs or insights about that.

Cheers,
X

同学，您这个项目引入了67个开源组件，存在2个漏洞，辛苦升级一下

检测到 soyking/e3ch 一共引入了67个开源组件，存在2个漏洞

漏洞标题：go-yaml < 2.2.8拒绝服务漏洞
漏洞编号：CVE-2019-11254
漏洞描述：gopkg.in/yaml.v2是go语言中用于处理yaml格式的包。
在2.2.8之前的版本中，处理恶意的yaml数据时，会导致CPU资源耗尽。
漏洞由Kubernetes开发者在fuzz测试中发现并提交修复补丁。
国家漏洞库信息：https://www.cnvd.org.cn/flaw/show/CNVD-2020-35519
影响范围：(∞, 2.2.8)
最小修复版本：2.2.8
缺陷组件引入路径：github.com/soyking/e3ch@->go.etcd.io/etcd/api/[email protected]>github.com/grpc-ecosystem/[email protected]>gopkg.in/[email protected]

另外还有2个漏洞，详细报告：https://mofeisec.com/jr?p=n661af

key is not under a directory or key is a directory or key is not empty

Hi i trying to create a directory with this client but as you can see it's not going very well it throws an error stating key is not under a directory or key is a directory or key is not empty not just this but i cannot even list directories

// initial etcd v3 client
	e3Clt, err := clientv3.New(clientv3.Config{Endpoints: []string{"127.0.0.1:2379"}})
	if err != nil {
		panic(err)
	}

	// new e3ch client with namespace(rootKey)
	clt, err := client.New(e3Clt, "my_space")
	if err != nil {
		panic(err)
	}

	// set the rootKey as directory
	err = clt.FormatRootKey()
	if err != nil {
		panic(err)
	}


	res :=  clt.CreateDir("/dir1") 

	fmt.Println(res)