didip / godown

Distributed, fault-tolerant key-value storage written in go.

Godown

A simple, distributed, fault-tolerant key-value storage inspired by Redis. It uses Raft protocotol as consensus algorithm. It supports the following data structures: String, Bitmap, Map, List.

How to install

Install via binaries

You can find binaries on the Github releases page.

/application/godown/godown-server -dir=/application/godown/data01 -id=01 -listen=127.0.0.1:14001 -raft=127.0.0.1:24001
/application/godown/godown-server -dir=/application/godown/data02 -id=02 -listen=127.0.0.1:14002 -raft=127.0.0.1:24002 -join=127.0.0.1:14001
/application/godown/godown-server -dir=/application/godown/data03 -id=03 -listen=127.0.0.1:14003 -raft=127.0.0.1:24003 -join=127.0.0.1:14001

Install via docker

# creating a network
docker network create godown 

# creating a volume
docker volume create godown 

# bootstrap a cluster with a single node
docker run -it --rm -v godown:/var/lib/godown --name=godown_1 --net=godown -p 5000:5000 \
    namreg/godown-server -id 1 -listen godown_1:5000 -raft godown_1:6000

# join the second node to the cluster
docker run -it --rm -v godown:/var/lib/godown --name=godown_2 --net=godown -p 5001:5001 \
    namreg/godown-server -id 2 -listen godown_2:5001 -join godown_1:5000 -raft godown_2:6001

# join the third node to the cluster
docker run -it --rm -v godown:/var/lib/godown --name=godown_3 --net=godown -p 5002:5002  \
    namreg/godown-server -id 3 -listen godown_3:5001 -join godown_1:5000 -raft godown_3:6002

Available options to run a server:

  -dir string
        Directory where data is stored.
  -gc duration
        Garbage collector interval.
  -id string
        Server unique id.
  -join string
        Server address to join.
  -listen string
        Server address to listen.
  -raft string
        Raft protocol listen address.
  -resp string
        Redis Serialization Protocol listen address.      
  -version
        Show version.

How to connect

You can connect to any godown node. All modifications will be replicated to all nodes.

Connect via any redis client

If you have specified resp address while starting a node, you can connect to the one by any redis client.

package main

import (
	"fmt"
	"net"
	"time"

	"github.com/garyburd/redigo/redis"
)

const connectionTimeout = 100 * time.Millisecond

func main() {
	conn, err := net.Dial("tcp", "6380")
	if err != nil {
		panic(err)
	}
	rconn := redis.NewConn(conn, connectionTimeout, connectionTimeout)

	reply, err := rconn.Do("LRANGE", "list", 0, 100)
	vals, err := redis.Strings(reply, err)

	if err != nil {
		panic(err)
	}

	fmt.Println(vals)
}

Connect via CLI

godown-cli

Available options:

  -host string
    	Host to connect to a server (default "127.0.0.1")
  -port string
    	Port to connect to a server (default "4000")
  -version
    	Show godown version.

Supported commands:

Command	Description
HELP command	Show the usage of the given command.
TYPE key	Returns the type stored at key.
KEYS pattern	Find all keys matching the given pattern.
PING [message]	Returns PONG if no argument is provided, otherwise return a copy of the argument as a bulk.
EXPIRE key seconds	Set a timeout on key. After the timeout has expired, the key will automatically be deleted.
TTL key	Returns the remaining time to live of a key. -1 returns if key does not have timeout.
---	---
SET key value	Set key to hold the string value. If key already holds a value, it is overwritten.
GET key	Get the value by key. If provided key does not exist NIL will be returned.
STRLEN key	Returns length of the given key. If key does not exists, 0 will be returned.
DEL key	Delete the given key.
---	---
SETBIT key offset value	Sets or clears the bit at offset in the string value stored at key.
GETBIT key offset	Returns the bit value at offset in the string value stored at key.
---	---
LPUSH key value [value ...]	Prepend one or multiple values to a list.
LPOP key	Removes and returns the first element of the list stored at key.
RPUSH key value [value ...]	Append one or multiple values to a list.
RPOP key	Removes and returns the last element of the list stored at key.
LLEN key	Returns the length of the list stored at key. If key does not exist, it is interpreted as an empty list and 0 is returned.
LINDEX key index	Returns the element at index index in the list stored at key. The index is zero-based, so 0 means the first element, 1 the second element and so on. Negative indices can be used to designate elements starting at the tail of the list.
LRANGE key start stop	Returns the specified elements of the list stored at key. The offsets start and stop are zero-based indexes, with 0 being the first element of the list (the head of the list), 1 being the next element and so on.
LREM key value	Removes all occurrences of elements equal to value from the list stored at key.
---	---
HSET key field value	Sets field in the hash stored at key to value.
HGET key field	Returns the value associated with field in the hash stored at key.
HKEYS key	Returns all field names in the hash stored at key. Order of fields is not guaranteed.
HVALS key	Returns all values in the hash stored at key.
HDEL key field [field ...]	Removes the specified fields from the hash stored at key. Returns the number of fields that were removed.

Connect via go client

package main

import (
	"fmt"

	"github.com/namreg/godown/client"
)

func main() {
	c, err := client.New("127.0.0.1:4000")
	if err != nil {
		panic(err)
	}
	defer c.Close()

	res := c.Get("key")
	if res.Err() != nil {
		panic(res.Err())
	}

	if res.IsNil() {
		fmt.Print("key does not exist")
	} else {
		fmt.Println(res.Int64())
	}
}

Client documentation available at godoc

TODO

• Write more docs
• Write more tests