kvz / logstreamer

Prefixes streams (e.g. stdout or stderr) in Go

logstreamer

Prefixes streams (e.g. stdout or stderr) in Go.

If you are executing a lot of (remote) commands, you may want to indent all of their output, prefix the loglines with hostnames, or mark anything that was thrown to stderr red, so you can spot errors more easily.

For this purpose, Logstreamer was written.

You pass 3 arguments to NewLogstreamer():

• Your *log.Logger
• Your desired prefix ("stdout" and "stderr" prefixed have special meaning)
• If the lines should be recorded true or false. This is useful if you want to retrieve any errors.

This returns an interface that you can point exec.Command's cmd.Stderr and cmd.Stdout to. All bytes that are written to it are split by newline and then prefixed to your specification.

Don't forget to call Flush() or Close() if the last line of the log might not end with a newline character!

A typical usage pattern looks like this:

// Create a logger (your app probably already has one)
logger := log.New(os.Stdout, "--> ", log.Ldate|log.Ltime)

// Setup a streamer that we'll pipe cmd.Stdout to
logStreamerOut := NewLogstreamer(logger, "stdout", false)
defer logStreamerOut.Close()
// Setup a streamer that we'll pipe cmd.Stderr to.
// We want to record/buffer anything that's written to this (3rd argument true)
logStreamerErr := NewLogstreamer(logger, "stderr", true)
defer logStreamerErr.Close()

// Execute something that succeeds
cmd := exec.Command(
	"ls",
	"-al",
)
cmd.Stderr = logStreamerErr
cmd.Stdout = logStreamerOut

// Reset any error we recorded
logStreamerErr.FlushRecord()

// Execute command
err := cmd.Start()

Test

$ cd src/pkg/logstreamer/
$ go test

Here I issue two local commands, ls -al and ls nonexisting:

Over at Transloadit we use it for streaming remote commands. Servers stream command output over SSH back to me, and every line is prefixed with a date, their hostname & marked red in case they wrote to stderr.

License

This project is licensed under the MIT license, see LICENSE.txt.

logstreamer's Issues

panics when accessing from multiple goroutines

I observed this behaviour a few times in production as well as in test.

panic: runtime error: slice bounds out of range

goroutine 9 [running]:
bytes.(*Buffer).readSlice(0xc8200603f0, 0x10a, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/Cellar/go/1.5.1/libexec/src/bytes/buffer.go:382 +0x113
bytes.(*Buffer).ReadString(0xc8200603f0, 0xc82009070a, 0x0, 0x0, 0x0, 0x0)
    /usr/local/Cellar/go/1.5.1/libexec/src/bytes/buffer.go:395 +0x4d
github.com/viorama/chronos/backend/Godeps/_workspace/src/github.com/kvz/logstreamer.(*Logstreamer).OutputLines(0xc820060460, 0x0, 0x0)
    /Users/max/Code/go/src/github.com/viorama/chronos/backend/Godeps/_workspace/src/github.com/kvz/logstreamer/logstreamer.go:95 +0x4b
github.com/viorama/chronos/backend/Godeps/_workspace/src/github.com/kvz/logstreamer.(*Logstreamer).Write(0xc820060460, 0xc820180120, 0x115, 0x120, 0x115, 0x0, 0x0)
    /Users/max/Code/go/src/github.com/viorama/chronos/backend/Godeps/_workspace/src/github.com/kvz/logstreamer/logstreamer.go:71 +0x8c
fmt.Fprintln(0x75eb20, 0xc820060460, 0xc82000ea20, 0x2, 0x2, 0x1, 0x0, 0x0)

Thanks :)

Great library, just wanted to say that :) does one thing and does it well --kudos! :)

How do you output in red?

I'm trying to get my stderr prefix in red, but it is only red half the time.

Here is the code I'm using:

const (
	ErrorColor = "\033[1;31m%s\033[0m "
	InfoColor  = "\033[1;34m%s\033[0m "
)

	serverCmd = exec.Command("go", "run", "./src/main/main.go")
	logger := log.New(os.Stdout, "api", log.Ldate|log.Ltime)
	logStreamerOut := logstreamer.NewLogstreamer(logger, "stdout ", false)
	defer logStreamerOut.Close()
	logStreamerErr := logstreamer.NewLogstreamer(logger, fmt.Sprintf(ErrorColor, "err"), true)
	defer logStreamerErr.Close()
	serverCmd.Stdout = logStreamerOut
	serverCmd.Stderr = logStreamerErr
	serverCmd.Run()