kenshaw / sdhook

Package sdhook provides a Google Stackdriver logging hook for logrus

About sdhook

Package sdhook provides a logrus compatible hook for Google Stackdriver logging.

Installation

Install in the usual Go way:

go get -u github.com/kenshaw/sdhook

Usage

Simply create the hook, and add it to a logrus logger:

// create hook using service account credentials
h, err := sdhook.New(
	sdhook.GoogleServiceAccountCredentialsFile("./credentials.json"),
)

// create logger with extra fields
//
// logrus fields will be converted to Stackdriver labels
logger := logrus.New().WithFields(logrus.Fields{
	"field1": 15,
	"field2": 20,
})

// add hook
logger.Hooks.Add(h)

// log something
logger.Printf("something %d", 15)

The example above sends log entries directly to the logging API. If you have the logging agent running, you can send log entries to it instead, with the added benefit of having extra instance metadata added to your log entries by the agent. In the example above, the initialization would simply be:

// create hook using the logging agent
h, err := sdhook.New(
	sdhook.GoogleLoggingAgent(),
)

Please also see _example/example.go for a more complete example.

Error Reporting

If you'd like to enable sending errors to Google's Error Reporting (https://cloud.google.com/error-reporting/), you have to set the name of the service, app or system you're running. Following the example above, the initialization would then be:

// create hook using the logging agent
h, err := sdhook.New(
	sdhook.GoogleLoggingAgent(),
	sdhook.ErrorReportingService("your-great-app"),
)

The value of the ErrorReportingService function parameter above corresponds to the string value you'd like to see in the service field of the Error Reporting payload, as defined by https://cloud.google.com/error-reporting/docs/formatting-error-messages

Also note that, if you enable error reporting, errors and messages of more severe levels go into the error log and will not be displayed in the regular log. The error log name is either defined by the ErrorReportingLogName function or defaults to <regular-log-name>_errors. This fulfills Google's Error Reporting requirement that the log name should have the string err in its name. See more in: https://cloud.google.com/error-reporting/docs/setup/ec2

To fulfill Google's Error Reporting requirement of a payload containing error stack frame information (file name, function name and line number), it assumes that this information has been added as a logrus.Field of name caller and type stack.Frame from Facebook's stack package. One way to easily achieve this transparently is to use another logrus Hook like Gurpartap's logrus-stack.

See GoDoc for a full API listing.

sdhook's Issues

JSON Payload

Is it possible to attach a JSON payload to the log message?

Logging Errors entries to Stackdriver Logging

Hi guys, how are you doing? Does anyone experienced errors while sending log entries (ERROR LEVEL) to Logentries?

Here we did the basic, got the log and the hook working but we can't see ERROR level entries.

We log this

log.Info("Consumer LOG Info")
log.Warn("Consumer LOG Warn")
log.Warning("Consumer LOG Warning")
log.Error("Consumer LOG Error")

In the console we have

INFO[2019-03-29T18:33:44-03:00] Consumer LOG Info
WARN[2019-03-29T18:33:44-03:00] Consumer LOG Warn
WARN[2019-03-29T18:33:44-03:00] Consumer LOG Warning
ERRO[2019-03-29T18:33:44-03:00] Consumer LOG Error

But in the Stackdriver logs all we see is:

2019-03-29 18:33:44.000 BRT Consumer LOG Info
2019-03-29 18:33:44.000 BRT Consumer LOG Warn
2019-03-29 18:33:44.000 BRT Consumer LOG Warning

We have even tried to set all log levels in logger config doing this

stackdriverHook, err := sdhook.New(
	sdhook.GoogleServiceAccountCredentialsFile(logStackdriverCredentialsFile),
	sdhook.LogName(logAppName),
	sdhook.Resource(sdhook.ResTypeGlobal, globalLabels),
	sdhook.Levels(logrus.PanicLevel, logrus.FatalLevel, logrus.ErrorLevel, logrus.WarnLevel, logrus.InfoLevel),
)

:( Can anyone help us?

Thank you,

Lucas

Update dependencies

Hi,

Can we update the dependencies of the package ?

Best,

error reporting nil pointer

When i log error log with error reporting configured, code crash due of a nil pointer

Minimum code to reproduce

package main

import (
	"time"

	"github.com/knq/sdhook"
	"github.com/sirupsen/logrus"
)

func main() {
	// create a logger with some fields
	logger := logrus.New()
	logger.WithFields(logrus.Fields{
		"my_field":  115888,
		"my_field2": 898858,
	})

	// create stackdriver hook
	hook, err := sdhook.New(
		sdhook.GoogleServiceAccountCredentialsFile("credentials.json"),
		sdhook.LogName("some_log"),
		sdhook.ErrorReportingService("some_log"),
	)
	if err != nil {
		logger.Fatal(err)
	}

	// add to logrus
	logger.Hooks.Add(hook)

	// log some message
	logger.Errorf("a random message @ %s", time.Now().Format("15:04:05"))

	// wait for the writes to finish
	time.Sleep(10 * time.Second)
}

Error is just here https://github.com/knq/sdhook/blob/master/sdhook.go#L235 sh.errorService is nil

Any idea why ?

Thank you

link req and app logs

It seems like link req and app logs is not supported.
https://cloud.google.com/logging/docs/view/service/appengine-logs#linking_application_logs_and_requests

Am I right?

Question on packages used

Hi,
I see the current Stackdriver docs reference cloud.google.com/go/logging and cloud.google.com/go/errorreporting as the client library packages to use when you want to call Stackdriver from your Go app, however I see the code use different packages(google.golang.org/api/clouderrorreporting/v1beta1 and
google.golang.org/api/logging/v2)

Is there a specific reason for this?

Pieter

Cannot write logs to Google Logging Agent

Using head of master at sdhook (7a285b4) with head of master at fluent/fluent-logger-golang (fluent/fluent-logger-golang@90f0f02) cannot write any logs to Google Logging Agent due to following issues:

• AsyncConnect option was deprecated and replaced with Async, and
• MaxRetry option is -1, fluent-logger-golang never write logs.

opts.go

sh.agentClient, err = fluent.New(fluent.Config{
	AsyncConnect: true,
	MaxRetry:     -1,
})

These issues are comes from commit fluent/fluent-logger-golang#56.

New Trace log level in logrus

Hello, I'm just opening this issue to let you know, the next release of logrus (v1.2.0) will add a new trace level named Trace below Debug.
Here is the PR for reference sirupsen/logrus#844
You may want to take into account this new level.