samber / slog-formatter

🚨 slog: Attribute formatting

slog: Attribute formatting

Common formatters for slog library + helpers for building your own.

Handlers:

• NewFormatterHandler: main handler
• NewFormatterMiddleware: compatible with slog-multi middlewares

Common formatters:

• TimeFormatter: transforms a time.Time into a readable string
• UnixTimestampFormatter: transforms a time.Time into a unix timestamp.
• TimezoneConverter: set a time.Time to a different timezone
• ErrorFormatter: transforms a go error into a readable error
• HTTPRequestFormatter: transforms a *http.Request into a readable object
• HTTPResponseFormatter: transforms a *http.Response into a readable object
• PIIFormatter: hide private Personal Identifiable Information (PII)
• IPAddressFormatter: hide ip address from logs
• FlattenFormatterMiddleware: returns a formatter middleware that flatten attributes recursively

Custom formatter:

• Format: pass any attribute into a formatter
• FormatByKind: pass attributes matching slog.Kind into a formatter
• FormatByType: pass attributes matching generic type into a formatter
• FormatByKey: pass attributes matching key into a formatter
• FormatByFieldType: pass attributes matching both key and generic type into a formatter
• FormatByGroup: pass attributes under a group into a formatter
• FormatByGroupKey: pass attributes under a group and matching key, into a formatter
• FormatByGroupKeyType: pass attributes under a group, matching key and matching a generic type, into a formatter

See also:

• slog-multi: slog.Handler chaining, fanout, routing, failover, load balancing...
• slog-formatter: slog attribute formatting
• slog-sampling: slog sampling policy

HTTP middlewares:

• slog-gin: Gin middleware for slog logger
• slog-echo: Echo middleware for slog logger
• slog-fiber: Fiber middleware for slog logger
• slog-chi: Chi middleware for slog logger
• slog-http: net/http middleware for slog logger

Loggers:

• slog-zap: A slog handler for Zap
• slog-zerolog: A slog handler for Zerolog
• slog-logrus: A slog handler for Logrus

Log sinks:

• slog-datadog: A slog handler for Datadog
• slog-betterstack: A slog handler for Betterstack
• slog-rollbar: A slog handler for Rollbar
• slog-loki: A slog handler for Loki
• slog-sentry: A slog handler for Sentry
• slog-syslog: A slog handler for Syslog
• slog-logstash: A slog handler for Logstash
• slog-fluentd: A slog handler for Fluentd
• slog-graylog: A slog handler for Graylog
• slog-slack: A slog handler for Slack
• slog-telegram: A slog handler for Telegram
• slog-mattermost: A slog handler for Mattermost
• slog-microsoft-teams: A slog handler for Microsoft Teams
• slog-webhook: A slog handler for Webhook
• slog-kafka: A slog handler for Kafka
• slog-nats: A slog handler for NATS
• slog-parquet: A slog handler for Parquet + Object Storage
• slog-channel: A slog handler for Go channels

🚀 Install

go get github.com/samber/slog-formatter

Compatibility: go >= 1.21

No breaking changes will be made to exported APIs before v2.0.0.

⚠️ Warnings:

• in some case, you should consider implementing slog.LogValuer instead of using this library.
• use this library carefully, log processing can be very costly (!)

🚀 Getting started

The following example has 3 formatters that anonymize data, format errors and format user. 👇

import (
	slogformatter "github.com/samber/slog-formatter"
	"log/slog"
)

formatter1 := slogformatter.FormatByKey("very_private_data", func(v slog.Value) slog.Value {
    return slog.StringValue("***********")
})
formatter2 := slogformatter.ErrorFormatter("error")
formatter3 := slogformatter.FormatByType(func(u User) slog.Value {
	return slog.StringValue(fmt.Sprintf("%s %s", u.firstname, u.lastname))
})

logger := slog.New(
    slogformatter.NewFormatterHandler(formatter1, formatter2, formatter3)(
        slog.NewTextHandler(os.Stdout, nil),
    ),
)

err := fmt.Errorf("an error")
logger.Error("a message",
    slog.Any("very_private_data", "abcd"),
    slog.Any("user", user),
    slog.Any("err", err))

// outputs:
// time=2023-04-10T14:00:0.000000+00:00 level=ERROR msg="a message" error.message="an error" error.type="*errors.errorString" user="John doe" very_private_data="********"

💡 Spec

GoDoc: https://pkg.go.dev/github.com/samber/slog-formatter

NewFormatterHandler

Returns a slog.Handler that applies formatters to.

import (
	slogformatter "github.com/samber/slog-formatter"
	"log/slog"
)

type User struct {
	email     string
	firstname string
	lastname  string
}

formatter1 := slogformatter.FormatByKey("very_private_data", func(v slog.Value) slog.Value {
    return slog.StringValue("***********")
})
formatter2 := slogformatter.ErrorFormatter("error")
formatter3 := slogformatter.FormatByType(func(u User) slog.Value {
	return slog.StringValue(fmt.Sprintf("%s %s", u.firstname, u.lastname))
})

logger := slog.New(
    slogformatter.NewFormatterHandler(formatter1, formatter2, formatter3)(
        slog.NewTextHandler(os.StdErr, nil),
    ),
)

err := fmt.Errorf("an error")
logger.Error("a message",
    slog.Any("very_private_data", "abcd"),
    slog.Any("user", user),
    slog.Any("err", err))

// outputs:
// time=2023-04-10T14:00:0.000000+00:00 level=ERROR msg="a message" error.message="an error" error.type="*errors.errorString" user="John doe" very_private_data="********"

NewFormatterMiddleware

Returns a slog-multi middleware that applies formatters to.

import (
	slogformatter "github.com/samber/slog-formatter"
	slogmulti "github.com/samber/slog-multi"
	"log/slog"
)

formatter1 := slogformatter.FormatByKey("very_private_data", func(v slog.Value) slog.Value {
    return slog.StringValue("***********")
})
formatter2 := slogformatter.ErrorFormatter("error")
formatter3 := slogformatter.FormatByType(func(u User) slog.Value {
	return slog.StringValue(fmt.Sprintf("%s %s", u.firstname, u.lastname))
})

formattingMiddleware := slogformatter.NewFormatterHandler(formatter1, formatter2, formatter3)
sink := slog.NewJSONHandler(os.Stderr, slog.HandlerOptions{})

logger := slog.New(
    slogmulti.
        Pipe(formattingMiddleware).
        Handler(sink),
)

err := fmt.Errorf("an error")
logger.Error("a message",
    slog.Any("very_private_data", "abcd"),
    slog.Any("user", user),
    slog.Any("err", err))

// outputs:
// time=2023-04-10T14:00:0.000000+00:00 level=ERROR msg="a message" error.message="an error" error.type="*errors.errorString" user="John doe" very_private_data="********"

TimeFormatter

Transforms a time.Time into a readable string.

slogformatter.NewFormatterHandler(
    slogformatter.TimeFormatter(time.DateTime, time.UTC),
)

UnixTimestampFormatter

Transforms a time.Time into a unix timestamp.

slogformatter.NewFormatterHandler(
    slogformatter.UnixTimestampFormatter(time.Millisecond),
)

TimezoneConverter

Set a time.Time to a different timezone.

slogformatter.NewFormatterHandler(
    slogformatter.TimezoneConverter(time.UTC),
)

ErrorFormatter

Transforms a Go error into a readable error.

import (
	slogformatter "github.com/samber/slog-formatter"
	"log/slog"
)

logger := slog.New(
    slogformatter.NewFormatterHandler(
        slogformatter.ErrorFormatter("error"),
    )(
        slog.NewTextHandler(os.Stdout, nil),
    ),
)

err := fmt.Errorf("an error")
logger.Error("a message", slog.Any("error", err))

// outputs:
// {
//   "time":"2023-04-10T14:00:0.000000+00:00",
//   "level": "ERROR",
//   "msg": "a message",
//   "error": {
//     "message": "an error",
//     "type": "*errors.errorString"
//     "stacktrace": "main.main()\n\t/Users/samber/src/github.com/samber/slog-formatter/example/example.go:108 +0x1c\n"
//   }
// }

HTTPRequestFormatter and HTTPResponseFormatter

Transforms *http.Request and *http.Response into readable objects.

import (
	slogformatter "github.com/samber/slog-formatter"
	"log/slog"
)

logger := slog.New(
    slogformatter.NewFormatterHandler(
        slogformatter.HTTPRequestFormatter(false),
        slogformatter.HTTPResponseFormatter(false),
    )(
        slog.NewJSONHandler(os.Stdout, nil),
    ),
)

req, _ := http.NewRequest(http.MethodGet, "https://api.screeb.app", nil)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("X-TOKEN", "1234567890")

res, _ := http.DefaultClient.Do(req)

logger.Error("a message",
    slog.Any("request", req),
    slog.Any("response", res))

PIIFormatter

Hides private Personal Identifiable Information (PII).

IDs are kept as is. Values longer than 5 characters have a plain text prefix.

import (
	slogformatter "github.com/samber/slog-formatter"
	"log/slog"
)

logger := slog.New(
    slogformatter.NewFormatterHandler(
        slogformatter.PIIFormatter("user"),
    )(
        slog.NewTextHandler(os.Stdout, nil),
    ),
)

logger.
    With(
        slog.Group(
            "user",
            slog.String("id", "bd57ffbd-8858-4cc4-a93b-426cef16de61"),
            slog.String("email", "[email protected]"),
            slog.Group(
                "address",
                slog.String("street", "1st street"),
                slog.String("city", "New York"),
                slog.String("country", "USA"),
                slog.Int("zip", 12345),
            ),
        ),
    ).
    Error("an error")

// outputs:
// {
//   "time":"2023-04-10T14:00:0.000000+00:00",
//   "level": "ERROR",
//   "msg": "an error",
//   "user": {
//     "id": "bd57ffbd-8858-4cc4-a93b-426cef16de61",
//     "email": "foob*******",
//     "address": {
//       "street": "1st *******",
//       "city": "New *******",
//       "country": "*******",
//       "zip": "*******"
//     }
//   }
// }

IPAddressFormatter

Transforms an IP address into "********".

import (
	slogformatter "github.com/samber/slog-formatter"
	"log/slog"
)

logger := slog.New(
    slogformatter.NewFormatterHandler(
        slogformatter.IPAddressFormatter("ip_address"),
    )(
        slog.NewTextHandler(os.Stdout, nil),
    ),
)

logger.
    With("ip_address", "1.2.3.4").
    Error("an error")

// outputs:
// {
//   "time":"2023-04-10T14:00:0.000000+00:00",
//   "level": "ERROR",
//   "msg": "an error",
//   "ip_address": "*******",
// }

FlattenFormatterMiddleware

A formatter middleware that flatten attributes recursively.

import (
	slogformatter "github.com/samber/slog-formatter"
	slogmulti "github.com/samber/slog-multi"
	"log/slog"
)

logger := slog.New(
    slogmulti.
        Pipe(slogformatter.FlattenFormatterMiddlewareOptions{Separator: ".", Prefix: "attrs", IgnorePath: false}.NewFlattenFormatterMiddlewareOptions()).
        Handler(slog.NewJSONHandler(os.Stdout, nil)),
)

logger.
    With("email", "[email protected]").
    With("environment", "dev").
    WithGroup("group1").
    With("hello", "world").
    WithGroup("group2").
    With("hello", "world").
    Error("A message", "foo", "bar")

// outputs:
// {
//   "time": "2023-05-20T22:14:55.857065+02:00",
//   "level": "ERROR",
//   "msg": "A message",
//   "attrs.email": "[email protected]",
//   "attrs.environment": "dev",
//   "attrs.group1.hello": "world",
//   "attrs.group1.group2.hello": "world",
//   "foo": "bar"
// }

Format

Pass every attributes into a formatter.

slogformatter.NewFormatterHandler(
    slogformatter.Format(func(groups []string, key string, value slog.Value) slog.Value {
        // hide everything under "user" group
        if lo.Contains(groups, "user") {
            return slog.StringValue("****")
        }

        return value
    }),
)

FormatByKind

Pass attributes matching slog.Kind into a formatter.

slogformatter.NewFormatterHandler(
    slogformatter.FormatByKind(slog.KindDuration, func(value slog.Value) slog.Value {
        return ...
    }),
)

FormatByType

Pass attributes matching generic type into a formatter.

slogformatter.NewFormatterHandler(
    // format a custom error type
    slogformatter.FormatByType[*customError](func(err *customError) slog.Value {
        return slog.GroupValue(
            slog.Int("code", err.code),
            slog.String("message", err.msg),
        )
    }),
    // format other errors
    slogformatter.FormatByType[error](func(err error) slog.Value {
        return slog.GroupValue(
            slog.Int("code", err.Error()),
            slog.String("type", reflect.TypeOf(err).String()),
        )
    }),
)

⚠️ Consider implementing slog.LogValuer when possible:

type customError struct {
    ...
}

func (customError) Error() string {
    ...
}

// implements slog.LogValuer
func (customError) LogValue() slog.Value {
	return slog.StringValue(...)
}

FormatByKey

Pass attributes matching key into a formatter.

slogformatter.NewFormatterHandler(
    slogformatter.FormatByKey("abcd", func(value slog.Value) slog.Value {
        return ...
    }),
)

FormatByFieldType

Pass attributes matching both key and generic type into a formatter.

slogformatter.NewFormatterHandler(
    slogformatter.FormatByFieldType[User]("user", func(u User) slog.Value {
        return ...
    }),
)

FormatByGroup

Pass attributes under a group into a formatter.

slogformatter.NewFormatterHandler(
    slogformatter.FormatByGroup([]{"user", "address"}, func(attr []slog.Attr) slog.Value {
        return ...
    }),
)

FormatByGroupKey

Pass attributes under a group and matching key, into a formatter.

slogformatter.NewFormatterHandler(
    slogformatter.FormatByGroupKey([]{"user", "address"}, "country", func(value slog.Value) slog.Value {
        return ...
    }),
)

FormatByGroupKeyType

Pass attributes under a group, matching key and matching a generic type, into a formatter.

slogformatter.NewFormatterHandler(
    slogformatter.FormatByGroupKeyType[string]([]{"user", "address"}, "country", func(value string) slog.Value {
        return ...
    }),
)

🤝 Contributing

• Ping me on twitter @samuelberthe (DMs, mentions, whatever :))
• Fork the project
• Fix open issues or request new features

Don't hesitate ;)

# Install some dev dependencies
make tools

# Run tests
make test
# or
make watch-test

👤 Contributors

💫 Show your support

Give a ⭐️ if this project helped you!

📝 License

Copyright © 2023 Samuel Berthe.

This project is MIT licensed.

slog-formatter's Issues

Upgrade for log/slog

Is there a plan for slog-gorm?

HI @samber

Thanks for your awesome work for slog, is there a plan for slog-gorm? You know, gorm is an famouse ORM framework, it will be wonderful if there is a slog adapter for it.

Thank you.

[formatter_http] cannot use lo.MapToSlice(req.Header, ...)

It doesn't compile.

Environment

• OS: macOS Ventura 13.3.1 (a)
• Go: go1.20.4

How to reproduce

$ mkdir slog-demo
$ cd slog-demo
$ go mod init slog-demo

main.go

package main

import (
	"os"
	"time"

	slogfiber "github.com/samber/slog-fiber"
	slogformatter "github.com/samber/slog-formatter"
	"golang.org/x/exp/slog"
)

func main() {
	// Create a slog logger, which:
	//   - Logs to stdout.
	//   - RFC3339 with UTC time format.
	logger := slog.New(
		slogformatter.NewFormatterHandler(
			slogformatter.TimezoneConverter(time.UTC),
			slogformatter.TimeFormatter(time.RFC3339, nil),
		)(
			slog.NewTextHandler(os.Stdout, nil),
		),
	)
	slogfiber.New(logger)
}

$ go mod tidy

$ go build

Expected Result

build with no error

Actual Result

# github.com/samber/slog-formatter
../../../.gvm/pkgsets/go1.20.4/global/pkg/mod/github.com/samber/[email protected]/formatter_http.go:19:5: cannot use lo.MapToSlice(req.Header, func(key string, values []string) slog.Attr {…}) (value of type []slog.Attr) as []any value in argument to slog.Group
../../../.gvm/pkgsets/go1.20.4/global/pkg/mod/github.com/samber/[email protected]/formatter_http.go:39:6: cannot use lo.MapToSlice(req.URL.Query(), func(key string, values []string) slog.Attr {…}) (value of type []slog.Attr) as []any value in argument to slog.Group
../../../.gvm/pkgsets/go1.20.4/global/pkg/mod/github.com/samber/[email protected]/formatter_http.go:57:5: cannot use lo.MapToSlice(res.Header, func(key string, values []string) slog.Attr {…}) (value of type []slog.Attr) as []any value in argument to slog.Group