Make log level a label about loki

[tomwilkie]

Going to close as a dupe of #88, although not quite the same the solutions are consistent - we want to extract data from the log lines themselve into labels. A good example is log level.

[klausenbusk]

How exactly is the current solution implemented? I assume it is implemented client side and Grafana parse every logline for a log level (warn, info, ...)? But I haven't found the relevant code yet.

Edit: Found it: https://github.com/grafana/grafana/blob/e1acc772979e4dd8db4fbaeda58b842991c00f99/packages/grafana-data/src/types/logs.ts#L9

[davkal]

To have the log level as a label, it must be done by the logging agent, e.g., Promtail is doing it via pipelines: https://github.com/grafana/loki/blob/master/docs/clients/promtail/pipelines.md

Grafana cannot create its own set of labels. It does two different, additional things: using the level label (if present) as a hint to color the log volume histogram. If it cannot find level as a label, it parses each log line like you described. The other thing grafana does is parse log lines into fields, i.e., JSON or logfmt structured logs. Those fields are then shown separately when expanding a single log row in Explore.

[klausenbusk]

Grafana cannot create its own set of labels. It does two different, additional things: using the level label (if present) as a hint to color the log volume histogram. If it cannot find level as a label, it parses each log line like you described. The other thing grafana does is parse log lines into fields, i.e., JSON or logfmt structured logs. Those fields are then shown separately when expanding a single log row in Explore.

Thanks for the explanation :)

For anyone else stumbling on this issue and wondering how it is implemented, the relevant code is here:
https://github.com/grafana/grafana/blob/e1acc772979e4dd8db4fbaeda58b842991c00f99/packages/grafana-data/src/types/logs.ts#L9
https://github.com/grafana/grafana/blob/e1acc772979e4dd8db4fbaeda58b842991c00f99/public/app/core/logs_model.ts#L298-L300
https://github.com/grafana/grafana/blob/e1acc772979e4dd8db4fbaeda58b842991c00f99/public/app/core/logs_model.ts#L314-L320
https://github.com/grafana/grafana/blob/e1acc772979e4dd8db4fbaeda58b842991c00f99/packages/grafana-data/src/utils/logs.ts#L13-L42

The other thing grafana does is parse log lines into fields, i.e., JSON or logfmt structured logs. Those fields are then shown separately when expanding a single log row in Explore.

Explained here: https://grafana.com/blog/2019/11/25/grafana-v6.5-released/#explore-logs-log-row-details