pamburus / hl Goto Github PK

Is the following supported by having two filtered substrings?

./hl final.log -f message~=status -f container_name=datadog

I have it working if I only use one filter (each tested correctly by themselves) but when I combine both, the output is nothing. Trying to get logs for a container name with the word status in the message field.

There is hl example.log -f component=tsdb and its negation hl example.log -f component!=tsdb.

Then there is hl example.log -f provider~=string, but its negation doesn't work hl example.log -f provider!~=string.

Thank you.

Thanks for hl, it's a useful tool!

I'm dealing with a rather unfortunate log format where the message field is not located at the top level of the JSON object, but in a nested field. Luckily, hl supports reading the message from a nested field, but I found that it duplicates the message in the output while doing so. This does not happen if the message is in a top level field.

Case 1 (good)

Contents of good.json:

{"ts": "1711402897", "level": "info", "message": "test message", "field1": "value1"}

Command: hl good.json

Output:

Mar 25 21:41:37.000 |INF| test message field1=value1

This is what I'd expect to happen. The message is not shown as a field but is displayed right after the log level.

Case 2 (unexpected)

Contents of bad.json:

{"ts": "1711402897", "level": "info", "message": {"text": "test message", "nested1": "value2"}, "field1": "value1"}

Contents of config.yaml:

fields:
  predefined:
    message:
      names: [message.text]

Command: HL_CONFIG=config.yaml hl bad.json

Output:

Mar 25 21:41:37.000 |INF| test message message={ text=test message nested1=value2 } field1=value1

In this case, the message is correctly displayed after the log level, but I would not have expected it to also still be displayed in the fields.

I'm not sure if this is intended behavior, and if it isn't, whether it's even desirable to change this to avoid the duplication.

Predefined field names like message, logger, and caller should be case-insensitive in filters.

I noticed if I put the timezone in EST and downloaded recent logs (within a couple of minutes). The hour is off by 1 because EST is on DST currently right now. Can this be corrected?

When configuring Kubernetes and components thereof to log in json format the standard is to put a timestamp in the field ts as float signifying milliseconds since epoch. For example 1710858913.7978325 for Tue Mar 19 14:35:13 2024 (UTC).

Can you make hl parse this and output a human readable time string? If yes, how?

Here is the responsible line of code: https://github.com/kubernetes/component-base/blob/9d90bf7eaa57df6d6e0dd316529274b3c5747f5f/logs/json/json.go#L71

Thank you for #154 , which improved parsing of kubernetes json logs: https://kubernetes.io/docs/concepts/cluster-administration/system-logs/#json-log-format

The remaining thing that would be nice to have addressed is level. As you can see above it is handled a bit differently.

Info

The presence of the field v means info. But a higher number is reasonable to interpret as debug.

So a variant for the level configuration could look like:

    - names: [v]
      values:
        debug: [2, 3, 4]
        info: [0,1]

Except that it doesn't work. Going back to the example I gave in #153:

{"ts":1710923450736.8257,"caller":"eligibility/eligibility.go:104","msg":"Scale-down calculation: ignoring 2 nodes unremovable in the last 5m0s\n","v":1}

For this with the configuration above hl still show (?) as level. But if I put the value of v in quotes, making it a string, it does work. I.e. with this example hl shows the log level as info:

{"ts":1710923450736.8257,"caller":"eligibility/eligibility.go:104","msg":"Scale-down calculation: ignoring 2 nodes unremovable in the last 5m0s\n","v":"1"}

So I propose support for converting field value and values in the configuration to a common type before comparison.

Error

An error in these logs is signified by the presence of the field err, the content of which you would also like to see. As far as I can see this is not possible to configure at the moment. So I propose a new feature to make the presence of a field indicate a level. Maybe the configuration could look like this:

    - names: [err]
      values:
        error: exists 

I would love to be able to use hl for logfmt logs.
Any chance this could be added as supported format?
For me, that's the only reason i keep using humanlog

Hi

It would be very useful if hl can merge logs based on time.
Those logs might comes from different component in a big system. In order to trace issues, having a time ordered a logs is very useful. Hopefully it can be supported in this tool

best regard

Some programs writes logs with custom field names. In my case, datetime field instead of time and level_name instead of level:

{"datetime":"2021-05-21T02:53:11+03:00","level_name":"info","message":"Rabbitmq connection created"}

And the hl output:

        ---         |(?)| Rabbitmq connection created datetime='2021-05-21T02:53:11+03:00' level-name='info'

Hi, this little change (or similar) would make our life easier. Thank you for your work!

diff --git a/src/model.rs b/src/model.rs
index d209177..2a6fc94 100644
--- a/src/model.rs
+++ b/src/model.rs
@@ -159,7 +159,7 @@ impl<'de: 'a, 'a> Visitor<'de> for RecordVisitor<'a> {
                 "__REALTIME_TIMESTAMP" => {
                     rts = rts.or(Some(access.next_value()?));
                 }
-                "msg" | "MESSAGE" | "Message" => {
+                "msg" | "MESSAGE" | "Message" | "message" => {
                     message = access.next_value()?;
                 }
                 "level" | "LEVEL" | "Level" => {

$ echo '{"filename": "whtvr", "lineno": 1}' | hl -P
        ---         |(?)| @ whtvr:

$ echo '{"filename": "whtvr", "lineno": "1"}' | hl -P
        ---         |(?)| @ whtvr:1

I tried making a PR but had to change the second type of FileLine to String, which doesn't look like it's the best way. First time diving into a rust codebase.

How does this compare to https://github.com/brocode/fblog ?

The performance section mentions a log file used for testing, can you provide a link to that or what you used to generate it?

I'd like to try it out with lnav to see how it performs and if there is anything I can improve.

Thanks

Hello,

First, thanks for the great work!

While trying to tweak the config file, I realized that the info level does not parse INF correctly. Lowercase inf works fine. You can try with the attached files.

I'm on Windows with v0.29.5.

Thanks,
Laurent
example.log
example-hl-config.yaml.txt

It would be useful if hl had support for logs following the ECS standard:

https://www.elastic.co/guide/en/ecs/8.10/ecs-reference.html

Especially the general log related fields:

https://www.elastic.co/guide/en/ecs/8.10/ecs-log.html

As an example here is log line from a python application using the ecs_logging library (filtered through jq for readability):

{
  "@timestamp": "2023-11-02T08:09:10.423Z",
  "log.level": "info",
  "message": "backup for database bar in foo is already done today",
  "application": "backup-rds",
  "ecs": {
    "version": "1.6.0"
  },
  "language": "python",
  "log": {
    "logger": "__main__",
    "origin": {
      "file": {
        "line": 107,
        "name": "app.py"
      },
      "function": "is_backup_done"
    },
    "original": "backup for database bar in foo is already done today"
  },
  "process": {
    "name": "MainProcess",
    "pid": 1,
    "thread": {
      "id": 140610736142144,
      "name": "MainThread"
    }
  }
}

Adding support for log.level is just a matter of adding the field name log.level to fileds.predefined.logger.names in config.yaml. But I have also tried to add caller information, but as far as I can see it is not possible to specify nested fields as predefined fields.

I'm using version 0.20.0-beta.14.9 of hl.

Below is an example format from our service, which logs to syslog and got some prefix:

Dec 12 02:20:57 abc.def myserver[12606]: {"level":"info","action":"LOGIN","clientId":"myserver:v1.2.3","deviceId":"abcd", "msg":"login succeeded"}

It would be nice if hl can leave the prefix part as is and format the json part, like what jl does

With `jl':

echo 'Dec 12 02:20:57 abc.def myserver[12606]: {"level":"info","action":"LOGIN","clientId":"myserver:v1.2.3","deviceId":"abcd", "msg":"login succeeded"}' | jl
Dec 12 02:20:57 abc.def myserver[12606]:    INFO: login succeeded [action=LOGIN clientId=myserver:v1.2.3 deviceId=abcd]

With hl:

echo 'Dec 12 02:20:57 abc.def myserver[12606]: {"level":"info","action":"LOGIN","clientId":"myserver:v1.2.3","deviceId":"abcd", "msg":"login succeeded"}' | hl -P
Dec 12 02:20:57 abc.def myserver[12606]: {"level":"info","action":"LOGIN","clientId":"myserver:v1.2.3","deviceId":"abcd", "msg":"login succeeded"}