`PatternResolver` does not support substitution about logging-log4j2

[vy]

@ngocnhan-tran1996, okay. I am closing the issue then.

Note that StrSubstitutor bug will be addressed in the newly created #2679.

[vy]

@ngocnhan-tran1996, thanks for the report, I've submitted #2667 fixing the issue. I would really appreciate it if you can

1. Checkout the fix/2.x/JTL-prop-subs branch
2. Quickly build it using ./mvnw install -DskipTests – see BUILDING.adoc for details
3. And test the installed 2.24.0-SNAPSHOT artifacts in your project

[ngocnhan-tran1996]

@ngocnhan-tran1996, thanks for the report, I've submitted #2667 fixing the issue. I would really appreciate it if you can

1. Checkout the fix/2.x/JTL-prop-subs branch
2. Quickly build it using ./mvnw install -DskipTests – see BUILDING.adoc for details
3. And test the installed 2.24.0-SNAPSHOT artifacts in your project

@vy

Thanks for the answer, I have test on cases with below EcsLayout.json and log4j2.xml. I type wrong timezone value, GMT instead UTC.

File EcsLayout.json

{
  "message1": {
    "$resolver": "pattern",
    "pattern": "${env:TZ:-%d{yyyy-MM-dd HH:mm:ss}{GMT+00}}"
  },
  "message2": {
    "$resolver": "pattern",
    "pattern": "${env:TZ:-${TIMEZONE}}"
  },
  "message3": {
    "$resolver": "pattern",
    "pattern": "${env:TZ:-%d{yyyy-MM-dd HH:mm:ss}}"
  }
}

File log4j2.xml

<Properties>
    <Property name="TIMEZONE">%d{yyyy-MM-dd HH:mm:ss}{GMT+03}</Property>
</Properties>

1. Environment TZ does not exists

The result

{"message1":"2024-06-17 21:49:18,263","message2":"2024-06-17 17:49:18","message3":"2024-06-17 21:49:18"}

message2 and message3 work as my expected

2. Environment TZ do exists

The result

{"message1":"2024-06-17 14:53}","message2":"2024-06-17 21:53","message3":"2024-06-17 21:53}"}

message2 works as my expected

I dont know why message1 and message3 have brace } at the end

Note: GMT in my local is setting to +07:00

File test in here
https://github.com/ngocnhan-tran1996/log4j2-demo

[vy]

@ngocnhan-tran1996, I collaborated on this issue with @ppkarwasz, and I decided to not update JTL (JSON Template Layout) to perform property substitution on resolver configuration objects. Yet the issue you reported about the trailing } is indeed a StrSubstitutor bug, which I will address in another ticket.

JTL and property substitution

JTL supports property substitution on string members of a JSON object, granted they are not part of a resolver configuration. Let me make this clear with an example:

{
   "x": "${bar}",
   "y": {
     "$resolver": "...",
     "z": "${bar}"
   }
}

${bar} for x will be substituted at configuration-time (i.e., while compiling JTL templates), whereas ${bar} of z will not be substituted, since it is part of a template configuration object. In #2667, I tried various approaches to improve the situation, but it always caused other issues.

It is important to understand property substitution gets performed at three levels in your example:

• For literals in log4j2.xml (at configuration-time)
• For literals in a JTL template (at configuration-time)
• By Pattern Layout wired by pattern resolver (at runtime)

When you use ${env:TZ:-${TIMEZONE}}, you actually want this to happen at configuration-time, since

1. For performance reasons, it better be executed once instead of for every log event
2. It accesses to a configuration property: TIMEZONE

Hence, it doesn't make sense to access to a Log4j configuration property (TIMEZONE) from an external JSON file, i.e., event template provided by the eventTemplateUri attribute. That said, as I tried to explain, if you would have inlined your template into log4j2.xml using the eventTemplate attribute, ${env:TZ:-${TIMEZONE}} would have just worked fine.

I advise you to inline your "JTL template fields accessing to configuration properties" using event template additional fields:

<JsonTemplateLayout eventTemplateUri="classpath:MyLayout.json">
  <EventTemplateAdditionalField
      key="instant"
      format="JSON"
      value='{"$resolver": "pattern", "pattern": "${env:TZ:-%d{yyyy-MM-dd HH:mm:ss}}"}'/>
</JsonTemplateLayout>

All ${foo} in the above example will be subject to property substitution at configuration-time.

I will repurpose #2667 to improve JTL docs on this subject.

Notes on time zone

It is also peculiar that you need to change the time zone in your application depending on an environment variable. Would you mind sharing a little bit more detail about your use case, please? Where do you eventually persist/forward these logs? Why not sticking to UTC doesn't work for you?

Notes on trailing }

You stated

I dont know why message1 and message3 have brace } at the end

It is a StrSubstitutor bug, you can also reproduce it with PatternLayout too:

<PatternLayout pattern="${env:TZ:-%d{yyyy-MM-dd HH:mm:ss}{GMT+00}}"/>

I will create a separate issue for this.

Conclusion

In short,

• We don't think JTL should be fixed
• I will update JTL docs on this subject
• You can/should use EventTemplateAdditionalFields
• Trailing } is a bug, which I will address in another ticket/PR

[rgoers]

What I find peculiar about this example is that the reporter has the default value for the time zone as a time value. Normally I would expect something like "America/.New York".

[ngocnhan-tran1996]

@vy

About time zone

Sorry, this is my bad approach for configuring time zone.

I have difference timezone bewteen local and server, and my expect is application's console will print timezone lịke my local.
And I added config timezone like %d{yyyy-MM-dd HH:mm:ss}{GMT+03}
But I realize I were shooting myself in the foot. I just only config timezone in application instead.

About JTL and property substitution

Thanks for the advice, I will try it in my project.

I am looking forward to the log4j2 next version