Broken timestamp since Log4J 2.25

[fabian-braun]

Description

We sometimes encounter broken timestamp fields in our logs since we upgraded to the latest Log4J version which has been released a few days ago.

The logged timestamp is cut off and cannot be parsed by our ElasticSearch backend. For example it looks like 2025-06-28T10:20: (note how seconds, etc are missing)

I suspect the culprit is your switch to Java’s standard DateTimeFormatter in the latest release

Configuration

Version: 2.25.0

Operating system: Linux, 6.8.0-1024-aws

JDK: amazoncorretto:21-al2023

Reproduction

We cannot reproduce the behavior in a unit tests. It only happens roughly 20 times per day, out of more than 10k total log messages

We are using JsonTemplateLayout with the following config (excerpt):

{
  "@timestamp": {
    "$resolver": "timestamp",
    "pattern": {
      "format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
      "timeZone": "UTC"
    }
  },

[ppkarwasz]

@fabian-braun,

Thank you for bringing this to our attention! We’ll investigate further. Please note that formatters are recalculated every minute and cached. Could you clarify whether all the log events at 2025-06-28T10:20 are missing the end, or is it just a subset of them?

[fabian-braun]

@fabian-braun,

Thank you for bringing this to our attention! We’ll investigate further. Please note that formatters are recalculated every minute and cached. Could you clarify whether all the log events at 2025-06-28T10:20 are missing the end, or is it just a subset of them?

@ppkarwasz , it is only a subset. I see log events on the same pod, for example with a timestamp of 2025-06-28T10:20:49.554Z and 2025-06-28T10:20:01.173Z

we now enabled the legacy formatters via the ENV var. I will let you know tomorrow if that fixed the issue (would be another indicator that it's related to the DateTimeFormat change)

[fabian-braun]

These are all the times for which we did observe log events in that same minute on the same pod

scratch_3.txt

[fabian-braun]

I now took a look at all the failed log attempts on that day. We have 14 broken log events.

5 out of 14 have the broken timestamp with missing seconds-part.
The other 9 are missing the timestamp completely

[fabian-braun]

Since we re-deployed the app with LOG4J_INSTANT_FORMATTER=legacy the problem is gone. In the last 24 hours since the deployment all logs could be parsed. So, I'm quite convinced now that the problem is indeed related to the latest Log4J release

[fabian-braun]

Before switching back to the legacy format, I also observed a case now, where the sub second part is missing:
"@timestamp": "2025-07-01T04:24:02.",