Error L10 - No new line after msg drop log about logplex

[ferd]

The logplex logs are length-prefixed.

If you take the first 142 (and the space), the next 142 bytes give you the string:

<172>1 2014-11-11T08:24:25+00:00 host heroku logplex - Error L10 (output buffer overflow): 1 messages dropped since 2014-11-07T23:29:45+00:00.

What you have left is therefore

93 <45>1 2014-11-11T08:24:24.064853+00:00 host heroku web.1 - State changed from starting to up

Which can be parsed by using 93 bytes of the content (starting at the first <), so you get:

<45>1 2014-11-11T08:24:24.064853+00:00 host heroku web.1 - State changed from starting to up

Which is the correct log message. The format is detailed in RFC 6587 - Transmission of Syslog Messages over TCP, specifically section 3.4.1, which describes the addition of the message size followed by a space preceding the actual RFC 5424 message. This gives the overall log message the form:

MSG-LEN SP PRI VERSION SP TIMESTAMP SP HOSTNAME SP APP-NAME SP PROCID SP MSGID STRUCTURED-DATA [SP MSG]

The end of lines are not mandated nor required by the format, and are only accidentally present when the original log message did include them in some way. In the case of L10 messages, they are injected in people's log streams without that trailing line-break.

Do note that your log parser should also avoid relaying on line-breaks given the Syslog-over-TCP mechanism also means that a single log message may include line breaks -- length delimits the message. If you parse on line-breaks, you would then risk breaking up messages that shouldn't have been so.

[bergold]

Thank you for your really helpful reply. I think I can close this.