bug: HTTPHandler doesn't respect formatter, only produces logs in key=val format about meltano

[edgarrmondragon]

Hey @BillyUdders, thanks for logging!

I think this is unfortunately expected from Python's builtin logging.handlers.HTTPHandler. From the docs¹:

Since preparing a record for sending it to a web server is not the same as a generic formatting operation, using setFormatter() to specify a Formatter for a HTTPHandler has no effect. Instead of calling format(), this handler calls mapLogRecord() and then urllib.parse.urlencode() to encode the dictionary in a form suitable for sending to a web server.

Perhaps you could try using a package like python-logstash-async². You would need to install it in the same venv as Meltano. If you're using pipx³, it's as simple as

pipx inject meltano python-logstash-async

Footnotes

1. Official docs for logging.handlers.HTTPHandler ↩

2. GitHub repo for python-logstash-async ↩

3. GitHub repo for pypa/pipx ↩

[edgarrmondragon]

Can confirm the above:

import logging
from logging.handlers import HTTPHandler
from urllib.parse import urlencode

record = logging.LogRecord("my_logger", logging.INFO, "/path/to/file.py", 10, "EVENT!", (), None)
handler = HTTPHandler("localhost", 5000)
print(urlencode(handler.mapLogRecord(record)))
# name=my_logger&msg=EVENT%21&args=%28%29&levelname=INFO&levelno=20&pathname=%2Fpath%2Fto%2Ffile.py&filename=file.py&module=file&exc_info=None&exc_text=None&stack_info=None&lineno=10&funcName=None&created=1706751680.052684&msecs=52.0&relativeCreated=8.797883987426758&thread=8648734912&threadName=MainThread&processName=MainProcess&process=39301&taskName=None

Closing, since this is not a bug and is not Meltano's doing.