logfmt (second article) is a format meant to be between JSON and normal logs for (human) readability and (machine) parse-ability.

Any chance it could be added to pygogo? I might have a go myself, but thought I'd ask at least.

The current version in PyPI (0.10.0) lacks some important patches such as 0feab9e, 910fa6e or the not-yet-merged 530d82d which makes a simple 'pip install' of this package or anything else (such as https://github.com/reubano/csv2ofx) impossible. A manual workaround is possible but it's a hassle.

Hi, i'm using Pygogo to log my project but in the wild the pygogo have some weird behavior, like this:
{"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"} {"time": "2018-01-22 10:59:47.279", "name": "json.base", "level": "INFO", "message": "Connected to:PRDSLMAI"}

It all has the same time. So I think this is the wrong behavior, and this is my function that calls pygogo:
def get_logger(): file = utils.open_file_from_project_root('resources\\log_file.json') log_json = json.loads(file) log_path = os.path.join(log_json['path'], log_json['filename']) return gogo.Gogo('json', low_hdlr=gogo.handlers.file_hdlr(log_path), low_formatter=gogo.formatters.json_formatter, high_level='error', high_formatter=gogo.formatters.json_formatter)\ .get_logger()

What should I do?

Traceback (most recent call last):
  File "testscript.py", line 9, in <module>
    import pygogo as gogo
  File ".../virtualenv/lib/python2.7/site-packages/pygogo/__init__.py", line 52, in <module>
    from builtins import *
ImportError: No module named builtins

Say what?

Include some benchmarks to show the improvement gained by different implementations of #1

I think below stdout.log should changed to stdout
Thanks for the package

Disabled dual logging

import pygogo as gogo

logger = gogo.Gogo(monolog=True).logger
logger.debug('debug message')
logger.info('info message')
logger.warning('warning message')
logger.error('error message')
logger.critical('critical message')

Prints the following to stdout.log (all messages at level INFO or below):

Does this library have loggly (https://www.loggly.com/) support ?

• factor out the builtin logging calls to create a pluggable backend (via #1)
• add twisted backend
• add tornado backend
• add asyncio backend

Testing 1.3.0, running tests fails like this:

$ PYTHONPATH=/sw/build.build/pygogo-py38-1.3.0-1/pygogo-1.3.0/build/lib /sw/bin/python3.8 tests/test.py
Script result: /sw/build.build/pygogo-py38-1.3.0-1/pygogo-1.3.0/bin/gogo --help
  return code: 1
-- stderr: --------------------
Traceback (most recent call last):
  File "/sw/build.build/pygogo-py38-1.3.0-1/pygogo-1.3.0/bin/gogo", line 13, in <module>
    from pygogo import main
  File "/sw/build.build/pygogo-py38-1.3.0-1/pygogo-1.3.0/build/lib/pygogo/__init__.py", line 49, in <module>
    from . import formatters, handlers, utils
  File "/sw/build.build/pygogo-py38-1.3.0-1/pygogo-1.3.0/build/lib/pygogo/formatters.py", line 405
    logging.DEBUG: f"{debug_color} {self._fmt} {RESET}",
                                                      ^
SyntaxError: invalid syntax

Traceback (most recent call last):
  File "tests/test.py", line 98, in <module>
    main(script, tests)
  File "tests/test.py", line 44, in main
    result = env.run(command, cwd=p.abspath(p.dirname(p.dirname(__file__))))
  File "/sw/lib/python3.8/site-packages/scripttest.py", line 273, in run
    result.assert_no_error(quiet)
  File "/sw/lib/python3.8/site-packages/scripttest.py", line 426, in assert_no_error
    raise AssertionError(
AssertionError: Script returned code: 1

/sw/bin/nosetests3.8 -xv finished fine with no errors in 57 tests. This failure happens with py3.8, 3.9, and 3.10. This is macOS 10.14.5, but I don't think this would matter given the error type.

the built-in logging module isn't broken so don't reinvent the wheel

Hmm.. Do you have any performance benchmarks that prove that logging is really isn't broken in terms of performance?

It logging wasn't broken then I suppose there won't be such things as https://twistedmatrix.com/documents/15.2.1/core/howto/logger.html

Which also says that:

logging is a blocking API, and logging can be configured to block for long periods (eg. it may write to the network). No protection is provided to prevent blocking

As for don't reinvent the wheel here is a good example of "logging with superpowers". https://eliot.readthedocs.org/en/0.11.0/index.html =)

A welcome feature would be built-in log rotation for the file handler, enabled via kwarg for max file size, and maybe another kwarg for the number of old/rotated logs to keep around.

Would you consider that in scope for this project?

Is pygogo support log-rotation?
One single file is huge for my case.

9dccca6

>>> gogo.Gogo(name="root", low_formatter=gogo.formatters.csv_formatter).info('..."hmm"')
# stdout:
# 2019-02-23 10:00:02.825,root.base,INFO,"..."hmm""

It should use a real csv formatter

Hi 👊

This is my first visit to this fine repo, but it seems you have been working hard to keep all dependencies updated so far.

Once you have closed this issue, I'll create seperate pull requests for every update as soon as I find one.

That's it for now!

Happy merging! 🤖

Hi! I've recently become more interested in structured logging, and have looked into a few structured logging libraries.

You get amazing power when you dump the logs from all of your different systems and sources into a centralized log store, and can then view and analyze them as one whole.

What I've noticed though is that the various structured logging frameworks all save JSON log entries in similar, but slightly different schemas.

For example, for storing timestamps, this library uses the JSON key "time", while other libraries use "timestamp" or "at". Another area where libraries differ is in how they store log levels.

These small differences cause friction when analyzing the central logstore, which contains structured logs that have been collected from multiple systems/microservices.

For example, one service might tag warnings with the string "WARN" while another with the string "warning". So if I want to view only warnings, I need to take this difference into account and write a tricky "OR" filter expression. This may seem minor, but these small inconsistencies cause great pain.

I believe that these small differences between the various structured logging libraries exist not because of any strongly held opinions, but simply because mainstream structured and centralized logging is still relatively young, and so there is no standard or common consensus.

I think it would be very beneficial to everyone if we could all unite around a common format.

To get things started, I have created a GitHub repository to centralize discussions here: https://github.com/bitc/structured-logging-schema

It contains a comparison between several structured logging libraries, summarizing the differences between them all. This can hopefully be a start to help us arrive at common ground.

I encourage the authors of this library (and anyone else who has an opinion) to participate in the discussion!

Discussion takes place in the issues for this repo: https://github.com/bitc/structured-logging-schema