puentesarrin / mongodb-log

MongoDB Python logging handler, Centralized logging made simple using MongoDB.

MongoLog: Centralized logging made simple using MongoDB

Info

MongoDB python logging handler. Python centralized logging made easy.

Author

Andrei Savu

Maintainer

Jorge Puente Sarrín

Setup

Before using this handler for logging you will need to create a capped collection on the MongoDB server.

You can do this using the following commands in the MongoDB shell:

> use mongolog
> db.createCollection('log', {capped:true, size:100000})

... and you are ready. Running stats() function on log collection should show something like this:

> db.log.stats()
{
        "ns" : "mongolog.log",
        "count" : 0,
        "size" : 0,
        "storageSize" : 102400,
        "numExtents" : 1,
        "nindexes" : 1,
        "lastExtentSize" : 102400,
        "paddingFactor" : 1,
        "systemFlags" : 1,
        "userFlags" : 0,
        "totalIndexSize" : 8176,
        "indexSizes" : {
                "_id_" : 8176
        },
        "capped" : true,
        "max" : 2147483647,
        "ok" : 1
}

Usage

>>> import logging >>> from mongolog.handlers import MongoHandler >>> >>> log = logging.getLogger('demo') >>> log.setLevel(logging.DEBUG) >>> log.addHandler(MongoHandler.to(db='mongolog', collection='log')) >>> >>> log.debug('Some message')

Check the samples folder for more details.

Why centralized logging?

• Easy troubleshouting:

  • Having the answers to why? quickly and accurately.
  • For troubleshouting while the system is down.
  • Removed risk of loss of log information.

• Resource tracking.
• Security.

What is MongoDB?

"MongoDB is a high-performance, open source, schema-free document-oriented database."

It can eficiently store arbitrary JSON objects. You can read more at MongoDB website.

Why MongoDB is great for logging?

• MongoDB inserts can be done asynchronously.
• Old log data automatically LRU's out thanks to capped collections.
• It's fast enough for the problem.
• Document-oriented / JSON is a great format for log information.

Read more about this subject on the MongoDB blog.

Have fun!

mongodb-log's Issues

difference between this and log4mongo-python

May I know anyone knows the difference between this project and https://github.com/log4mongo/log4mongo-python?

Seems both are some logging handler.

Travis integration

The unit tests are running perfectly. So, it's better to work in usage Travis-CI.

Update Readme file

Could you update the Read.me file for the usage of this module and how could I integrate with the mongo process running on the machine
the old repo doesnt have the documentation as well
https://github.com/andreisavu/mongodb-log

NameError: name 'unicode' is not defined

'unicode' not defined in Python3. Instead str should be used to support Python3.

Custom fields

Hi

Thanks for creating this package, I have a question, How can I add some custom fields in logs?

mongodb4.0+ error

mongodb4
Database.authenticate and Database.logout are removed

error like this:
'Collection' object is not callable. If you meant to call the 'authenticate' method on a 'Database' object it is failing because no such method exists.

Unexpected pymongo warning

My code:

logger.addHandler(MongoHandler.to(collection="log",
    host="<url>",port=<port>,
password="<password>",username="<username>",db="<db>"))

Warning:

/env2/lib/python3.3/site-packages/pymongo/mongo_client.py:343: UserWarning: database name or authSource in URI is being ignored. If you wish to authenticate to <db>, you must provide a username and password.
  "must provide a username and password." % (db_name,))