logging在一个工厂模式的flask中使用的demo。实现通过flask自带的logger和自定义的logger来记录日志。
-
安装依赖。
pip3 install -r requirements.txt
-
启动LogServer。
python3 LogServer.py
-
启动flask
python3 manager.py runserver
-
使用flask的logger记录日志:
- view模块访问:http://localhost:5000/test1
- function模块:http://localhost:5000/test2
-
使用自定义的logger记录日志:
- view模块访问:http://localhost:5000/module/test1
- function模块:http://localhost:5000/module/test2
-
在
LogServer
的console
中查看日志输出。
-
Python
的logging
模块先看下对于
logging
模块的官方介绍Loggers have the following attributes and methods. Note that Loggers are never instantiated directly, but always through the module-level function
logging.getLogger(name)
. Multiple calls togetLogger()
with the same name will always return a reference to the same Logger object.The
name
is potentially a period-separated hierarchical value, likefoo.bar.baz
(though it could also be just plainfoo
, for example). Loggers that are further down in the hierarchical list are children of loggers higher up in the list. For example, given a logger with a name offoo
, loggers with names offoo.bar
,foo.bar.baz
, andfoo.bam
are all descendants offoo
. The logger name hierarchy is analogous to the Python package hierarchy, and identical to it if you organise your loggers on a per-module basis using the recommended constructionlogging.getLogger(__name__)
. That’s because in a module,__name__
is the module’s name in the Python package namespace.https://docs.python.org/3/library/logging.html#logger-objects
上面主要告诉我们两点,
- 可以通过
logging.getLogger(name)
来获取一个logger
,相同名字的logger
,其实是同一个logger
。 logger
是通过name
进行继承的,比如foo.bar
就是foo
的子logger
。就可以是实现我们通过配置一个rootLogger
,然后直接使用rootLogger.sublogger
来记录一下内容,而不需要单独再配置一遍。- 当使用
logging.getLogger(__name__)
时,__name__
就是这个模块所在的python package
的namespace
。
- 可以通过
-
flask提供的logger
再看下flask中的logging模块:
Flask uses standard Python
logging
. All Flask-related messages are logged under the'flask'
logger namespace.Flask.logger
returns the logger named'flask.app'
, and can be used to log messages for your application.Depending on the situation, an extension may choose to log to
app.logger
or its own named logger. Consult each extension’s documentation for details.我们可以知道flask的logger就是一个标准的Python logging,它的命名是
flask
。我们既可以使用app.logger
,也可以自己定义一个logger
。那么如何使用
app.logger
呢?有两种方式:
-
直接调用
logger = logging.getLogger('flask.app') logger.info('flask.app')
-
使用
Flask
提供的接口from flask import current_app current_app.logger.info('logged by current_app from main')
这里推荐还是使用第二种,
current_app
是一个单例,可以直接引用到app.logger
。 -
-
通过修改
app.logger
的name
,可以实现子logger
的继承么?答案是否定的。
-
修改
app.logger
的name
:# app/__init__.py app.logger.name = 'app'
然后在子模块中定义一个
app.module
的logger
来记录:from flask import current_app import logging logger = logging.getLogger('app.module') @module.route('/test', methods=['GET']) def test(): logger.info('logged by app.module') current_app.logger.info('logged by current_app.logger')
输出结果:
2019-02-01 10:56:01,877 - Thread-2 - app - INFO - logged by current_app.logger
只有
current_app.logger
的输出。 -
修改
app.logger
的name
是不是无效呢?我们把子模块中的
logger
的name
修改为flask.app.module
:from flask import current_app import logging logger = logging.getLogger('flask.app.module') @module.route('/test', methods=['GET']) def test(): logger.info('logged by flask.app.module') current_app.logger.info('logged by current_app.logger')
输出结果:
2019-02-01 11:00:10,944 - Thread-2 - flask.app.module - INFO - logged by flask.app.module 2019-02-01 11:00:10,946 - Thread-2 - app - INFO - logged by current_app.logger
两个
logger
均输出了。
可见,通过修改
app.logger.name
可以在记录的时候显示为我们设置的名称,但实际上这个logger
还是flask.app
。 -
-
__name__
的使用在自定义
logger
的情况下,为了方便起见,我们可以利用__name__
这个参数。前面说到:当使用
logging.getLogger(__name__)
时,__name__
就是这个模块所在的python package
的namespace
。一般
Flask
的工厂模式结构如下:app ├── __init__.py ├── main │ ├── __init__.py │ ├── functions.py │ └── views.py └── module ├── __init__.py ├── functions.py └── views.py
那么我们在先在
app.__init__
中定义rootLogger
,然后再在app.module.functions.py
中定义子Logger
,均使用logging.getLogger(__name__)
:# app.__init__.py 初始化rootlogger rootLogger = logging.getLogger(__name__) rootLogger.setLevel(logging.DEBUG) socketHandler = logging.handlers.SocketHandler('localhost',logging.handlers.DEFAULT_TCP_LOGGING_PORT) rootLogger.addHandler(socketHandler) rootLogger.setLevel(logging.DEBUG) # app.module.functions.py import logging logger = logging.getLogger(__name__) def record_from_logging(): logger.info('logged by logging from __name__')
输出:
2019-02-01 12:18:34,743 - MainThread - app - INFO - register root logger by __name__ 2019-02-01 12:19:24,954 - Thread-4 - app.module.functions - INFO - logged by logging from __name__
可以发现输出的
logger.name
就是所在的文件目录,logger
之间的继承关系与整个程序包保持一致。
根据上面分析,那么怎么优雅的记录logger
呢?
-
如果没有对模块进行分
logger
记录要求的话。可以直接使用在程序初始化的时候配置app.logger
(可以自行设置logger.name
)。参考main
模块。# app.__init__.py def register_logging(app): app.logger.name = 'app' # logstash_handler stashHandler = logstash.LogstashHandler('app.config.get('ELK_HOST')', 'app.config.get('ELK_PORT')') app.logger.addHandler(stashHandler) # socket_handler socketHandler = logging.handlers.SocketHandler('localhost', logging.handlers.DEFAULT_TCP_LOGGING_PORT) app.logger.addHandler(socketHandler) # app.module.function.py from flask import current_app @module.route('/test', methods=['GET']) def test(): current_app.logger.info('logging someting') return 'logged by current_app.logger'
输出效果:
2019-02-01 13:49:28,998 - Thread-2 - app - INFO - logged by current_app from main 2019-02-01 13:49:38,346 - Thread-3 - app - INFO - logged by current_app of functions
注意: 对于
current_app.logger
的引用不能通过如下方式,会有RuntimeError
的报错。from flask import current_app logger = current_app.logger ## 异常 raise RuntimeError(_app_ctx_err_msg) RuntimeError: Working outside of application context. This typically means that you attempted to use functionality that needed to interface with the current application object in some way. To solve this, set up an application context with app.app_context(). See the documentation for more information.
-
如果希望按自己的实际需求,对模块进行分
logger
记录要求的话。那么建议自己设置logger
。参考module
模块。# app.__init__.py def register_logging(): # set own root logger rootLogger = logging.getLogger(__name__) rootLogger.setLevel(logging.DEBUG) # socketHandler socketHandler = logging.handlers.SocketHandler('localhost',logging.handlers.DEFAULT_TCP_LOGGING_PORT) rootLogger.addHandler(socketHandler) # logstash_handler stashHandler = logstash.LogstashHandler('app.config.get('ELK_HOST')', 'app.config.get('ELK_PORT')') rootLogger.addHandler(stashHandler) rootLogger.setLevel(logging.DEBUG) # app.module.function.py import logging logger = logging.getLogger(__name__) @module.route('/test', methods=['GET']) def test(): logger.info('logging someting') return 'logged by logging module'
输出效果:
2019-02-01 13:49:49,297 - Thread-5 - app.module.views - INFO - logged by flask.app.module 2019-02-01 13:50:01,013 - Thread-7 - app.module.functions - INFO - logged by logging module of functions
关于python
中logging
的配置可参考官网:
https://docs.python.org/3/library/logging.config.html?highlight=logging
在配置handler
时,经常会希望日志可以按时间分割(TimedRotatingFileHandler)或者按大小分割(RotatingFileHandler).
但是在flask
项目中,尤其开启多线程之后,在分割日志(doRollover()
)时会有文件读写的异常:
WindowsError: [Error 32]
建议使用SocketHandler,将日志发送给单独的LogServer
来进行二次处理。
一个简单的接受SocketLog的Server参考LogServer.py
。
或者现在流行的stashHandler,将日志发送给ELK来进行二次处理。