Coder Social home page Coder Social logo

airbrake / pybrake Goto Github PK

View Code? Open in Web Editor NEW
36.0 14.0 19.0 512 KB

Python exception notifier for Airbrake

Home Page: https://airbrake.io

License: Other

Python 100.00%
python python3 airbrake notifier error-reporting logging crash crash-reporting monitoring

pybrake's Introduction

Python exception notifier for Airbrake

Build Status

Installation

pybrake requires Python 3.6+.

pip install -U pybrake

Configuration

You must set both project_id & project_key.

To find your project_id and project_key navigate to your project's Settings and copy the values from the right sidebar.

project-idkey

import pybrake

notifier = pybrake.Notifier(project_id=123,
                            project_key='FIXME',
                            environment='production')

Sending errors to Airbrake

try:
    raise ValueError('hello')
except Exception as err:
    notifier.notify(err)

Sending errors synchronously

By default, the notify function sends errors asynchronously using ThreadPoolExecutor and returns a concurrent.futures.Future, a synchronous API is also made available with the notify_sync function:

notice = notifier.notify_sync(err)
if 'id' in notice:
    print(notice['id'])
else:
    print(notice['error'])

Adding custom params

To set custom params you can build and send notice in separate steps:

notice = notifier.build_notice(err)
notice['params']['myparam'] = 'myvalue'
notifier.send_notice(notice)

You can also add custom params to every error notice before it's sent to Airbrake with the add_filter function.

def my_filter(notice):
    notice['params']['myparam'] = 'myvalue'
    return notice

notifier.add_filter(my_filter)

Ignoring notices

There may be some notices/errors thrown in your application that you're not interested in sending to Airbrake, you can ignore these using the add_filter function.

def my_filter(notice):
    if notice['context']['environment'] == 'development':
        # Ignore notices in development environment.
        return None
    return notice

notifier.add_filter(my_filter)

Filtering keys

With keys_blocklist option you can specify list of keys containing sensitive information that must be filtered out, e.g.:

notifier = pybrake.Notifier(
    ...
    keys_blocklist=[
        'password',           # exact match
        re.compile('secret'), # regexp match
    ],
)

Logging integration

pybrake provides a logging handler that sends your logs to Airbrake.

import logging
import pybrake


airbrake_handler = pybrake.LoggingHandler(notifier=notifier,
                                          level=logging.ERROR)

logger = logging.getLogger('test')
logger.addHandler(airbrake_handler)

logger.error('something bad happened')

Disabling pybrake logs

The pybrake logger can be silenced by setting the logging level to logging.CRITICAL.

import logging


logging.getLogger("pybrake").setLevel(logging.CRITICAL)

Sending route stats

notifier.routes.notify allows sending route stats to Airbrake. The library provides integrations with Django and Flask. (your routes are tracked automatically). You can also use this API manually:

from pybrake import RouteMetric

metric = RouteMetric(method=request.method, route=route)
metric.status_code = response.status_code
metric.content_type = response.headers.get("Content-Type")
metric.end_time = time.time()

notifier.routes.notify(metric)

Sending route breakdowns

notifier.routes.breakdowns.notify allows sending performance breakdown stats to Airbrake. You can use this API manually:

from pybrake import RouteMetric

metric = RouteMetric(
    method=request.method,
    route='/things/1',
    status_code=200,
    content_type=response.headers.get('Content-Type'))
metric._groups = {'db': 12.34, 'view': 56.78}
metric.end_time=time.time()

notifier.routes.breakdowns.notify(metric)

Sending query stats

notifier.queries.notify allows sending SQL query stats to Airbrake. The library provides integration with Django (your queries are tracked automatically). You can also use this API manually:

notifier.queries.notify(
    query="SELECT * FROM foos",
    method=request.method,
    route=route,
    function="test",
    file="test",
    line=10,
    start_time=time.time(),
    end_time=time.time(),
)

Sending queue stats

notifier.queues.notify allows sending queue (job) stats to Airbrake. The library provides integration with Celery (your queues are tracked automatically). You can also use this API manually:

from pybrake import QueueMetric

metric = QueueMetric(queue="foo_queue")
metric._groups = {'redis': 24.0, 'sql': 0.4}
notifier.queues.notify(metric)

Framework Integration

Pybrake provides a ready-to-use solution with minimal configuration for python frameworks.

Development

Running the tests

pip install -r requirements.txt
pip install -r test-requirements.txt
pytest

Uploading to PyPI

python setup.py sdist upload

Remote configuration

Every 10 minutes the notifier issues an HTTP GET request to fetch remote configuration. This might be undesirable while running tests. To suppress this HTTP call, you need to pass remote_config=False to the notifier.

pybrake's People

Contributors

andriyreznik avatar anmic avatar camuthig avatar dependabot[bot] avatar gzub avatar kyrylo avatar mattrasband avatar mmcdaris avatar momyc avatar phumpal avatar salonijain0918 avatar shifi avatar smurf-u avatar sturmianseq avatar tearoom6 avatar thompiler avatar vmihailenco avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pybrake's Issues

tdigest should be added to install_requires of setup.py

tdigest should be added to install_requires of setup.py.
Right now it's impossible to install pybrake from github unless tdigest is already installed.

Obtaining pybrake from git+https://github.com/airbrake/pybrake.git#egg=pybrake (from -r requirements.txt (line 22))
  Cloning https://github.com/airbrake/pybrake.git to ./src/pybrake
  Running setup.py (path:/var/app/src/pybrake/setup.py) egg_info for package pybrake
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/var/app/src/pybrake/setup.py", line 3, in <module>
        import pybrake
      File "/var/app/src/pybrake/pybrake/__init__.py", line 1, in <module>
        from .notifier import Notifier
      File "/var/app/src/pybrake/pybrake/notifier.py", line 14, in <module>
        from .routes import RouteStats
      File "/var/app/src/pybrake/pybrake/routes.py", line 7, in <module>
        from tdigest import TDigest
    ImportError: No module named 'tdigest'
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/var/app/src/pybrake/setup.py", line 3, in <module>

    import pybrake

  File "/var/app/src/pybrake/pybrake/__init__.py", line 1, in <module>

    from .notifier import Notifier

  File "/var/app/src/pybrake/pybrake/notifier.py", line 14, in <module>

    from .routes import RouteStats

  File "/var/app/src/pybrake/pybrake/routes.py", line 7, in <module>

    from tdigest import TDigest

ImportError: No module named 'tdigest'

pip install pybrake==0.4.0 fails when using the python:3.7-alpine Docker image

Collecting pybrake==0.4.0
  Downloading https://files.pythonhosted.org/packages/d8/a5/e04a19105bcf7a3cfb5fdc499c51d7794b5a28dd0c6f708396b8681cf61a/pybrake-0.4.0.tar.gz
Collecting tdigest (from pybrake==0.4.0)
  Downloading https://files.pythonhosted.org/packages/32/72/f420480118cbdd18eb761b9936f0a927957130659a638449575b4a4f0aa7/tdigest-0.5.2.2-py2.py3-none-any.whl
Collecting pyudorandom (from tdigest->pybrake==0.4.0)
  Downloading https://files.pythonhosted.org/packages/13/14/6fc20ea903eda547d6a255e995f8d4a09fdc3cf8bfacb6f85e6d669bc259/pyudorandom-1.0.0.tar.gz
Collecting accumulation-tree (from tdigest->pybrake==0.4.0)
  Downloading https://files.pythonhosted.org/packages/e9/18/73c11ed9d379b5efea5cabcce4b53762ee4b0c3aea42bd944e992f8ee307/accumulation_tree-0.6.tar.gz (81kB)
     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 92kB 17.0MB/s 
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-pjyl9ael/accumulation-tree/setup.py'"'"'; __file__='"'"'/tmp/pip-install-pjyl9ael/accumulation-tree/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
         cwd: /tmp/pip-install-pjyl9ael/accumulation-tree/
    Complete output (161 lines):
    Compiling /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Plex/Scanners.py because it changed.
    Compiling /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Plex/Actions.py because it changed.
    Compiling /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Compiler/Scanning.py because it changed.
    Compiling /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Compiler/Visitor.py because it changed.
    Compiling /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Compiler/FlowControl.py because it changed.
    Compiling /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Runtime/refnanny.pyx because it changed.
    Compiling /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Compiler/FusedNode.py because it changed.
    Compiling /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Tempita/_tempita.py because it changed.
    [1/8] Cythonizing /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Compiler/FlowControl.py
    [2/8] Cythonizing /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Compiler/FusedNode.py
    [3/8] Cythonizing /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Compiler/Scanning.py
    [4/8] Cythonizing /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Compiler/Visitor.py
    [5/8] Cythonizing /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Plex/Actions.py
    [6/8] Cythonizing /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Plex/Scanners.py
    [7/8] Cythonizing /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Runtime/refnanny.pyx
    [8/8] Cythonizing /tmp/easy_install-yeca46ke/Cython-0.29.13/Cython/Tempita/_tempita.py
    Unable to find pgen, not compiling formal grammar.
    warning: no files found matching 'Doc/*'
    warning: no files found matching '*.pyx' under directory 'Cython/Debugger/Tests'
    warning: no files found matching '*.pxd' under directory 'Cython/Debugger/Tests'
    warning: no files found matching '*.pxd' under directory 'Cython/Utility'
    warning: no files found matching 'pyximport/README'
    gcc: fatal error: Killed signal terminated program cc1
    compilation terminated.
    Traceback (most recent call last):
      File "/usr/local/lib/python3.7/distutils/unixccompiler.py", line 118, in _compile
        extra_postargs)
      File "/usr/local/lib/python3.7/distutils/ccompiler.py", line 909, in spawn
        spawn(cmd, dry_run=self.dry_run)
      File "/usr/local/lib/python3.7/distutils/spawn.py", line 36, in spawn
        _spawn_posix(cmd, search_path, dry_run=dry_run)
      File "/usr/local/lib/python3.7/distutils/spawn.py", line 159, in _spawn_posix
        % (cmd, exit_status))
    distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/usr/local/lib/python3.7/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/local/lib/python3.7/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/usr/local/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 172, in run
        cmd = self.call_command('install_lib', warn_dir=0)
      File "/usr/local/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 158, in call_command
        self.run_command(cmdname)
      File "/usr/local/lib/python3.7/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
        self.build()
      File "/usr/local/lib/python3.7/distutils/command/install_lib.py", line 107, in build
        self.run_command('build_ext')
      File "/usr/local/lib/python3.7/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 78, in run
        _build_ext.run(self)
      File "/usr/local/lib/python3.7/distutils/command/build_ext.py", line 340, in run
        self.build_extensions()
      File "/usr/local/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
        self._build_extensions_serial()
      File "/usr/local/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
        self.build_extension(ext)
      File "/usr/local/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 199, in build_extension
        _build_ext.build_extension(self, ext)
      File "/usr/local/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
        depends=ext.depends)
      File "/usr/local/lib/python3.7/distutils/ccompiler.py", line 574, in compile
        self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
      File "/usr/local/lib/python3.7/distutils/unixccompiler.py", line 120, in _compile
        raise CompileError(msg)
    distutils.errors.CompileError: command 'gcc' failed with exit status 1
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/usr/local/lib/python3.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
        yield saved
      File "/usr/local/lib/python3.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/usr/local/lib/python3.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
        _execfile(setup_script, ns)
      File "/usr/local/lib/python3.7/site-packages/setuptools/sandbox.py", line 45, in _execfile
        exec(code, globals, locals)
      File "/tmp/easy_install-yeca46ke/Cython-0.29.13/setup.py", line 285, in <module>
      File "/usr/local/lib/python3.7/site-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/usr/local/lib/python3.7/distutils/core.py", line 163, in setup
        raise SystemExit("error: " + str(msg))
    SystemExit: error: command 'gcc' failed with exit status 1
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/usr/local/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 1144, in run_setup
        run_setup(setup_script, args)
      File "/usr/local/lib/python3.7/site-packages/setuptools/sandbox.py", line 253, in run_setup
        raise
      File "/usr/local/lib/python3.7/contextlib.py", line 130, in __exit__
        self.gen.throw(type, value, traceback)
      File "/usr/local/lib/python3.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/usr/local/lib/python3.7/contextlib.py", line 130, in __exit__
        self.gen.throw(type, value, traceback)
      File "/usr/local/lib/python3.7/site-packages/setuptools/sandbox.py", line 166, in save_modules
        saved_exc.resume()
      File "/usr/local/lib/python3.7/site-packages/setuptools/sandbox.py", line 141, in resume
        six.reraise(type, exc, self._tb)
      File "/usr/local/lib/python3.7/site-packages/setuptools/_vendor/six.py", line 685, in reraise
        raise value.with_traceback(tb)
      File "/usr/local/lib/python3.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
        yield saved
      File "/usr/local/lib/python3.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/usr/local/lib/python3.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
        _execfile(setup_script, ns)
      File "/usr/local/lib/python3.7/site-packages/setuptools/sandbox.py", line 45, in _execfile
        exec(code, globals, locals)
      File "/tmp/easy_install-yeca46ke/Cython-0.29.13/setup.py", line 285, in <module>
      File "/usr/local/lib/python3.7/site-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/usr/local/lib/python3.7/distutils/core.py", line 163, in setup
        raise SystemExit("error: " + str(msg))
    SystemExit: error: command 'gcc' failed with exit status 1
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-pjyl9ael/accumulation-tree/setup.py", line 28, in <module>
        Extension('accumulation_tree.accumulation_tree', ['accumulation_tree/accumulation_tree.pyx'])
      File "/usr/local/lib/python3.7/site-packages/setuptools/__init__.py", line 144, in setup
        _install_setup_requires(attrs)
      File "/usr/local/lib/python3.7/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/usr/local/lib/python3.7/site-packages/setuptools/dist.py", line 717, in fetch_build_eggs
        replace_conflicting=True,
      File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 782, in resolve
        replace_conflicting=replace_conflicting
      File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1065, in best_match
        return self.obtain(req, installer)
      File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1077, in obtain
        return installer(requirement)
      File "/usr/local/lib/python3.7/site-packages/setuptools/dist.py", line 784, in fetch_build_egg
        return cmd.easy_install(req)
      File "/usr/local/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 679, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/usr/local/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 705, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/usr/local/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 890, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/usr/local/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 1158, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/usr/local/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 1146, in run_setup
        raise DistutilsError("Setup script exited with %s" % (v.args[0],))
    distutils.errors.DistutilsError: Setup script exited with error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

I could, but I'd prefer not to, add gcc to my build image.

v0.3.4 and python 3.7.0 compatibility

Lib/re.py is causing pybrake v0.3.4 to swallow errors in Python 3.7.0

      re.compile('password'),
      re.compile('secret'),

A quick fix seems to be addingg

โ€‹import re; re._pattern_type = re.Pattern

prior to compiling the regex object.

diff --git a/pybrake/notifier.py b/pybrake/notifier.py
index 4aab92e..67c4836 100644
--- a/pybrake/notifier.py
+++ b/pybrake/notifier.py
@@ -1,5 +1,5 @@
 import os
-import re
+import re; re._pattern_type = re.Pattern
 import sys
 import platform
 import socket
diff --git a/pybrake/test_notifier.py b/pybrake/test_notifier.py
index 113ec80..c63a6c2 100644
--- a/pybrake/test_notifier.py
+++ b/pybrake/test_notifier.py
@@ -1,4 +1,4 @@
-import re
+import re; re._pattern_type = re.Pattern
 from urllib.error import URLError

 from .notifier import Notifier

I suspect we can use the typing module to fix this in 3.5, 3.6 and 3.7 but not 3.4.

Test are still broken in 3.7.

Flask integration order of filters does not apply blocklist to request object

Hello, I'm noticing that with the flask integration, there is a filter added that adds the request object to the params section of the notice. That is great and contains useful info, like the JSON payload. However, that filter occurs after the make_blacklist_filter is applied, so the blocklist doesn't operate on the request object. And it turns out that request -> json contains much of the info we want to block.

So I am getting around that order of filters by redoing the blocklist filtering after calling init_app:

        app = pybrake.flask.init_app(app)
        notifier = app.extensions['pybrake']
        # calling the make_blacklist_filter again, even though it occurs during init_app,
        # so that the request info also gets filtered
        notifier.add_filter(pybrake.notifier.make_blacklist_filter(my_blocklist))

I don't have a fix in mind, but thought it should be mentioned.

Using pybrake version 0.4.6

Can't send an Airbrake notification from within a Django test?

As described in this StackOverflow question, I'm unable to send Airbrake notifications from a Django test, although I was able to send one from the Django shell.

In the end, I worked around the problem by turning logger.error into a Mock object. I wonder, however, why the send_notice method is not being called in the test case? Also, it might be useful for Airbrake to include some best practices for unit testing manually logged errors in its documentation.

log message arguments not merged as expected by Python's standard log interface

According to the official documentation for Python's logging module, the interface for logging should be as follows:

logging.debug(msg, *args, **kwargs)
Logs a message with level DEBUG on the root logger. The msg is the message format string, and the args are the arguments which are merged into msg using the string formatting operator.

Here's an example:

import logging
log = logging.getLogger(__name__)
log.error("This is my error: %s", 123)

One would expect the logged message to be "This is my error: 123."

Pybrake, however, does not respect this interface. The logged message is "This is my error: %s". The "args" are not merged into "msg" as per the standard interface.

After reviewing the source code, I'm fairly certain the problem is here:

message=record.msg,

This should be changed to

message=record.getMessage()

Using the standard library's getMessage() method.

Empty airbrake routes-stats response body makes pybrake log error every 5 seconds

We have deployed current master branch on our staging server and realized we are getting the following error logged every 5 seconds:

Expecting value: line 1 column 1 (char 0)
[SITE_PACKAGES]/pybrake/routes.py:121

It's caused by empty airbrake response body and JSON parser failing because of that. Should it just check if body is empty and not try to parse it instead?

Update routes API

  • incRequest -> notifyRequest
  • statusCode -> status_code
  • /api/v4/projects/{}/routes-stats -> /api/v5/projects/{}/routes-stats

setup.py requiring pytest-runner ?

Is there an actual reason for having this dependency in the setup.py file?

In my case, it prevents my application to build/install properly: pipenv/pip/setuptools attempts to install this dependency without using my PyPi mirror index (that is, apparently a "known" issue when using pipenv behind a proxy or using a mirror - pypa/pipenv#3430).

Sending notifications on exceptions at Celery tasks stopped after 0.4.5 upgrade

We use celery for monitoring celery tasks. Before the 0.4.5 upgrade, Airbrake was getting notified on Celery task exceptions. After the 0.4.5 this stopped and we can only manually notify Airbrake through notifier.notify(err)

the setup is the following

    notifier = pybrake.Notifier(project_id=settings.AIRBRAKE['project_id'],
                                project_key=settings.AIRBRAKE['project_key'],
                                environment="enviropment")
    patch_celery(notifier)

def add(x, y):
     raise ValueError("bad luck")

Altering system modules causes context building to fail

How to reproduce:

  1. Add pytest-xdist to your environment requirements.
  2. Run pytest

Details

I ran into an issue today while writing a test for my custom filter for the pybrake notifier. When calling the code notice = Notice(err), the _build_context fails with a traceback similar to the below.

Traceback (most recent call last):
  File "/Users/camuthig/.pyenv/versions/3.6.5/envs/pybrake/lib/python3.6/site-packages/celery/app/trace.py", line 385, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/Users/camuthig/projects/oss/pybrake/pybrake/celery.py", line 39, in wrapper
    notifier.notify(exc)
  File "/Users/camuthig/projects/oss/pybrake/pybrake/notifier.py", line 235, in notify
    notice = self.build_notice(err)
  File "/Users/camuthig/projects/oss/pybrake/pybrake/notifier.py", line 122, in build_notice
    context=self._build_context(),
  File "/Users/camuthig/projects/oss/pybrake/pybrake/notifier.py", line 349, in _build_context
    for name, mod in sys.modules.items():
RuntimeError: dictionary changed size during iteration

This issue appears related to this issue logged against the inspect module: https://bugs.python.org/issue13487 .

In this particular case, it appears something about how the xdist plugin is loaded for pytest is lazily evaluating imports and eventually execnet is imported, which runs another import in __init__, altering the modules dict.

I have been unable to reproduce this error outside of my own tests, and I was unable to force a recreation of it on this project directly without installing the xdist plugin.

notify in pybrake/routes.py: "TypeError: argument of type 'NoneType' is not iterable"

def notify(self, metric):
        if self._apm_disabled:
            return

        if self._stats is None:
            self._stats = {}
            self._thread = Timer(metrics.FLUSH_PERIOD, self._flush)
            self._thread.start()

        key = route_stat_key(
            method=metric.method,
            route=metric.route,
            status_code=metric.status_code,
            time=metric.start_time,
        )
        with self._lock:
            if key in self._stats:
                stat = self._stats[key]

At if key in self._stats: I'm getting:

TypeError: argument of type 'NoneType' is not iterable

How is this possible when self._stats is set to {} if None above? Thanks.

Add aiohttp support

Hello, guys!

I recently added airbrake logger to our aiohttp server as a middleware. So, I think it would be useful for others. Here is #54 PR.

Broken Flask sqlalchemy integration

pybrake version: 0.4.3

Issue 1

If flask-sqlalchemy is importable but not yet configured, flask.init_app will throw. We share a virtual env between multiple services and at least 1 of these doesn't setup flask-sqlalchemy and crashes.

There should be some docs around the flask-sqlalchemy integration and it shouldn't crash if flask-sqlalchemy is not set up.

Issue 2

SQLAlchemy(...).init_app breaks pybrake integration because flask-sqlalchemy can't get the app outside of app_context

def test_pybrakes_sqlalchemy_integration():
    from flask_sqlalchemy import SQLAlchemy

    app = Flask("test")
    app.config["PYBRAKE"] = ...
    app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///:memory"
    db = SQLAlchemy(app)
    app = pybrake.flask.init_app(app) # ok

def test_pybrakes_sqlalchemy_integration_init_app():
    from flask_sqlalchemy import SQLAlchemy

    app = Flask("test")
    app.config["PYBRAKE"] = ...
    app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///:memory"
    SQLAlchemy(...).init_app(app)
    app = pybrake.flask.init_app(app) # breaks because `flask-sqlalchemy` has no reference to `app` and not inside app_context

File "app/error_reporting/flask.py", line 53, in initialize_airbrake
app = pybrake.flask.init_app(app)

File ".cache/pypoetry/virtualenvs/app/lib/python3.6/site-packages/pybrake/flask.py", line 91, in init_app
_sqla_instrument(app)
File ".cache/pypoetry/virtualenvs/app/lib/python3.6/site-packages/pybrake/flask.py", line 143, in _sqla_instrument
engine = sqla.db.get_engine()
File ".cache/pypoetry/virtualenvs/app/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 946, in get_engine
app = self.get_app(app)
File ".cache/pypoetry/virtualenvs/app/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 982, in get_app
'No application found. Either work inside a view function or push'
RuntimeError: No application found. Either work inside a view function or push an application context. See http://flask-sqlalchemy.pocoo.org/contexts/."


No way to filter sensitive args or kwargs

Many companies may provide functions with sensitive data that they do not want to expose, there should be a way to filter/scrub/mask function arguments from a source that can throw an exception. In the docs, keys_blacklist could be misinterpreted to indicate this.

Notifier incorrectly returning errors

notifier running in k8s in gcp returns notifier.py:179 pybrake ERROR - airbrake: unexpected response status_code=500 sporadically. Even though the message arrives in Airbrake correctly.

Other erroneous messages from pybrake I see are

routes.py:133 pybrake ERROR - airbrake: unexpected response status_code=500
and
routes.py:119 pybrake ERROR - The read operation timed out

even on successful responses, when no notification is expected.

UnboundLocalError: local variable 'source' referenced before assignment

When using global excepthook exception catching and throwing an uncaught exception pybrake fails to send the error through to airbrake. The error occurs in code_hunks.py line 28 where source is undefined.

Python version: 3.6.4
Pybrake version: 0.3.2

import pybrake
import logging
import sys


def main():
    notifier = pybrake.Notifier(project_id=ID,
                                project_key='KEY',
                                environment='production')

    logger = logging.getLogger('test')
    logger.addHandler(
        pybrake.LoggingHandler(notifier=notifier, level=logging.ERROR)
    )

    def handle_exception(exc_type, exc_value, exc_traceback):
        logger.error("Uncaught exception", exc_info=(exc_type, exc_value, exc_traceback))

    sys.excepthook = handle_exception

    raise Exception('show-me-in-airbrake')


if __name__ == "__main__":
    main()

I can temporarily fix this by moving line 28/29 inside the try/except above.

Trace:

--- Logging error ---
Traceback (most recent call last):
File "/Users/cj/.local/share/virtualenvs/tmp/lib/python3.6/site-packages/pybrake/logging.py", line 38, in emit
notice = self.build_notice(record)
File "/Users/cj/.local/share/virtualenvs/tmp/lib/python3.6/site-packages/pybrake/logging.py", line 45, in build_notice
notice['errors'].append(self._build_error(record))
File "/Users/cj/.local/share/virtualenvs/tmp/lib/python3.6/site-packages/pybrake/logging.py", line 52, in _build_error
return self._build_error_from_exc_info(record.exc_info)
File "/Users/cj/.local/share/virtualenvs/tmp/lib/python3.6/site-packages/pybrake/logging.py", line 63, in _build_error_from_exc_info
backtrace = self._notifier._build_backtrace_tb(tb)
File "/Users/cj/.local/share/virtualenvs/tmp/lib/python3.6/site-packages/pybrake/notifier.py", line 237, in _build_backtrace_tb
f = self._build_frame(frame, lineno)
File "/Users/cj/.local/share/virtualenvs/tmp/lib/python3.6/site-packages/pybrake/notifier.py", line 274, in _build_frame
module_name=module_name)
File "/Users/cj/.local/share/virtualenvs/tmp/lib/python3.6/site-packages/pybrake/notifier.py", line 284, in _frame_with_code
lines = get_code_hunk(filename, line, loader=loader, module_name=module_name)
File "/Users/cj/.local/share/virtualenvs/tmp/lib/python3.6/site-packages/pybrake/code_hunks.py", line 6, in get_code_hunk
lines = _get_lines_from_file(filename, loader=loader, module_name=module_name)
File "/Users/cj/.local/share/virtualenvs/tmp/lib/python3.6/site-packages/pybrake/code_hunks.py", line 28, in _get_lines_from_file
if source is not None:
UnboundLocalError: local variable 'source' referenced before assignment

Loss of external internet connection causes runaway Pybrake process

We use pybrake in a Flask app running in a docker container that is intended to be deployed on local networks with external internet access, not in the cloud. We have chosen to configure it using the logging integration instead of the Flask integration. Recently, one of our team members discovered that if pybrake is unable to successfully send a notice to the airbrake.io server, it enters a loop that eventually uses up all local memory then crashes. It appears that failing to connect to its home server throws an exception, which is logged to Airbrake as an error, which fails to connect and throw an exception, ad infinitum. Is this a known bug or configuration option?

Version: up to and including 0.4.2

Very partial stdout log output:

2019-09-26 18:47:38,766 notifier.py:152 pybrake ERROR - <urlopen error [Errno -2] Name or service not known>
[2019-09-26 18:47:38,766] {notifier:152} ERROR - <urlopen error [Errno -2] Name or service not known>
2019-09-26 18:47:38,779 notifier.py:152 pybrake ERROR - <urlopen error [Errno -2] Name or service not known>
[2019-09-26 18:47:38,779] {notifier:152} ERROR - <urlopen error [Errno -2] Name or service not known>

get_git_revision failed error

We have the following errors in our log files - Our deployment packages do not include the git directory.

pybrake - ERROR - get_git_revision failed: [Errno 2] No such file or directory: '/var/task/.git/HEAD'

Is it possible suppress these errors so they do not show in our application logs?

Facing issues in pybrake==0.4.0

@vmihailenco I have been facing issue in pybrake version 0.4.0. It works perfectly while running locally but when I deployed my django application to AWS lambda via Zappa, pybrake throws error in application.

Below is the error snippet. It seems that there is an error while importing pybrake.
pybrake 0.3.5 works perfectly well on both scenarios. The newest version has some problem.

sk/handler.py", line 245, in lambda_handler
  handler = cls()
  File "/var/task/handler.py", line 151, in __init__
  wsgi_app_function = get_django_wsgi(self.settings.DJANGO_SETTINGS)
  File "/var/task/zappa/ext/django_zappa.py", line 20, in get_django_wsgi
  return get_wsgi_application()
  File "/var/task/django/core/wsgi.py", line 12, in get_wsgi_application
  django.setup(set_prefix=False)
  File "/var/task/django/__init__.py", line 19, in setup
  configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
  File "/var/task/django/conf/__init__.py", line 56, in __getattr__
  self._setup(name)
  File "/var/task/django/conf/__init__.py", line 43, in _setup
  self._wrapped = Settings(settings_module)
  File "/var/task/django/conf/__init__.py", line 106, in __init__
  mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/var/lang/lib/python3.6/importlib/__init__.py", line 126, in import_module
  return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/var/task/pos_authoriser/settings.py", line 15, in <module>
  import pybrake
  File "/var/task/pybrake/__init__.py", line 1, in <module>
  from .notifier import Notifier
  File "/var/task/pybrake/notifier.py", line 14, in <module>
  from .routes import RouteStats
  File "/var/task/pybrake/routes.py", line 7, in <module>
  from .tdigest import as_bytes, TDigestStat
  File "/var/task/pybrake/tdigest.py", line 4, in <module>
  import tdigest
  File "/var/task/tdigest/__init__.py", line 1, in <module>
  from .tdigest import TDigest
  File "/var/task/tdigest/tdigest.py", line 5, in <module>
  from accumulation_tree import AccumulationTree
  File "/var/task/accumulation_tree/__init__.py", line 2, in <module>
  from .accumulation_tree import AccumulationTree
ImportError: /var/task/accumulation_tree/accumulation_tree.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PyFPE_jbuf
[1562696051691] [DEBUG] 2019-07-09T18:14:11.691Z 9cb5828e-6072-4ae4-a0c3-2be817ac9c13 Zappa Event: {'resource': '/', 'path': '/', 'httpMethod': 'GET', 'headers': {'Accept':
'*/*', 'accept-encoding': 'gzip, deflate', 'Authorization': 'Token token=abxckjhd btoken=oiyasdfjkhgk', 'Cache-Control': 'no-cache', 'Host': '2cv1y57cua.execute-api.us-east-1.amazonaws.com', 'Postman-Token': 'b0d1b3b3-0c01-420d-ac4f-f0f829c0d44e', 'User-Agent': 'PostmanRuntime/7.15.0', 'X-Amzn-Trace-Id': 'Root=1-5d24d973-59bcdf2478839d3a4b9ed36a', 'X-Forwarded-For': '150.107.189.220', 'X-Forwarded-Port': '443', 'X-Forwarded-Proto': 'https'}, 'multiValueHeaders': {'Accept': ['*/*'], 'accept-encoding': ['gzip, deflate'], 'Authorization': ['Token token=abxckjhd btoken=oiyasdfjkhgk'], 'Cache-Control': ['no-cache'], 'Host': ['2cv1y57cua.execute-api.us-east-1.amazonaws.com'], 'Postman-Token': ['b0d1b3b3-0c01-420d-ac4f-f0f829c0d44e'], 'User-Agent': ['PostmanRuntime/7.15.0'], 'X-Amzn-Trace-Id': ['Root=1-5d24d973-59bcdf2478839d3a4b9ed36a'], 'X-Forwarded-For': ['150.107.189.220'], 'X-Forwarded-Port': ['443'], 'X-Forwarded-Proto': ['https']}, 'queryStringParameters': None, 'multiValueQueryStringParameters': None, 'pathParameters': None, 'stageVariables': None, 'requestContext': {'resourceId': 'o3li2ozlsb', 'resourcePath': '/', 'httpMethod': 'GET', 'extendedRequestId': 'ckbqEEFYIAMF2qQ=', 'requestTime':
'09/Jul/2019:18:14:11 +0000', 'path': '/dev/', 'accountId': '232720106935', 'protocol': 'HTTP/1.1', 'stage': 'dev', 'domainPrefix': '2cv1y57cua', 'requestTimeEpoch': 1562696051622, 'requestId': '59fe1dce-a275-11e9-877b-6778d891367b', 'identity': {'cognitoIdentityPoolId': None, 'accountId': None, 'cognitoIdentityId': None, 'caller': None, 'sourceIp': '150.107.189.220', 'principalOrgId': None, 'accessKey': None, 'cognitoAuthenticationType': None, 'cognitoAuthenticationProvider': None, 'userArn': None, 'userAgent': 'PostmanRuntime/7.15.0', 'user': None}, 'domainName': '2cv1y57cua.execute-api.us-east-1.amazonaws.com', 'apiId': '2cv1y57cua'}, 'body': None, 'isBase64Encoded': False}
[1562696051691] [DEBUG] 2019-07-09T18:14:11.691Z 9cb5828e-6072-4ae4-a0c3-2be817ac9c13 host found: [2cv1y57cua.execute-api.us-east-1.amazonaws.com]
[1562696051691] [DEBUG] 2019-07-09T18:14:11.691Z 9cb5828e-6072-4ae4-a0c3-2be817ac9c13 amazonaws found in host
[1562696051698] 'NoneType' object is not callable
[1562696103648] Instancing..
[1562696106711] /var/task/accumulation_tree/accumulation_tree.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PyFPE_jbuf: ImportError
Traceback (most recent call last):
  File "/var/task/handler.py", line 602, in lambda_handler
  return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 245, in lambda_handler
  handler = cls()
  File "/var/task/handler.py", line 151, in __init__
  wsgi_app_function = get_django_wsgi(self.settings.DJANGO_SETTINGS)
  File "/var/task/zappa/ext/django_zappa.py", line 20, in get_django_wsgi
  return get_wsgi_application()
  File "/var/task/django/core/wsgi.py", line 12, in get_wsgi_application
  django.setup(set_prefix=False)
  File "/var/task/django/__init__.py", line 19, in setup
  configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
  File "/var/task/django/conf/__init__.py", line 56, in __getattr__
  self._setup(name)
  File "/var/task/django/conf/__init__.py", line 43, in _setup
  self._wrapped = Settings(settings_module)
  File "/var/task/django/conf/__init__.py", line 106, in __init__
  mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/var/lang/lib/python3.6/importlib/__init__.py", line 126, in import_module
  return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/var/task/pos_authoriser/settings.py", line 15, in <module>
  import pybrake
  File "/var/task/pybrake/__init__.py", line 1, in <module>
  from .notifier import Notifier
  File "/var/task/pybrake/notifier.py", line 14, in <module>
  from .routes import RouteStats
  File "/var/task/pybrake/routes.py", line 7, in <module>
  from .tdigest import as_bytes, TDigestStat
  File "/var/task/pybrake/tdigest.py", line 4, in <module>
  import tdigest
  File "/var/task/tdigest/__init__.py", line 1, in <module>
  from .tdigest import TDigest
  File "/var/task/tdigest/tdigest.py", line 5, in <module>
  from accumulation_tree import AccumulationTree
  File "/var/task/accumulation_tree/__init__.py", line 2, in <module>
  from .accumulation_tree import AccumulationTree
ImportError: /var/task/accumulation_tree/accumulation_tree.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PyFPE_jbuf

Broken LoggingHandler

LoggingHandler builds notification on its own, so the context is missing a lot of information being added by Notifier.build_notice like 'environment' for instance. Why don't you use build_notice from Notifier and just extend it?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.