korfuri / django-prometheus Goto Github PK
View Code? Open in Web Editor NEWExport Django monitoring metrics for Prometheus.io
License: Apache License 2.0
Export Django monitoring metrics for Prometheus.io
License: Apache License 2.0
I am looking in to using this for several projects, but all of them don't use pip in the production environment, and currently as far as I can tell, using this would necessitate using the pip prometheus exporter, which requires the presence of pip.
Of course, as this is free software (thanks for that :) ), I can just patch this out (it should be as easy as removing this line [1]), but I though I would ask if you would consider either removing this (its easy enough for those who want it to add it in to their applications directly), or making it configurable in some way such that I can avoid the effort of maintaining a patch?
1: https://github.com/korfuri/django-prometheus/blob/master/django_prometheus/__init__.py#L13
Hi,
I followed the documentation and changed my CACHES settings from
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
to
CACHES = {
'default': {
'BACKEND': 'django_prometheus.core.cache.backends.filebased.FileBasedCache',
But now Django errors with:
InvalidCacheBackendError at /api/get_all_rooms_as_json
Could not find backend 'django_prometheus.core.cache.backends.filebased.FileBasedCache': No module named core.cache.backends.filebased
Versions:
Python 2.7.6
Django: 1.11.10
django-prometheus (1.0.15)
prometheus-client (0.3.1)
Right now, exporters are created regardless of the configuration, just via an import. We should rely on django signals and read the configuration before starting an exporter.
Currently I'm trying to add this to my Django, project however it seems this effects the way the database is connected to in a way that causes https://github.com/celery/django-celery to have issues and causes all my workers to have OperationalError: SSL SYSCALL error: Bad file descriptor
errors.
Removing this library fixes it.
Is there a way to setup the metrics page in a way that it's accessbile from prometheus but blocked y normale requests? right now if i access mydomain.com/metrics
i can see the metrics exposed and I would like to avoid it.
Would it even be feasible to use something like this when you're running a django app under a wsgi container such as uwsgi with multiple processes and then apache/nginx via the native uwsgi module?
This seems great if you're only going to run a single process, but when your application needs to scale due to traffic load (as several of our internal apps do), this model breaks down.
Any advice on setting up this middleware for a use case like that? Also, for shops that have multiple django applications, it might make sense to put some sort of a configurable prefix in the settings to differentiate between different applications. Otherwise, there could be clashes when the same urlconf is defined in two different applications with entirely different code.
I'm getting an odd error when using the MySQL backend with a fairly ordinary django (1.8.X) project:
ie:
DATABASES = {
'default': {
'ENGINE': 'django_prometheus.db.backends.mysql',
.
.
error (on start up of the dev server):
Traceback (most recent call last):
.../manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
.../django/db/backends/base/base.py", line 137, in _cursor
return self.create_cursor()
.../django-prometheus/django_prometheus/db/common.py", line 48, in create_cursor
self.CURSOR_CLASS, self.alias, self.vendor))
TypeError: cursor() got an unexpected keyword argument 'factory'
I've had a look at the common.py DatabaseWrapperMixin, but couldn't get things working.
Any ideas?
Hi,
I'm trying expose user count in multiproccess mode. I trying do it with gauge because number of user can be reduced. To user model signals is connected function to change gauge. Because of multiproccess mode it will change only for one pid metric. There is any solution for this problem?
According to the django documentations here https://docs.djangoproject.com/en/1.9/ref/request-response/#django.http.HttpRequest.resolver_match resolver_match
is only set after url resolving, which is not done in the Middleware. Therefore requests_by_view_transport_method is never incremented.
referring to https://github.com/korfuri/django-prometheus/blob/master/django_prometheus/middleware.py#L120
Thank you, this package is very useful otherwise.
Because the content of a StreamingHttpResponse
is an iterable, https://github.com/korfuri/django-prometheus/blob/master/django_prometheus/middleware.py#L135 cannot get the size of the data. One solution is to replace:
if response.streaming:
responses_streaming.inc()
responses_body_bytes.observe(len(response.content))
by:
if response.streaming:
responses_streaming.inc()
else:
responses_body_bytes.observe(len(response.content))
Shall I make a patch implementing that or is there a better solution?
some way to use django >= 1.10?
Using django-prometheus 1.0.9 with django 1.11.2.
I have some django-rest-framework views that are being cached with the cache_page decorator like:
@method_decorator(cache_page(settings.CACHE_TIMEOUT_API))
def dispatch(self, *args, **kwargs):
return super(BaseAPICachedView, self).dispatch(*args, **kwargs)
When there is a cache hit, the django-prometheus middleware fails in middleware.py:135 with AttributeError: 'Response' object has no attribute 'template_name'
def process_template_response(self, request, response):
responses_by_templatename.labels(str(
response.template_name)).inc()
return response
Sure enough, the cached views don't seem to have template_name. I fixed it by checking for the attribute first, not sure if that's the best way but it works for me:
def process_template_response(self, request, response):
if hasattr(response, 'template_name'):
responses_by_templatename.labels(str(
response.template_name)).inc()
return response
It seems django_prometheus is lacking support for the LocMemCache
cache backend.
This:
CACHES = {
'default': {
'BACKEND': 'django_prometheus.cache.backends.locmem.LocMemCache',
'LOCATION': 'webhooks',
}
}
Results in:
django.core.cache.backends.base.InvalidCacheBackendError: Could not find backend 'django_prometheus.cache.backends.locmem.LocMemCache': No module named 'django_prometheus.cache.backends.locmem'
Version: 1.0.6
...
<redacted>/python3.5/site-packages/django_prometheus/utils.py
<redacted>/python3.5/site-packages/tests/__init__.py
<redacted>/python3.5/site-packages/tests/__pycache__/__init__.cpython-35.pyc
<redacted>/python3.5/site-packages/tests/__pycache__/test_django_prometheus.cpython-35.pyc
<redacted>/python3.5/site-packages/tests/__pycache__/test_testutils.cpython-35.pyc
<redacted>/python3.5/site-packages/tests/test_django_prometheus.py
<redacted>/python3.5/site-packages/tests/test_testutils.py
Suggested solution:
test_suite="tests",
-> test_suite="django_prometheus.tests,
Please publish a new one which has the latest fixes
I would like to not have Http404
registered by the exception counters because it's not a "hard" kind of error and django_http_exceptions_total_by_view
becomes a bit useless to use properly beacuse of it.
django_http_exceptions_total_by_type{type="Http404"} 10.0
django_http_exceptions_total_by_view{view_name="wagtail_serve"} 10.0
The two metrics could also be merged into a new one instead even though It might add a lot of combinations in some cases.
import django_prometheus
Traceback (most recent call last):
File "", line 1, in
File "django_prometheus/init.py", line 9, in
import django_prometheus.middleware
File "django_prometheus/middleware.py", line 2, in
from django_prometheus.utils import Time, TimeSince, PowersOf
File "django_prometheus/utils.py", line 2, in
from prometheus_client import _INF
ImportError: cannot import name _INF
What 's up? why?
I would like to use django-prometheus for my own project but I'm not sure if it works with PostGIS.
Is there work that needs to be done or does it work out of the box?
I was looking for a configuration to run with nginx + uwsgi.
The only thing you need to do in order to make this work is adding the following line in uwsgi.ini:
enable-threads=True
This will enable threads raised by the app in uwsgi.
BUT, When I go to the expression browser or promdash, it doesn't seem to report anything from the app, it seems it's instrumenting from nowhere.
I have a django app running behind gunicorn, where the workers recycle regularly.
When this happens, certain time series create separate metrics because they specify a pid
label. The pid
changes at every recycle. This then causes the metrics page to take a long time to return, even causing Prometheus monitoring to timeout with context deadline exceeded
. I know I could extend the timeout past the default 15 seconds, but that isn't ideal.
Over a 12 hour period, I end up with 3000+ of time series like django_migrations_unapplied_total{connection="default",pid="21204"} 0.0
and django_migrations_applied_total{connection="default",pid="7175"} 0.0
Is there a way to turn off certain metrics, or remove the pid
label, so it can be collected within a single series? I'm sure pid
can be valuable in certain situations, but in this case, its causing performance issues.
I haven't seen a way to dump or ignore them at the prometheus-client level, but I apologize if there is, as I don't have in-depth knowledge of these libraries.
Thanks!
Mike
ExportModelOperationsMixin doesn't register update/delete/insert when query was enqueued in django-rq worker
/db/common.py's ExceptionCounterByType context manager passes a dict to labels
; however, version 0.0.15 of prometheus-client removes this usage, so that keyword arguments must be passed instead. requirements.txt
only specifies prometheus-client>=0.0.13, so installing django-prometheus right now will install the latest, incompatible prometheus-client.
When I use it with Django 1.10, the middleware will fail as
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/exception.py", line 39, in inner
response = get_response(request)
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 244, in _legacy_get_response
response = middleware_method(request)
File "/usr/local/lib/python2.7/dist-packages/django_prometheus/middleware.py", line 114, in process_request
requests_body_bytes.observe(len(request.body))
File "/usr/local/lib/python2.7/dist-packages/django/http/request.py", line 267, in body
int(self.META.get('CONTENT_LENGTH', 0)) > settings.DATA_UPLOAD_MAX_MEMORY_SIZE):
ValueError: invalid literal for int() with base 10: ''
It works well with Django 1.8
anyone that set up the grafana dashboard to have all the metrics that are collected?
Partial stack trace:
File "venv/local/lib/python2.7/site-packages/django/utils/functional.py", line 35, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "venv/local/lib/python2.7/site-packages/django/db/backends/mysql/features.py", line 67, in is_sql_auto_is_null_enabled
with self.connection.cursor() as cursor:
File "venv/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 254, in cursor
return self._cursor()
File "venv/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 231, in _cursor
return self._prepare_cursor(self.create_cursor(name))
TypeError: create_cursor() takes exactly 1 argument (2 given)
PostgreSQL is very used in the Django world and missing support for it is a pity. The backend README says «Doing so should be straightforward, but testing that it works and maintaining it is a lot of busywork».
Could this be added to the roadmap?
The MySQL backend is basically all extracted into the common mixin. What kind of troubles would I encounter doing the same for Postgres?
Django 1.10 drops support for django.conf.urls.patterns
. This isn't actually used in django-prometheus but the import is still there in https://github.com/korfuri/django-prometheus/blob/master/django_prometheus/urls.py#L1.
I can't start a Django shell on a host which is currently running my Django project because python manage.py shell
tries to start the metrics endpoint on port 8001.
Traceback (most recent call last):
File "manage.py", line 15, in <module>
execute_from_command_line(sys.argv)
File "/venv/lib/python3.6/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
utility.execute()
File "/venv/lib/python3.6/site-packages/django/core/management/__init__.py", line 347, in execute
django.setup()
File "/venv/lib/python3.6/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/venv/lib/python3.6/site-packages/django/apps/registry.py", line 120, in populate
app_config.ready()
File "/venv/lib/python3.6/site-packages/django_prometheus/apps.py", line 22, in ready
SetupPrometheusExportsFromConfig()
File "/venv/lib/python3.6/site-packages/django_prometheus/exports.py", line 102, in SetupPrometheusExportsFromConfig
SetupPrometheusEndpointOnPort(port, addr)
File "/venv/lib/python3.6/site-packages/django_prometheus/exports.py", line 44, in SetupPrometheusEndpointOnPort
prometheus_client.start_http_server(port, addr=addr)
File "/venv/lib/python3.6/site-packages/prometheus_client/exposition.py", line 129, in start_http_server
httpd = _ThreadingSimpleServer((addr, port), CustomMetricsHandler)
File "/usr/local/lib/python3.6/socketserver.py", line 453, in __init__
self.server_bind()
File "/usr/local/lib/python3.6/http/server.py", line 136, in server_bind
socketserver.TCPServer.server_bind(self)
File "/usr/local/lib/python3.6/socketserver.py", line 467, in server_bind
self.socket.bind(self.server_address)
OSError: [Errno 98] Address in use
Prometheus thread should not be started in a Django shell environment.
It would be helpful to be able to instrument a model to export the rate of creations, deletions and updates (and selections?). This would make it trivial to graph account creations over time.
Bonus point if this can be done as automagically as possible for the standard user model.
I am getting AttributeError: module 'django_prometheus.models' has no attribute 'Mixin' when I try to run python manage.py migrate command, I have followed steps in the README.md file.
I am trying to use Prometheus with
django 1.8
Python 3.5.3
Fedora 25.
It is good to document this somewhere. If you are running django inside a pod then prometheus will not be able to scrape the /metrics unless you allow the ip address in the ALLOWED_HOSTS
In django settings.py
from socket import gethostname, gethostbyname
ALLOWED_HOSTS = [ gethostname(), gethostbyname(gethostname()), ]
It took a while to figure out why prometheus was not able to reach the /metrics.
Hi, I wanna use prometheus to generate an app-metric dashboard, but since I'm using multitenancy gem I'm not able to monitor a single endpoint. Also, I'm running prometheus in a docker container, so at this time I'm not able to monitor my django app.
Is there any alternative to send metrics to prometheus instead prometheus monitoring my endpoints?
Prometheus/OpenMetrics/prometheus_client
has standardized on making counters end in _total
: https://github.com/prometheus/client_python/releases/tag/v0.4.0
Counter time series will now always be exposed with _total, and counter metrics will have a _total suffix stripped. This is as the internal data model is now OpenMetrics, rather than Prometheus Text Format
This means all the total_by_xxx
counters are now total_by_xxx_total
. Which is awkward. Examples:
django_http_responses_total_by_templatename_total
django_http_requests_total_by_method_total
django_http_requests_total_by_view_transport_method_total
django_http_exceptions_total_by_type_total
I'm not sure what the new names should be, but hopefully someone has a good idea.
First of all, thanks for this cool project.
I'm using uwsgi + Kubernetes and I'd like to add this to my Django app but I'm not sure what would happen if I have one Service but more than one Pod in my cluster?
Imagine my export port is 9090
, then Prometheus sends request to scrap metrics and save them but requests could reach different pods / uwsgi processes. Therefore, the metrics are not accurate.
What should I do in this situation? I presume this is a traditional problem.
Thanks!
I use django without any database and since version 1.0.1
django_prometheus doesn't start due to an ImproperlyConfigured exception raised by django,
Here is the stack trace
File "frontend/manage.py", line 30, in <module>
execute_from_command_line(sys.argv)
File "~/.virtualenvs/frontend/lib/python3.4/site-packages/django/core/management/__init__.py", line 351, in execute_from_command_line
utility.execute()
File "~/.virtualenvs/frontend/lib/python3.4/site-packages/django/core/management/__init__.py", line 325, in execute
django.setup()
File "~/.virtualenvs/frontend/lib/python3.4/site-packages/django/__init__.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File "~/.virtualenvs/frontend/lib/python3.4/site-packages/django/apps/registry.py", line 115, in populate
app_config.ready()
File "~/.virtualenvs/frontend/lib/python3.4/site-packages/django_prometheus/apps.py", line 22, in ready
ExportMigrations()
File "~/.virtualenvs/frontend/lib/python3.4/site-packages/django_prometheus/migrations.py", line 30, in ExportMigrations
executor = MigrationExecutor(connections[alias])
File "~/.virtualenvs/frontend/lib/python3.4/site-packages/django/db/migrations/executor.py", line 19, in __init__
self.loader = MigrationLoader(self.connection)
File "~/.virtualenvs/frontend/lib/python3.4/site-packages/django/db/migrations/loader.py", line 47, in __init__
self.build_graph()
File "~/.virtualenvs/frontend/lib/python3.4/site-packages/django/db/migrations/loader.py", line 182, in build_graph
self.applied_migrations = recorder.applied_migrations()
File "~/.virtualenvs/frontend/lib/python3.4/site-packages/django/db/migrations/recorder.py", line 59, in applied_migrations
self.ensure_schema()
File "~/.virtualenvs/frontend/lib/python3.4/site-packages/django/db/migrations/recorder.py", line 49, in ensure_schema
if self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()):
File "~/.virtualenvs/frontend/lib/python3.4/site-packages/django/db/backends/base/base.py", line 162, in cursor
cursor = self.make_debug_cursor(self._cursor())
File "~/.virtualenvs/frontend/lib/python3.4/site-packages/django/db/backends/dummy/base.py", line 21, in complain
raise ImproperlyConfigured("settings.DATABASES is improperly configured. "
django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.
Maybe we should be able to disable migration monitoring feature ?
From the Prometheus docs:
Use labels to differentiate the characteristics of the thing that is being measured... Do not put the label names in the metric name, as this introduces redundancy and will cause confusion if the respective labels are aggregated away.
I therefore propose the the following be combined using suitable labels:
requests_total
:
reponses_total
:
exceptions_total
:
@korfuri, would you be interested in receiving pull requests to this effect?
EDIT: I'm open to suggestions regarding how to prevent this change breaking existing monitoring infrastructure.
Hi, I'm getting the following exception with django 1.9:
Unhandled exception in thread started by <pydev_monkey._NewThreadStartupWithTrace instance at 0x7f68ac317c20>
Traceback (most recent call last):
File "/data/pycharm-community-5.0.4/helpers/pydev/pydev_monkey.py", line 521, in __call__
return self.original_func(*self.args, **self.kwargs)
File "/data/venv/local/lib/python2.7/site-packages/django/utils/autoreload.py", line 226, in wrapper
fn(*args, **kwargs)
File "/data/venv/local/lib/python2.7/site-packages/django/core/management/commands/runserver.py", line 109, in inner_run
autoreload.raise_last_exception()
File "/data/venv/local/lib/python2.7/site-packages/django/utils/autoreload.py", line 249, in raise_last_exception
six.reraise(*_exception)
File "/data/venv/local/lib/python2.7/site-packages/django/utils/autoreload.py", line 226, in wrapper
fn(*args, **kwargs)
File "/data/venv/local/lib/python2.7/site-packages/django/__init__.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File "/data/venv/local/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate
app_config = AppConfig.create(entry)
File "/data/venv/local/lib/python2.7/site-packages/django/apps/config.py", line 116, in create
mod = import_module(mod_path)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/data/venv/local/lib/python2.7/site-packages/django_prometheus/apps.py", line 3, in <module>
from django_prometheus.migrations import ExportMigrations
File "/data/venv/local/lib/python2.7/site-packages/django_prometheus/migrations.py", line 3, in <module>
from django.db.migrations.executor import MigrationExecutor
File "/data/venv/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 7, in <module>
from .loader import MigrationLoader
File "/data/venv/local/lib/python2.7/site-packages/django/db/migrations/loader.py", line 10, in <module>
from django.db.migrations.recorder import MigrationRecorder
File "/data/venv/local/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 12, in <module>
class MigrationRecorder(object):
File "/data/venv/local/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 26, in MigrationRecorder
class Migration(models.Model):
File "/data/venv/local/lib/python2.7/site-packages/django/db/models/base.py", line 94, in __new__
app_config = apps.get_containing_app_config(module)
File "/data/venv/local/lib/python2.7/site-packages/django/apps/registry.py", line 239, in get_containing_app_config
self.check_apps_ready()
File "/data/venv/local/lib/python2.7/site-packages/django/apps/registry.py", line 124, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
Hi,
I'm looking for documentation on how to use this with > 1 apache daemon mode processes.
Will all the metrics across all processes be aggregated by the main apache daemon and served at its port @ url /metrics?
Also, notes on the performance impact (roughly) per process.
This project is being highlighted by https://caniusepython3.com/ as not being compatible with Python 3 as it does not have the Python 3 trove classifier.
Would you accept a pull request which adds a docker-compose.yml
for postgres/mysql and a travis compatible tox setup so that it is easier to do local development and run all the tests?
Django version 1.11.5 does not render metrics when hitting /metrics
When I try to do collectstatic with PROMETHEUS_EXPORT_MIGRATIONS = True
it fails with the following error:
Traceback (most recent call last):
File "./manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
utility.execute()
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 327, in execute
django.setup()
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/__init__.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/apps/registry.py", line 115, in populate
app_config.ready()
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django_prometheus/apps.py", line 24, in ready
ExportMigrations()
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django_prometheus/migrations.py", line 46, in ExportMigrations
executor = MigrationExecutor(connections[alias])
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 20, in __init__
self.loader = MigrationLoader(self.connection)
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/db/migrations/loader.py", line 49, in __init__
self.build_graph()
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/db/migrations/loader.py", line 176, in build_graph
self.applied_migrations = recorder.applied_migrations()
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 65, in applied_migrations
self.ensure_schema()
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 52, in ensure_schema
if self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()):
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 233, in cursor
cursor = self.make_cursor(self._cursor())
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 204, in _cursor
self.ensure_connection()
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 199, in ensure_connection
self.connect()
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/db/utils.py", line 95, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 199, in ensure_connection
self.connect()
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 171, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/django/db/backends/postgresql/base.py", line 175, in get_new_connection
connection = Database.connect(**conn_params)
File "/home/user/proj/app/venv/local/lib/python2.7/site-packages/psycopg2/__init__.py", line 164, in connect
conn = _connect(dsn, connection_factory=connection_factory, async=async)
django.db.utils.OperationalError: could not connect to server: No route to host
Is the server running on host "databaseserver.example.com" (127.1.2.3) and accepting
TCP/IP connections on port 5432?
However with PROMETHEUS_EXPORT_MIGRATIONS = True
everything works fine.
Running collectstatic should not require a database connection.
Hi all,
does anyone of you use Django + django-prometheus and run it inside a docker container?
how do you do (i'm interest in the uwsgi.ini , .wsgi file and dockerfile)?
I'm able to run it, but the stats is not working (one remains 0, another one is not even exposed).
If i run it locally without docker it works..
Hi,
My build fails during the static file preparation. Is any one else experiencing this, and does any one have any ideas on how to solve the problem?
Thanks
Deploy detail:
python manage.py collectstatic --no-post-process --noinput
using Msgpack encoder
resetting queues. pids(old:None new:17646)
starting flush thread
cannot send services: [Errno 111] Connection refused
Traceback (most recent call last):
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/pymysql/connections.py", line 916, in connect
**kwargs)
File "/opt/circleci/python/3.6.1/lib/python3.6/socket.py", line 722, in create_connection
raise err
File "/opt/circleci/python/3.6.1/lib/python3.6/socket.py", line 713, in create_connection
sock.connect(sa)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
self.connect()
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/db/backends/base/base.py", line 189, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django_prometheus/db/common.py", line 41, in get_new_connection
*args, **kwargs)
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 274, in get_new_connection
conn = Database.connect(**conn_params)
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/pymysql/init.py", line 90, in Connect
return Connection(*args, **kwargs)
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/pymysql/connections.py", line 706, in init
self.connect()
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/pymysql/connections.py", line 963, in connect
raise exc
pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on 'kubernetes-apps-staging.cd2vfjltihkr.us-east-1.rds.amazonaws.com' (timed out)")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 35, in
execute_from_command_line(sys.argv)
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/core/management/init.py", line 363, in execute_from_command_line
utility.execute()
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/core/management/init.py", line 337, in execute
django.setup()
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/init.py", line 27, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/apps/registry.py", line 116, in populate
app_config.ready()
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django_prometheus/apps.py", line 24, in ready
ExportMigrations()
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django_prometheus/migrations.py", line 46, in ExportMigrations
executor = MigrationExecutor(connections[alias])
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/db/migrations/executor.py", line 20, in init
self.loader = MigrationLoader(self.connection)
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/db/migrations/loader.py", line 52, in init
self.build_graph()
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/db/migrations/loader.py", line 209, in build_graph
self.applied_migrations = recorder.applied_migrations()
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 65, in applied_migrations
self.ensure_schema()
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 52, in ensure_schema
if self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()):
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/ddtrace/contrib/django/db.py", line 27, in cursor
return TracedCursor(tracer, conn, conn._datadog_original_cursor())
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/db/backends/base/base.py", line 254, in cursor
return self._cursor()
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/db/backends/base/base.py", line 229, in _cursor
self.ensure_connection()
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
self.connect()
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/db/utils.py", line 94, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
self.connect()
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/db/backends/base/base.py", line 189, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django_prometheus/db/common.py", line 41, in get_new_connection
*args, **kwargs)
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 274, in get_new_connection
conn = Database.connect(**conn_params)
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/pymysql/init.py", line 90, in Connect
return Connection(*args, **kwargs)
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/pymysql/connections.py", line 706, in init
self.connect()
File "/home/ubuntu/virtualenvs/venv-3.6.1/lib/python3.6/site-packages/pymysql/connections.py", line 963, in connect
raise exc
django.db.utils.OperationalError: (2003, "Can't connect to MySQL server on 'kubernetes-apps-staging.cd2vfjltihkr.us-east-1.rds.amazonaws.com' (timed out)")
python manage.py collectstatic --no-post-process --noinput returned exit code 1
Action failed: python manage.py collectstatic --no-post-process --noinput
I m using uWSGI and django 1.11.14 and django_proometheus in multi-process mode
uwsgi.ini
env = prometheus_multiproc_dir=/path/to/django_metrics
wsgi.py
from prometheus_client import core
import uwsgi
# Use uwsgi's worker_id rather than system pids
core._ValueClass = core._MultiProcessValue(_pidFunc=uwsgi.worker_id)
Note: reverting to 0.3.1 works fine
Thank you for this great integration, I was able to get it up and running in no time.
Currently, the library uses the default buckets provided by the prometheus client for Histogram. Can we make it configurable? I will be happy to send a PR if you believe that's something we should be doing.
Default buckets: https://github.com/prometheus/client_python/blob/master/prometheus_client/core.py
Use cases:
We should export metrics relative to database i/o.
Wishlist:
I need the django_http_requests_latency_seconds_by_view_method
metrics to analysis server perf problem.
The metrics is already on the master. So just want to know when to last release next version.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.