boxine / django-huey-monitor Goto Github PK

View Code? Open in Web Editor NEW

86.0 4.0 20.0 1 MB

Django based tool for monitoring huey task queue: https://github.com/coleifer/huey

License: GNU General Public License v3.0

Makefile 3.03% Shell 1.82% Dockerfile 0.44% Python 91.05% HTML 3.58% CSS 0.09%

huey django monitoring

django-huey-monitor's People

Contributors

Stargazers

Watchers

Forkers

elpaso jlpoveda xolan formacube mrmilu pirate skrattoune mod0 supermrblob vidski rsp2k brahimmade joshuadavidthomas henribru stungkit

django-huey-monitor's Issues

No feedback on PR throughput formating for rate<1 #48 for one month

could you please give feedback and / or integrate PR #48 "throughput formating for rate<1" ?
It is availlable for one month already and there was not even a feedback even if it is quite straightforward.

Thanks in advance

'humanize_time' is not a registered tag library.

When clicking on a task-name this url is linked /admin/huey_monitor/taskmodel/<taskid>/change/. I then get the following error message.

'humanize_time' is not a registered tag library. Must be one of:
admin_list
admin_modify
admin_urls
cache
debugger_tags
guardian_tags
highlighting
i18n
import_export_tags
indent_text
l10n
log
rest_framework
static
syntax_color
tz
widont

PR closed without merging first

My PR has been closed without being merged.

My code is plenty of patches since begining of september because I'm waiting for you to validate the code.
You make me wait 6 months to change the formating of the code while it would take you less than 5 minutes to do it.

I have to wait for 6 month ...
and then you decide to kill everything because you have another idea ... that is not even yet implemented !

Please first merge my PR, so that I can have some code that works correctly and get rid of the patches.
Then when you have the time, there is no problem to implement your new idea.

But at least in the meantime, I can have a code that work, without monkey patch.
and also, it will be easy to test if your new idea is still functional for tasks that takes few days or few weeks to execute

Better monitoring of recursive tasks progression

Refering to issue #91,
I have implemented locally a few helpers which enable better monitoring of progression, especially useful in the case of recursive sub-tasks.

To document their usage, I have added functional example in https://github.com/boxine/django-huey-monitor/blob/master/huey_monitor_tests/test_app/tasks.py
Please have a look at the corresponding PR

DisallowedModelAdminLookup

Status of Parent task not depending on progress vs.total

Hi,
My tasks are organised as a set of sub-tasks

completion of each sub-task is adding 1 to the progress of the parent task

however if you look at the first snapshot that shows only the columns visible without scrolling, one could think that all activities are complete and that it would be ok to switch off the server.

But if you scroll right to see the progression data, the situation is very different:

At the moment, the status of the Parent task is not related at all to the task progression (TaskModel.progress_info[0] vs. TaskModel.total)

In my use case, that would be necessary to take task progression into account when a total is specified.

How could I ensure that the "signal_complete" is not sent before each sub-task is complete?
and that it is sent once the last sub-task has been completed?

Saving task arguments would be very beneficial

In the task model, a lot of info is saved which is very useful. But in a production scenario, it will also be super helpful if the args passed to the task function are also stored. If a task fails, the error message is there but it becomes difficult to map it to the input. If you think this is useful, I would be willing to work on a PR for this.

Deadlock on saving a task instance

Hello,

We have a production setup with lots of workers running in parallel and a deadlock error occur from time to time, I think when two workers are trying the update the same task at the same moment. Here is the traceback of the error:

DeadlockDetected: deadlock detected
DETAIL:  Process 3256555 waits for ShareLock on transaction 3220907091; blocked by process 3256561.
Process 3256561 waits for ShareLock on transaction 3220907090; blocked by process 3256555.
HINT:  See server log for query details.
CONTEXT:  while updating tuple (1239,37) in relation "huey_monitor_taskmodel"

  File "django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)

OperationalError: deadlock detected
DETAIL:  Process 3256555 waits for ShareLock on transaction 3220907091; blocked by process 3256561.
Process 3256561 waits for ShareLock on transaction 3220907090; blocked by process 3256555.
HINT:  See server log for query details.
CONTEXT:  while updating tuple (1239,37) in relation "huey_monitor_taskmodel"

  File "huey/consumer.py", line 94, in initialize
    startup_hook()
  File "huey_monitor/tasks.py", line 122, in startup_handler
    update_task_instance(
  File "huey_monitor/tasks.py", line 45, in update_task_instance
    instance.save(update_fields=update_fields)
  File "bx_django_utils/models/timetracking.py", line 59, in save
    super().save(**kwargs)
  File "django/db/models/base.py", line 814, in save
    self.save_base(
  File "django/db/models/base.py", line 877, in save_base
    updated = self._save_table(
  File "django/db/models/base.py", line 990, in _save_table
    updated = self._do_update(
  File "django/db/models/base.py", line 1054, in _do_update
    return filtered._update(values) > 0
  File "django/db/models/query.py", line 1231, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  File "django/db/models/sql/compiler.py", line 1982, in execute_sql
    cursor = super().execute_sql(result_type)
  File "django/db/models/sql/compiler.py", line 1560, in execute_sql
    cursor.execute(sql, params)
  File "django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
  File "django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)

Any idea on how to prevent this?
We are using postgreSQL if it matters.

TextField for `exception_line`?

Hi,

In both TaskModel and SignalInfoModel, the project uses CharField with a specified max_length. While this is okay for most fields, I was thinking that this could be a limitation for exception_line field of the SignalInfoModel. Can we move to TextField for that field and remove the max_length limitation?

Also, separately, if target database is Postgres, it seems to not be an issue and infact text is more perfmant than char: https://www.postgresql.org/docs/9.0/datatype-character.html

Regards,

Possible to monitor tasks enqueued from outside of Django?

Thanks for the awesome project! I have a question that might be out of scope, but I'd appreciate any pointers.

I've been using Huey in a django project without huey.contrib.djhuey, so that I can run a huey_consumer on a different machine where the django application is not fully set up (the code is obviously accessible). I've organized my code as described here and on the other machine, I start a worker like this: REDIS_HOST=172.0.20.2 huey_consumer app.huey.main.huey. This works great and makes it so that I don't have to install anything on the worker node other than huey and redis-py.

However, I can't figure out a way to wire up huey-monitor to all of this. Is there a mechanism by which I can provide it with a huey instance?

Thanks!

Issue with ManifestStaticFilesStorage due to TaskModelAdmin referencing non-existent CSS file

TaskModelAdmin references a CSS file called huey_monitor.css. This file doesn't actually exist, which causes problems if you're using Django's ManifestStaticFilesStorage or something inheriting from it like Whitenoise's CompressedManifestStaticFilesStorage. Specifically you end up with a ValueError and a 500 when navigating to the admin task list in production, unless you tweak the manifest_strict setting. I'm not sure if this CSS thing is leftover from something or if it's meant to be an optional file that can be added by users of the library to customize how the Huey admin looks. In the former case, could we remove it so that we don't have to disable manifest_strict? In the latter case, is disabling manifest_strict the intended workaround for this issue?

manage.py command to delete old data

In order to allow us do some regular house-cleaning, this extension should have a command to flush old (no longer relevant) task data.

Would be even more helpful if we could specify the number of days to keep,
something like:

python3 manage.py huey_monitor_flush 30

would delete task data older than 30 days.

This would be a great addition, as task data, if left unchecked, will start to grow larger and larger.

v0.7.0 gives error with django_huey's multi-queues

The version 0.7.0 gives a ConnectionError when using this library with django_huey's multi-queues.

Error seems to be here:

@register.simple_tag
def huey_counts_info():
    context = dict(
        huey_pending_count=HUEY.pending_count(),
        huey_scheduled_count=HUEY.scheduled_count(),
        huey_result_count=HUEY.result_count(),
    )

It tries to connect to a non-existing queue

Implement validator for TaskModel.desc field or crop to 64 char?

Hi,
I had a strange error error - (1406, "Data too long for column 'desc' at row 1")
After browsing the net, and then exploring the code of huey-monitor,

I finally manage to realize the reason my task was failling was that the description I was setting for my ProcessInfo was 67 char while only 64 were authorized by the TaskModel model.

This error was quite small, and may happen easily ... but it was quite hard to find-out.

I therefore would suggest to make this error much more easy to find-out for users.

One solution could be to implement a custom validator for the TaskModel.desc field to document specifically the error for the user.

However, having a desc bigger than 64 char should not for me prevent a task to run.
I would therefore suggest another simpler solution:
to implement in the ProcessInfo model a crop to 64 char when updating TaskModel.

A minimal implementation could be:

TaskModel.objects.filter(task_id=task.id).update(
            desc=self.desc[:64],
            total=self.total,
            unit=self.unit,
            unit_divisor=self.unit_divisor,
        )

Suggestion: showing task description when available on TaskAdmin

Hi, when going through the Tasks admin, I find it pretty difficult to know what the task is about as only the task name is availlable:

Through this view, we cannot know which task failed (and should trigger a corrective action) and which worked (and can eventually be deleted).

To have an idea about what the tasks are about, we need to edit each task until we find the right one:

then, first line is the only place where the task description (desc) is mentionned.

same with sub-tasks within the changelist view:

I would suggest to :
1/ display desc as the first parameter in list_view
2/ display desc as a parameter in change_view (in addition to the current title)

I'll be proposing a PR to implement this.

PS: What is your preference? that I continue creating Issues and corresponding PR ? or that I create PR directly, without Issue?

Cancel running Jobs

https://huey.readthedocs.io/en/latest/guide.html#canceling-or-pausing-tasks

TaskModelAdmin changelist view makes a gazillion db queries

On the first look this should be fixable with proper prefetching.

run migrate after installing

Hey,
I am currently testing huey with this monitor.

The QuickStart guide should contain that after installing it needs a python manage.py migrate.
Without it you will be greeted with an error message in the Django admin interface.

django.db.utils.OperationalError: (1114, "The table 'huey_monitor_taskprogressmodel' is full")

Hi @jedie,

I had a bad surprise this morning while checking on the progress of a process which had it's first sub-process running for over than 24h.

There was an error: django.db.utils.OperationalError: (1114, "The table 'huey_monitor_taskprogressmodel' is full")

Then I realized that for each tiny progress report, a record is created for TaskProgressModel

Could you please tell me what is the benefit of creating a new TaskProgressModel (even two when you want to report progress on both sub-task and maintask) each time you report a progress vs. incrementing directly the progress_count field of an existing TaskProgressModel ?

That would mean that in general, we would have only one TaskProgressModel for each TaskModel.

Thanks in advance for your feedback

Capture all log output in tests

Github seems to has here a bug, e.g.:

desc overwrites task name in admin view

Hello, happy new year.

When setting a task description like in the minimal example:

@task(context=True)
def foobar_task(task, list_to_process):
    process_info = ProcessInfo(
        task,
        desc='A description of this Job',
        total=len(list_to_process)
    )

The taskname column in the admin panel no longer shows the task name. It displays the complete description.
A long description results in a very ugly admin view.
Here is an example of my functionclean_huey_db() with desc='Huey DB Cleanup',:

Maybe it would be usefull to seperate this:

Add sth. like an verbose_name. If it is set, the task_name is overwritten with the verbose one.
Description only used to describe what this task is doing to display it in the admin panel.

Or am I wrong about the desc parameter?

Update of `Last update` value when subtasks are updated

Hi @jedie,

I realized that when tasks are running with subtasks, the Last update value is not updated when one of the sub-task is updated (as shown by the screenshot below).

Therefore, it is hard without opening the subtask to know if something has happened or not, or to get any sense of progress.

It would be great if the Last update value could also be updated each time one of the sub-task is updated, and not only when the main-task is updated (could be days in my case).

Thanks in advance

Restart jobs

https://huey.readthedocs.io/en/latest/guide.html#canceling-or-pausing-tasks

'exception_line': ['This field cannot be blank.']

Hi,

In SignalInfoModel, exception_line field is not nullable, blanks are also not allowed. When storing signals in store_signals if exc = None, it does not look like exception_line will be populated. Consequentially, I see:

django.core.exceptions.ValidationError: {'exception_line': ['This field cannot be blank.']}

I also noticed that in the TaskModel, field desc (in Huey-Monitor v0.4.6) can't be blank. But the default value for the field is '' and no value is passed when creating TaskModel in store_signals. Consequentially, I see:

django.core.exceptions.ValidationError: {'desc': ['This field cannot be blank.']}

Package Versions:
Python: 3.9
Django: 3.2.2
Huey: 2.4.3
Huey-Monitor: 0.4.6

huey_monitor generates error when launching Testing

Hi,

I've been implementing django-huey with django-huey-monitor.
It starts to do more or less what I want (please look also into the discussion thread #28).

... until I started to launch some testing on a totally unrelated part of my application.

And there, surprise, I have an avalanche of errors linked to django-huey-monitor during the setup of the Testing environement (cf Traceback below).

There is not much about huey_monitor on the net or in term of documentation, so I'll greatly appreciate I you could help me out.

Error occurred sending signal "executing"
Traceback (most recent call last):
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\sqlite3\base.py", line 423, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: no such table: huey_monitor_taskmodel

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\...\venv\lib\site-packages\huey\api.py", line 274, in _emit
    self._signal.send(signal, task, *args, **kwargs)
  File "C:\Users\...\venv\lib\site-packages\huey\signals.py", line 41, in send
    receiver(signal, task, *args, **kwargs)
  File "C:\Users\...\venv\lib\site-packages\huey_monitor\tasks.py", line 49, in store_signals
    defaults={'name': task.name}
  File "C:\Users\...\venv\lib\site-packages\django\db\models\manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "C:\Users\...\venv\lib\site-packages\django\db\models\query.py", line 581, in get_or_create
    return self.get(**kwargs), False
  File "C:\Users\...\venv\lib\site-packages\nplusone\core\signals.py", line 23, in wrapped
    ret = func(*args, **kwargs)
  File "C:\Users\...\venv\lib\site-packages\django\db\models\query.py", line 431, in get
    num = len(clone)
  File "C:\Users\...\venv\lib\site-packages\django\db\models\query.py", line 262, in __len__
    self._fetch_all()
  File "C:\Users\...\venv\lib\site-packages\nplusone\ext\django\patch.py", line 295, in fetch_all
    original_fetch_all(self)
  File "C:\Users\...\venv\lib\site-packages\django\db\models\query.py", line 1324, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "C:\Users\...\venv\lib\site-packages\django\db\models\query.py", line 51, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "C:\Users\...\venv\lib\site-packages\django\db\models\sql\compiler.py", line 1169, in execute_sql
    cursor.execute(sql, params)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "C:\Users\...\venv\lib\site-packages\django\db\utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\sqlite3\base.py", line 423, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: huey_monitor_taskmodel
Unhandled exception in task 20046722-1606-4d32-80a2-3f88ed08d4fe.
Traceback (most recent call last):
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\sqlite3\base.py", line 423, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: no such table: huey_monitor_taskmodel

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\...\venv\lib\site-packages\huey\api.py", line 367, in _execute
    task_value = task.execute()
  File "C:\Users\...\venv\lib\site-packages\huey\api.py", line 740, in execute
    return func(*args, **kwargs)
  File "C:\Users\...\venv\lib\site-packages\huey\contrib\djhuey\__init__.py", line 133, in inner
    return fn(*args, **kwargs)
  File "C:\Users\...\goodies\decorators\debug.py", line 74, in wrapper
    result = func(*args, **kwargs)
  File "C:\Users\...\puzzles\tasks\puzzles.py", line 49, in recalculate_pzl_medians
    unit='puzzles',
  File "C:\Users\...\venv\lib\site-packages\huey_monitor\tqdm.py", line 47, in __init__
    unit_divisor=self.unit_divisor,
  File "C:\Users\...\venv\lib\site-packages\django\db\models\query.py", line 783, in update
    rows = query.get_compiler(self.db).execute_sql(CURSOR)
  File "C:\Users\...\venv\lib\site-packages\django\db\models\sql\compiler.py", line 1535, in execute_sql
    cursor = super().execute_sql(result_type)
  File "C:\Users\...\venv\lib\site-packages\django\db\models\sql\compiler.py", line 1169, in execute_sql
    cursor.execute(sql, params)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "C:\Users\...\venv\lib\site-packages\django\db\utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\sqlite3\base.py", line 423, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: huey_monitor_taskmodel
Error occurred sending signal "error"
Traceback (most recent call last):
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\sqlite3\base.py", line 423, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: no such table: huey_monitor_taskmodel

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\...\venv\lib\site-packages\huey\api.py", line 367, in _execute
    task_value = task.execute()
  File "C:\Users\...\venv\lib\site-packages\huey\api.py", line 740, in execute
    return func(*args, **kwargs)
  File "C:\Users\...\venv\lib\site-packages\huey\contrib\djhuey\__init__.py", line 133, in inner
    return fn(*args, **kwargs)
  File "C:\Users\...\goodies\decorators\debug.py", line 74, in wrapper
    result = func(*args, **kwargs)
  File "C:\Users\...\puzzles\tasks\puzzles.py", line 49, in recalculate_pzl_medians
    unit='puzzles',
  File "C:\Users\...\venv\lib\site-packages\huey_monitor\tqdm.py", line 47, in __init__
    unit_divisor=self.unit_divisor,
  File "C:\Users\...\venv\lib\site-packages\django\db\models\query.py", line 783, in update
    rows = query.get_compiler(self.db).execute_sql(CURSOR)
  File "C:\Users\...\venv\lib\site-packages\django\db\models\sql\compiler.py", line 1535, in execute_sql
    cursor = super().execute_sql(result_type)
  File "C:\Users\...\venv\lib\site-packages\django\db\models\sql\compiler.py", line 1169, in execute_sql
    cursor.execute(sql, params)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "C:\Users\...\venv\lib\site-packages\django\db\utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\sqlite3\base.py", line 423, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: huey_monitor_taskmodel

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\...\venv\lib\site-packages\huey\api.py", line 274, in _emit
    self._signal.send(signal, task, *args, **kwargs)
  File "C:\Users\...\venv\lib\site-packages\huey\signals.py", line 41, in send
    receiver(signal, task, *args, **kwargs)
  File "C:\Users\...\venv\lib\site-packages\huey_monitor\tasks.py", line 49, in store_signals
    defaults={'name': task.name}
  File "C:\Users\...\venv\lib\site-packages\django\db\models\manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "C:\Users\...\venv\lib\site-packages\django\db\models\query.py", line 581, in get_or_create
    return self.get(**kwargs), False
  File "C:\Users\...\venv\lib\site-packages\nplusone\core\signals.py", line 23, in wrapped
    ret = func(*args, **kwargs)
  File "C:\Users\...\venv\lib\site-packages\django\db\models\query.py", line 431, in get
    num = len(clone)
  File "C:\Users\...\venv\lib\site-packages\django\db\models\query.py", line 262, in __len__
    self._fetch_all()
  File "C:\Users\...\venv\lib\site-packages\nplusone\ext\django\patch.py", line 295, in fetch_all
    original_fetch_all(self)
  File "C:\Users\...\venv\lib\site-packages\django\db\models\query.py", line 1324, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "C:\Users\...\venv\lib\site-packages\django\db\models\query.py", line 51, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "C:\Users\...\venv\lib\site-packages\django\db\models\sql\compiler.py", line 1169, in execute_sql
    cursor.execute(sql, params)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 78, in _execute
    self.db.validate_no_broken_transaction()
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\base\base.py", line 448, in validate_no_broken_transaction
    "An error occurred in the current transaction. You can't "
django.db.transaction.TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.
Traceback (most recent call last):
  File "C:\Users\...\venv\lib\site-packages\django\db\migrations\executor.py", line 229, in apply_migration
    self.record_migration(migration)
  File "C:\Users\...\venv\lib\site-packages\django\db\migrations\executor.py", line 244, in record_migration
    self.recorder.record_applied(migration.app_label, migration.name)
  File "C:\Users\...\venv\lib\site-packages\django\db\migrations\recorder.py", line 86, in record_applied
    self.ensure_schema()
  File "C:\Users\...\venv\lib\site-packages\django\db\migrations\recorder.py", line 63, in ensure_schema
    if self.has_table():
  File "C:\Users\...\venv\lib\site-packages\django\db\migrations\recorder.py", line 56, in has_table
    tables = self.connection.introspection.table_names(cursor)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\base\introspection.py", line 52, in table_names
    return get_names(cursor)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\base\introspection.py", line 47, in get_names
    return sorted(ti.name for ti in self.get_table_list(cursor)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\sqlite3\introspection.py", line 77, in get_table_list
    ORDER BY name""")
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 78, in _execute
    self.db.validate_no_broken_transaction()
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\base\base.py", line 448, in validate_no_broken_transaction
    "An error occurred in the current transaction. You can't "
django.db.transaction.TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manage.py", line 23, in <module>
    main()
  File "manage.py", line 19, in main
    execute_from_command_line(sys.argv)
  File "C:\Users\...\venv\lib\site-packages\django\core\management\__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "C:\Users\...\venv\lib\site-packages\django\core\management\__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "C:\Users\...\venv\lib\site-packages\django\core\management\commands\test.py", line 23, in run_from_argv
    super().run_from_argv(argv)
  File "C:\Users\...\venv\lib\site-packages\django\core\management\base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "C:\Users\...\venv\lib\site-packages\django\core\management\base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "C:\Users\...\venv\lib\site-packages\django\core\management\commands\test.py", line 55, in handle
    failures = test_runner.run_tests(test_labels)
  File "C:\Users\...\venv\lib\site-packages\django\test\runner.py", line 725, in run_tests
    old_config = self.setup_databases(aliases=databases)
  File "C:\Users\...\venv\lib\site-packages\django\test\runner.py", line 645, in setup_databases
    debug_sql=self.debug_sql, parallel=self.parallel, **kwargs
  File "C:\Users\...\venv\lib\site-packages\django\test\utils.py", line 183, in setup_databases
    serialize=connection.settings_dict['TEST'].get('SERIALIZE', True),
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\base\creation.py", line 79, in create_test_db
    run_syncdb=True,
  File "C:\Users\...\venv\lib\site-packages\django\core\management\__init__.py", line 181, in call_command
    return command.execute(*args, **defaults)
  File "C:\Users\...\venv\lib\site-packages\django\core\management\base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "C:\Users\...\venv\lib\site-packages\django\core\management\base.py", line 89, in wrapped
    res = handle_func(*args, **kwargs)
  File "C:\Users\...\venv\lib\site-packages\django\core\management\commands\migrate.py", line 246, in handle
    fake_initial=fake_initial,
  File "C:\Users\...\venv\lib\site-packages\django\db\migrations\executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "C:\Users\...\venv\lib\site-packages\django\db\migrations\executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "C:\Users\...\venv\lib\site-packages\django\db\migrations\executor.py", line 230, in apply_migration
    migration_recorded = True
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\sqlite3\schema.py", line 35, in __exit__
    self.connection.check_constraints()
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\sqlite3\base.py", line 329, in check_constraints
    violations = cursor.execute('PRAGMA foreign_key_check').fetchall()
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\utils.py", line 78, in _execute
    self.db.validate_no_broken_transaction()
  File "C:\Users\...\venv\lib\site-packages\django\db\backends\base\base.py", line 448, in validate_no_broken_transaction
    "An error occurred in the current transaction. You can't "
django.db.transaction.TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.

Verbose Name, Help Text appears as references and not actual text

Nice Library, the verbose name and help text appears as references and not actual text on many fields.

I'm considering heavily relying on this project for ArchiveBox.io and want to learn more

Hi @boxine team!

Thanks for making this awesome Huey monitor open source! It's clearly one of the best options for feature-complete job/worker handling in the Django ecosystem right now, and I'm sure it will grow quickly!

I have a large existing Django project https://github.com/ArchiveBox/ArchiveBox and I'm looking for a new async job system to move our project onto. Huey + your monitor is our highest candidate right now as it supports SQLite, nested tasks with hierarchy and global mutexes, live progress info, and provides a nice UI in the django admin.

If we do this refactor, we'll likely become big contributors to this project (and I'd be happy to send a subset of our donations your way).

I'd love to chat with any of the devs who've worked on this project to get more context on your goals with it, hear about any lessons you've learned with Huey while building this, and see how we can help out if it's a good fit.

If you're down with a quick 20min video call, that would be ideal https://calendly.com/nicksweeting/choose-a-time, otherwise happy to just continue the discussion here too.

Database queries take too much time when loading the task list

For the single list page of Task model in Admin, it takes 300+ queries for fetching query set.

I have eliminated a few columns and make some query optimizations, then the number of queries drops to only 11. Now it's smooth to load the list page.

I wish subsequently this project can have some optimization.

Live output of running task

Currently all web pages are static. Would be great if running task will be have live updating pages...

Add more info for huey

Hello, would it be possible to add a view to this package that would display info about huey ?

Like connection, queue size, schedule size etc. Unless there's a way to see that info and I've missed it....

New release?

Thank you again for this lib :)

I would like to see this fix soon be integrated.
#45

Could you please provide a new release or are you waiting for the completion of a new feature first?

Add a way to unlock tasks

TODO: Expand test matrix with new Django and Python versions

Find a better solution for died Huey worker...

In #10 we add a work-a-round for died Huey worker.

But this is no a good solution, because non affected, running tasks will also get a "unknown" signal :(

Setting DEFAULT_TASK_CUMULATE_PROGRESS in app settings

Hi Jedie,

I'm coming back to you about one of the first topics we discussed together a few years back.

For you, it is important to have cumulate_progress = True because of the nature of the tasks you're using.
So you decided to set cumulate_progress = True by default when creating a TaskModel object.

for me, it is the contrary, it is important to have cumulate_progress = False because of the nature of the tasks I'm using.

In most application, it will be either one way, or the other, but very rarely both ways in the same application.

What would you say about introducing a constant in the app.settings where we could define DEFAULT_TASK_CUMULATE_PROGRESS to be either True or False

in order to keep retrocompatibility, we could have cumulate_progress = settings.DEFAULT_TASK_CUMULATE_PROGRESS or True if there is no DEFAULT_TASK_CUMULATE_PROGRESS in settings.py

we would also use this as the default value for ProcessInfo.cumulate2parents

Would that be something ok for you ?

Collect information about long running tasks...

We should implement a kind of https://pypi.org/project/tqdm/ for long running task like: "Do something with 1000 items"...

Can we so something like the concept of tqdm and use the django cache to collect these kind of progess information?

[Question] Does this mean, dhm use the chosen database as the result storage and redis as the broker/queue?

I use celery with django. want to come over to huey.

in celery, it separates broker/queue from result storage.

so by using django-huey-monitor (dhm) are you making huey use the database purely for result storage? and leave redis to handle the broker/queue?

FieldError at /admin/huey_monitor/taskmodel/ID/change/

Environment:


Request Method: GET
Request URL: http://127.0.0.1:7000/admin/huey_monitor/taskmodel/e2fe16e4-2d57-45b5-95ae-db99b01b4659/change/

Django Version: 3.2.21
Python Version: 3.9.18
Installed Applications:
[...
 'bx_django_utils',
 'huey_monitor',
..]

Traceback (most recent call last):
  File "D:\Users\x\mambaforge\envs\production\lib\site-packages\django\contrib\admin\options.py", line 710, in get_form
    return modelform_factory(self.model, **defaults)
  File "D:\Users\x\mambaforge\envs\production\lib\site-packages\django\forms\models.py", line 563, in modelform_factory
    return type(form)(class_name, (form,), form_class_attrs)
  File "D:\Users\x\mambaforge\envs\production\lib\site-packages\django\forms\models.py", line 276, in __new__
    raise FieldError(message)

During handling of the above exception (Unknown field(s) (task_hierarchy_info, human_progress_string) specified for TaskModel), another exception occurred:
  File "D:\Users\x\mambaforge\envs\production\lib\site-packages\django\core\handlers\exception.py", line 47, in inner
    response = get_response(request)
  File "D:\Users\x\mambaforge\envs\production\lib\site-packages\django\core\handlers\base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "D:\Users\x\mambaforge\envs\production\lib\site-packages\django\contrib\admin\options.py", line 616, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File "D:\Users\x\mambaforge\envs\production\lib\site-packages\django\utils\decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "D:\Users\x\mambaforge\envs\production\lib\site-packages\django\views\decorators\cache.py", line 44, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "D:\Users\x\mambaforge\envs\production\lib\site-packages\django\contrib\admin\sites.py", line 232, in inner
    return view(request, *args, **kwargs)
  File "D:\Users\x\mambaforge\envs\production\lib\site-packages\django\contrib\admin\options.py", line 1660, in change_view
    return self.changeform_view(request, object_id, form_url, extra_context)
  File "D:\Users\x\mambaforge\envs\production\lib\site-packages\django\utils\decorators.py", line 43, in _wrapper
    return bound_method(*args, **kwargs)
  File "D:\Users\x\mambaforge\envs\production\lib\site-packages\django\utils\decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "D:\Users\x\mambaforge\envs\production\lib\site-packages\django\contrib\admin\options.py", line 1540, in changeform_view
    return self._changeform_view(request, object_id, form_url, extra_context)
  File "D:\Users\x\mambaforge\envs\production\lib\site-packages\django\contrib\admin\options.py", line 1574, in _changeform_view
    ModelForm = self.get_form(
  File "D:\Users\x\mambaforge\envs\production\lib\site-packages\django\contrib\admin\options.py", line 712, in get_form
    raise FieldError(

Exception Type: FieldError at /admin/huey_monitor/taskmodel/e2fe16e4-2d57-45b5-95ae-db99b01b4659/change/
Exception Value: Unknown field(s) (task_hierarchy_info, human_progress_string) specified for TaskModel. Check fields/fieldsets/exclude attributes of class CustomTaskModelAdmin.

admin.py

from django.contrib import admin
from huey_monitor.models import TaskModel, SignalInfoModel
from huey_monitor.admin import TaskModelAdmin, SignalInfoModelAdmin

admin.site.unregister(TaskModel)
admin.site.unregister(SignalInfoModel)



class CustomTaskModelAdmin(TaskModelAdmin):

    def has_view_permission(self, request, obj=None):
        return request.user.is_superuser

    def has_delete_permission(self, request, obj=None):
        return request.user.is_superuser

    def has_change_permission(self, request, obj=None):
        return request.user.is_superuser

    def has_add_permission(self, request, obj=None):
        return request.user.is_superuser

    def has_module_permission(self, request):
        return request.user.is_superuser


admin.site.register(TaskModel, CustomTaskModelAdmin)

class CustomSignalInfoModelAdmin(SignalInfoModelAdmin):

    def has_view_permission(self, request, obj=None):
        return request.user.is_superuser

    def has_delete_permission(self, request, obj=None):
        return request.user.is_superuser

    def has_change_permission(self, request, obj=None):
        return request.user.is_superuser

    def has_add_permission(self, request, obj=None):
        return request.user.is_superuser

    def has_module_permission(self, request):
        return request.user.is_superuser


admin.site.register(SignalInfoModel, CustomSignalInfoModelAdmin)

Reviewed PR waiting for merge for 1 month now

Hi, could you proceed with the merge of #44 ?

One month has passed since linting preferences have been adapted according to your preferences

Thanks in advance

django.db.utils.OperationalError: SSL SYSCALL error: EOF detected

Hi,

I'm testing this module with --worker-type process and 5 workers.

Huey consumer started with 5 process, PID 338935 at 2021-04-01 09:21:11.148253
Scheduler runs every 1 second(s).
Periodic tasks are enabled.
The following commands are available:
+ openrouteservice.tasks.isochrone_from_layer_task
Checking periodic tasks
Sleeping for 0.9916734490107046
calling startup hook "startup_handler"
calling startup hook "startup_handler"
calling startup hook "startup_handler"
calling startup hook "startup_handler"
calling startup hook "startup_handler"
startup hook "startup_handler" failed
Traceback (most recent call last):
  File "/home/ale/.virtualenvs/g3wsuite/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.OperationalError: SSL error: decryption failed or bad record mac


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ale/.local/lib/python3.8/site-packages/huey/consumer.py", line 94, in initialize
    startup_hook()
  File "/home/ale/.virtualenvs/g3wsuite/lib/python3.8/site-packages/huey_monitor/tasks.py", line 78, in startup_handler
    for task_model_instance in qs:
  File "/home/ale/.virtualenvs/g3wsuite/lib/python3.8/site-packages/django/db/models/query.py", line 274, in __iter__
    self._fetch_all()
  File "/home/ale/.virtualenvs/g3wsuite/lib/python3.8/site-packages/django/db/models/query.py", line 1242, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/home/ale/.virtualenvs/g3wsuite/lib/python3.8/site-packages/django/db/models/query.py", line 55, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/home/ale/.virtualenvs/g3wsuite/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1142, in execute_sql
    cursor.execute(sql, params)
  File "/home/ale/.virtualenvs/g3wsuite/lib/python3.8/site-packages/django/db/backends/utils.py", line 99, in execute
    return super().execute(sql, params)
  File "/home/ale/.virtualenvs/g3wsuite/lib/python3.8/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/ale/.virtualenvs/g3wsuite/lib/python3.8/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/ale/.virtualenvs/g3wsuite/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/home/ale/.virtualenvs/g3wsuite/lib/python3.8/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/ale/.virtualenvs/g3wsuite/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.OperationalError: SSL error: decryption failed or bad record mac

startup hook "startup_handler" failed
Traceback (most recent call last):
  File "/home/ale/.virtualenvs/g3wsuite/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.OperationalError: server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.

Note that the error does not appear when I use a single worker.

badly formed hexadecimal UUID string

Huey has the api to define your own task_id when scheduling a task. If we set a task_id=123 the code in huey_monitor/tasks.py: 53 it is forced the task_id to be a uuid.
If we pass the task_id = 123 in the schedule task function, Huey_monitor signal will fail with error

 ValueError
badly formed hexadecimal UUID string

boxine / django-huey-monitor Goto Github PK

django-huey-monitor's People

Contributors

Stargazers

Watchers

Forkers

django-huey-monitor's Issues

admin.py

Recommend Projects

Recommend Topics

Recommend Org