mozilla / django-tidings

Library for watching and rolling up events and sending email notifications in response

django-tidings

This project is deprecated. See issue #52 for details.

django-tidings is a framework for sending email notifications to users who have registered interest in certain events, such as the modification of some model object. Used by support.mozilla.org and developer.mozilla.org, it is optimized for large-scale installations. Its features include...

• Asynchronous operation using the celery task queue
• De-duplication of notifications
• Association of subscriptions with either registered Django users or anonymous email addresses
• Optional confirmation of anonymous subscriptions
• Hook points for customizing any page drawn and any email sent

Please see the full documentation at django-tidings.readthedocs.io.

django-tidings's Issues

Need a way to instrument with statsd

Somewhere we can pass a callable or something to, in our case, notify statsd that we've sent out notifications.

CODE_OF_CONDUCT.md file missing

As of January 1 2019, Mozilla requires that all GitHub projects include this CODE_OF_CONDUCT.md file in the project root. The file has two parts:

1. Required Text - All text under the headings Community Participation Guidelines and How to Report, are required, and should not be altered.
2. Optional Text - The Project Specific Etiquette heading provides a space to speak more specifically about ways people can work effectively and inclusively together. Some examples of those can be found on the Firefox Debugger project, and Common Voice. (The optional part is commented out in the raw template file, and will not be visible until you modify and uncomment that part.)

If you have any questions about this file, or Code of Conduct policies and procedures, please reach out to [email protected].

(Message COC001)

Support Django 1.9 - 2.0 in a new release

Kuma is updating to Django 1.11, and django-tidings needs an update to support it. Here's my plan for the next django-tidings release:

• Drop support for Django 1.7 and Python 2.6 (was dropped in 1.2)
• Add a migration for Django 1.8's new column size in EmailField (see #26)
• Drop jingo and convert the unsubscribe templates to Django-style templates (see #32)
• Convert the test app to Celery 4.x, drop django-celery, and add new task functions that can be serialized with JSON (see #33)
• Update the test application to Django 1.8+ compatible settings (like moving TEMPLATE_DEBUG into TEMPLATES, drop patterns, etc.).
• Update for new Django requirements, like explicit on_delete strategies
• Add some compatibility functions for Django 2.0 changes, like User.is_anonymous as a property instead of a function.

I'm new to semantic versioning, so I'm not sure if that is a point release to 1.3 or a breaking 2.0 release. My inclination is 1.3. Update: I'm convinced, it will be a 2.0 release

EventUnion.fire exception: __protected_call__() got multiple values for keyword argument 'self'

With django-tidings 2.0, calling EventUnion.fire raised an exception when loaded by Celery:

TypeError: __protected_call__() got multiple values for keyword argument 'self'
  File "celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "newrelic/hooks/application_celery.py", line 84, in wrapper
    return wrapped(*args, **kwargs)

This appears to be caused by passing self in the kwargs, but self is implied as a class member:

{
    'exclude': <User: username>, 
    'self': <tidings.events.EventUnion object at 0xdeadbeef>
}

Travis CI free usage ends Dec 3; mozilla repos should switch to other CI platforms

We're opening this issue because your project has used Travis CI within the last 6 months. If you have already migrated off it, you can close and ignore this issue.

Travis CI is ending free builds on public repositories. travis-ci.com stopped providingthem in early November, and travis-ci.org will stop after December 31, 2020. To avoid disruptions to your workflows, you must migrate to another CI service.

For production use cases, we recommend switching to CircleCI. This service is already widely used within Mozilla. There is a guide to migrating from Travis CI to CircleCI available here.

For non production use cases, we recommend either CircleCI or Github Actions. There is a guide to migrating from Travis CI to Github Actions available here. Github Actions usage within Mozilla is new, and you will have to work with our github administrators to enable specific actions following this process.

If you have any questions, reach out in #github-admin:mozilla.org on matrix.

Admin search/filter tools

It would be very useful to be able to search for notifications in the admin by username or email address.

Add John.Whitlock to pypi package maintainers

I would like to be able to release the current master to PyPI. I need an existing owner of django-tidings on PyPI to add John.Whitlock to the owners or maintainers lists.

My plans are:

• Add additional Makefile commands for package releases (similar to django-dnt)
• Release version 1.2 (adding Django 1.8 and Python 3 support)
• Drop support for Django 1.7 and earlier, and for South.
• Add support for Django 1.9, 1.10, and 1.11
• Fix #26 (increase email field to 254 characters)
• Release as version 1.3

Archive this repository

Actions

• Add deprecation notice to repo
• Add deprecation notice to docs
• Close existing issues
• Archive the project

Why?

This codebase was primarily used by two Mozilla projects:

• https://github.com/mdn/kuma - Django engine for MDN. Removed in mdn/kuma#7825 in June 2021, when removing the wiki features from the engine
• https://github.com/mozilla/kitsune - Django engine for SUMO. Pulled into the kitsune codebase in mozilla/kitsune#5137, and dependency removed in mozilla/kitsune#5155 in June 2022.

It appears in several branches, which all appear to be forks of kuma or kitsune.

Pass all notify() args through to _activation_email()

That way, the confirmation email can say "Really start watching all English edits?" without running a dictionary attack against the hash of "en-US". :-)

Add a 'prune_watches' management command

It would be nice to be able to prune old watches, and to have that we need a created column, or some other method. We could just make the management command take an ID, and prune all the unconfirmed watches below that.

Update MANIFEST-type stuff

Atm, docs, LICENSE, and README aren't getting included in the egg. Fix that.

Fix Read The Docs integration / webhook

The documentation is no longer updating on merges to master:

https://django-tidings.readthedocs.io/en/latest/

A RTD admin needs to enable the integration:

• https://readthedocs.org/projects/django-tidings/
• https://docs.readthedocs.io/en/stable/webhooks.html

and then a GitHub admin needs to setup the web hook.

Pass all matched watches to _mails()

...rather than just an arbitrary one after the deduplication. 2 motivations:

1. Allow for more complete in-mail explanations of why you're getting the notification.
2. Let _mails() in EventUnions make templating decisions based on what events the user was watching. For example, KB notifications might present a translate link to users watching the RevisionReadyForL10n event but a simple link to the doc's front page if they're watching only a RevisionApproved event.

Drop Python 2.7, Django 1.11 support

Python 2.7 is no longer supported, and Django 1.11 leaves extended support in April 2020. django-tidings should drop support for both in the next major release.

django-tidings has two known users:

• https://github.com/mdn/kuma - On Python 3.8, Django 2.2
• https://github.com/mozilla/kitsune/ - On Python 2.7, Django 1.11, but are working on the Python 3 upgrade. Pinning to an older release during the transition should not be difficult.

DeprecationWarning: Using or importing the ABCs from 'collections'

Tidings imports collections.Sequence instead of collections.abc.Sequence, which raises a DeprecationWarning

/usr/local/lib/python3.8/site-packages/tidings/events.py:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working

I'll toss a PR your way later today.

Upgrade test app to Celery 4, remove django-celery

Celery 4.x has a lot of changes. The ones that impact this project:

• The django-celery library's scope is narrowed to providing a database results backend and Django admin integration. They are no longer recommending it for configuring Celery, but instead to use the built-in integration
  (http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html#django-first-steps) provided in Celery 3.1. This will require a small change in the test app.
• The names of settings have changed. The tool celery upgrade settings proj/settings.py --django can automatically update CELERY_ALWAYS_EAGER to CELERY_TASK_ALWAYS_EAGER, etc.
• JSON replaces pickle as the default serializer. pickle is still supported, but requires enabling it (CELERY_TASK_SERIALIZER, CELERY_RESULT_SERIALIZER, CELERY_ACCEPT_CONTENT). Switching away from pickle will require moving from class methods like Event.fire to a separate task function like fire_event, and adding serialization / deserialization methods.

My proposal is to break this into two releases:

1. A release (like 1.3) that supports Celery 4.0, drops django-celery, and provides both the class-based pickle-using tasks, and new task functions that use JSON. Add deprecation warnings when the pickle versions are used. Users can update to this version, and convert their code to use the new task functions.
2. A release (like 2.0) that drops the class-based tasks.

Support for custom User model

Remove jingo library

Jingo, an adapter for using Jinja2 templates in Django, is deprecated and will not support Django 1.9 or above. I see a few options. Here they are ordered from my top to bottom preference:

1. Convert to Django templates, let users override with Jinja2 templates if desired.
2. Add Django templates, and configure Jinja2 templates to be supported by Django's Jinja2 backend (jinja2 folder, .html extension)
3. Convert to Django's Jinja2 backend, with no Django template variants
4. Require django-jinja, and use its default convention (Jinja templates in templates/ folder with .jinja extension).

I also like django-allauth's solution (partially implemented?) of a ACCOUNT_TEMPLATE_EXTENSION, which will allow users change the templates to .jinja2, .j2, etc. This would make it easier to support django-jinja in different configurations without rewriting the unsubscribe view.

tidings.models.multi_raw isn't hydrating models properly in django 1.5

It appears that something changed in the order of the fields/columns in the results for SQL queries.

Event._users_watching_by_filter does a query that starts with SELECT u.*, w.* .... That query gets passed to multi_raw which returns the users with the field values all scrambled. For example:

ipdb> user.__dict__
{'username': u'[email protected]', 'first_name': u'sha256$f538347e82$5098e89186fd307d4bb6fe29ac476e72cf96175617fa933a9bd6b3d89a8b0946', 'last_name': 1, 'is_active': datetime.datetime(2014, 1, 13, 13, 4, 8), '_state': <django.db.models.base.ModelState object at 0x10ded8e10>, 'email': 1, 'is_superuser': u'', 'is_staff': 1, 'last_login': u'', 'password': u'rrosario', 'id': 179845L, 'date_joined': datetime.datetime(2010, 6, 9, 11, 4, 8)}

Move tests outside app

Having them inside the app means using even a clueful testrunner like django-nose will run tidings' tests unnecessarily when it's installed as part of a larger project.

Decouple from "users" app

We're using the users() test helper at the moment.

Allow mapping from Watches back to Events

Stick a metaclass on Events to register them. Then we can move things like Event.description_of_watch(watch) onto Watch itself, a more natural spelling. It also gives us a more natural way to test what Event a Watch coming out of an EventUnion refers to.

Allow excluding multiple users from a fire()

For instance, on support.mozilla.com, we need to be able to exclude both the revision creator and the approver from the wiki-revision-approved notification. (The creator gets notified separately and regardless of whether he had an applicable Watch or not.)

Email field max_length of 75 is insufficient for some emails

The maximum email length is 254:

http://stackoverflow.com/questions/386294/what-is-the-maximum-length-of-a-valid-email-address

Since Django 1.8, the EmailField defaults to 254:

django/django@7fd55c3

Let tests run in place…

…rather than just inside some other Django app.