scanner / as_email_service Goto Github PK

• Unauthenticated
• Authenticated
• list
• create
• delete
• put
• get
• patch
• read only fields
• read write fields
• increase order
• decrease order
• move to order

The number of bounces for the email account is increased.

• If the bounce limit is exceeded then the email account is deactivated.
• A bounced email notification is created and delivered to the account (notification also indicates if the email account has been deactivated.

Most of the work dealing with a spam complaint is in the huey task process_email_spam().

Finish implementing this and test it.

When a user sets an email account to delivery type of forward and sets the email address to forward to a notification email is sent to that address.

The notification email includes a URL that when clicked confirms the forwarding and allows it to be set.

Modern times speak of using modern authentication systems. Do not need full oauth2 etc.. just need to let people use passkey's for their account (have to switch an existing account from using a password to a passkey, though.)

Currently deploying with a sqlite db. Use light stream to back this db up to an object store.

Have drone run the python tests when code is pushed.

Just get the basic vue app loading for the as_email index page.

Probably the best experience for user's not receiving email delivery via local delivery is to have the app talk to the IMAP server of their choice and deposit new email in to the folder of their choice on that IMAP server.

• auto-filing for spam email will deposit classified spam in to the specified spam folder
• if the specified folder for email (or spam) does not exist it will be created
• if we fail to deposit the email due to any sort of transient or auth failure we should retry a certain number of times
• apply message filter rules. However if folder does not exist, then it is delivered to inbox.

The initial version lets any email account add any other email account as the alias for emails sent to that email account.

Frequently I need to have things like "web" "root" "abuse", etc be aliased to other email accounts not necessarily owned by the same person. It would be nice if this required a confirmation so that a user could not sit there and basically make their an account an alias for anyone else and thus have a way to spam them with email they do not want to get.

Not too worried about this in the initial release because this service is mainly for friends and family, but if anyone else wants to use this a feature like this is a must.

• Also it would be nice of an email account that is an alias for another email account can turn that aliasing off.

Some inactive email's can be re-activated. Postmark tells us if this is allowed. I suppose for a first pass just make a link to where this can be done in the postmark UI, and make an effective admin interface for searching and reactivating inactive emails.

I do not expect this to get much use early on.

Need to update the serializer to list the aliases of an email account

should be able to remove or add an alias form the email account card.

Should we have both the alias_for and aliases inputs on a card? It might be confusing to have both.
I can see wanting to know what email accounts this email account is an alias for. Also definitely want to know what aliases this email account has.

You can only add alias_for for email accounts you own.
You can remove any alias even if the account that is an alias for your account is not owned by you.

Tested view that accepts incoming bounce notifications from postmark.

• #2
• #4

When postmark notifies us of inactive emails we need to keep track of those (and which ones are activatable)

Make a test for the make_delivery_status_notification function

Basically the work to get Vue loading and running, validate compression, defer's, etc.

People may need to change their user account's email address. Early on can do it by request, but need to automate it.
It needs to be an opt-in confirmation system where we send an email out that has a unique link to confirm the change.
Should we require the view that this link goes to require authentication?

When postmark received a spam complaint they send a message to this webhook.

When this happens we generate a notification message that encapsulates the spam as a rfc822 attachment and deliver it to the email account that was marked as sending the spam.

(in the future we may want to have a 'spam count' on the email account but for the user's of this project being probably only apricot systematic I do not think we have to worry about our users sending spam... but who knows what the future will hold.)

We have opt-in for forwarding with #9. It would be good to also let that email address at any time deactivate that email forwarding.

Maybe hook in to the same system that lets apple email users click a button to 'unsubscribe' from a mailing list.

Right now we are using hyperlinks to link to alias_fors and aliases. This is all fine and dandy except:

1. if you have as an alias or alias_for an email account that you are not the owner of you can not get its info from the rest api. For most users this will not be an issue but for a user that has a need to have other email accounts as aliases the interface becomes rather opaque (for example an email account that is the alias_for for things like "abuse@server" and such.
2. all you can do is add and remove entries from this list so all you need to know is the email address of the account you are an alias for or being aliased to.

So update the views and serializers to give a list of email addresses (and take a list of email addresses.)

When adding an alias or alias_for via the REST interface you can only add email accounts that you own.
However, you can remove any entry (whether or not you are the owner of the email account that that entity represents.)

When things fail due to misconfigurations, or transient failures we need a way to notify users. If we are forward messages, sending them email to their email account is not going to be of any use because we turned forwarding off!

So we want to associate an email address with the user account that owns the email account and use the email system configured in our django project to send them a notification.

When the user sets up their account we should tell them that if they are using a delivery method other than local delivery for their email account's then they should not use an email address hosted by this system, otherwise they may not get notification emails when there are problems (because we can not deliver their emails if that is the problem)

When we get a bounce notification that has "Inactive: True" set it means that postmark is going to prevent us from sending to that address again. If "CanActivate" is True then we can re-activate being able to send to that address.

We had interpreted as being unable to send anywhere.

So we need to mostly change our messaging to a user saying that they are blocked from sending to that address and if CanActivate is true say that this can be changed, please contact the admin of the system at ....

We should just count this as a regular hard bounce instead of deactivating their account.

• GH-32

With how I build this project in docker I make sure that the running service does not have access to write files in the installed python package directories.

So we need to think of a proper way to have django_simple_bulma write its generated css files to a writeable directory that the collectstatic command will find and properly install in the generated staticfiles directory.

• requirements for deployment via docker
• how to create a server, how to setup postmark a postmark server that sends and receives email for the as_email server
• how to deploy an update
• generation of SSL certificates for aiosmtpd
• setting up caddy for forwarding requests to as_email_service

The initial list of email accounts that the user owns.
Implemented using bulma cards and bulma-collapsible.

If there is only one it should be is-active

Just display the email account data. Initial page content will be statically generated by django. Django will also supply json data for all the email accounts.

In the app.js for the page it will create the Vue app and replace all the static data with the data from the json embedded in the page.

When an email account is has its delivery method set to 'fowarding' an email address to forward to is specified. However, due to various typos, failures, etc, that email address may not accept email generating bounces.

Postmark is fairly strict about making sure you do not send a lot of email that bounces.

So for an abundance of caution if email sent to a forwarding address bounces, then we change the email account delivery type from 'forwarding' to 'local delivery.' Also generate an email stating this.

Generate a notification that this has been done.

The rules are:

1. if bounce was from an email account that is forwarding
2. the email address that bounced is the one that the email account is forwarding to.
3. The actions specified in #2 still occur.

• #3

This is more of a project of 'apricot systematic' so we should move it in to that organization.
Need to update the drone build badge if we do that.

As unlikely as it is that I will get participation from other people on this project I believe that a CoC is as necessary as a LICENSE file for any public project (maybe time to make a project related to adding CoC's to all my public repos.)

Now that we know about content-disposition: inline, make the function that generates our forward email create forwards like that.

There is a large swath of bounce notification types. Many will never apply to most users (having to do with email lists, subscriptions, templates, etc). These include permanent bounces (like a bad destination email address or domain), but also include transient ones like unable to reach destination mail server.

So we should only increment the number of bounces for an EmailAccount when it sent an email that caused a non-transient bounce.

We have our own otel receiver.

Add otel support:

• standard django app hooks
• huey tasks
• aiosmtpd daemon

Even in the initial release I have remote users and non-technical users. I need documentation showing how to:

• Configure their email client to be able to send email
• how to set forwarding, imap delivery
• how to configure spam auto-filing
• how to setup message filter rules

For forwarding and and aliasing things like spam auto-filtering and message filter rules are not used so this may be moot for most people.

Postmark notifies us when an email address is "inactive" meaning that it will not let us send emails to that address.

Since we control the smtp relay, we could keep a list of the inactive email and raise an error when someone tries to send to one of them.

Probably better, though, is in the send email functions if one of the addresses is an inactive email address it generates a bounce message to the sender of the email (and removes that email from the addresses being sent to.)

• GH-24
• GH-25
• GH-26

When we get the peername for our connections in the aiosmtpd daemon, it is a docker internal ip address (and all connections come from the same ip address.)

This will break the authenticator's blacklist mechanism of black listing ip addresses that frequently produce failed logins.
Since every connection is coming from the same internal ip address one client getting black listed will black list everyone.

• Unauthenticated
• Authenticated
• list
• create
• delete
• put
• get
• patch
• read only fields
• read write fields
• set_password

Add a drone pipeline for testing and publishing images of the service.

• GH-31
• Notify success/failure of drone job via slack
• on tag on main, build, tag, and publish image to docker repo (ghcr, hub.docker)

django_bulma gives us really one useful thing: the bulma template tag that lets auto-fill a django form with bulma tags.

django_simple_bulma does not give us that. It does give us an easy way to add bulma extensions, and related js files for those extensions.

However django_simple_bulma's templatetag gives us both css and js files in one section and this would not let django_compressor do its work.. so we need to add a template tag to just produce the django_simple_bulma js lines (so we can put those in the right compressor block.)

(If django-bulma only gives us the form parsing tag, and we are only using that in the login pages maybe better to not use django_bulma at all, and for those forms just add the bulma tags manually... if we had a lot of html forms it might be useful, but we are going to be doing almost all our forms inside vue components and probably not using django's template forms stuff there.)

Need a user interface for users.

In the initial version users can:

• GH-46
• GH-47
• configure delivery type of: 1) local delivery, 2) alias, 3) forwarding, 4) imap
• set autofiling of spam, spam score threshold, spam delivery folder
• change user password
• change email account password
• add alias email addresses
• GH-57
• GH-48
• GH-45

We will have otel support, but we also have our own influxdb and grafana setup. We want to present data as metrics for easy use on our existing dashboards.

• telegraf demon as docker compose service. telegraf config file mounted (ro) and created by our deploy process
• have telegraf poke redis for its stats
• expose telegraf service to other services so they can post data to it (to be sent to our influxdb service)
• simple stats: total num inbound, total num outbound, spam reports, blocked reports, also totals by server

When you get too many bounces the EmailAccount is deactivated (deactivated attribute is set to True)

A ticket to collect the kinds of things we need to test for our drf views

• GH-34
• GH-35

If we are using DRYRestPermissions and have denied a user 'write' permission so that they can not create an object (but allow them 'update' permission) then the rest api does NOT return the object metadata in response to OPTIONS.

So for now allow True on method == PUT.

The email account has an "alias_for" m2m relationship that tells you for which other email accounts this email account is an alias.. ie: email sent to this email account will instead be delivered to the email accounts listed in the alias_for field.

In terms of how the object mode is designed this makes sense.

However, the model also defines an "aliases" backwards relationship. This tells you "what aliases this email account" has (which is, IMO, more how people think about this.)

So we should expose these aliases AND we should let people remove/add entries to this field.
NOTE: They can remove any alias listed, but they can only add aliases that are email accounts for which the user is the owner of.

When you create a server, create an EmailAccount for abuse@<server> ..

By default abuse@<server> should forward to the administrators of the entire service.

If an email address that is being sent to by the Server model is "inactive" then postmark will not send emails to that address. We need to post a message to the sender that the email is inactive (and not send it in the first place.

Right now you can turn on or off autofiling of spam to an indicated spam folder. This only has an effect if you are doing local delivery (and eventually imap delivery.)

But you may want to do nothing, let it go through as normal, or you may want to "mute" it so that it never gets delivered. This last bit is similar to what apricot.com email users currently see.

This latter bit is for those people using apricot.com email to forward entirely off-system. In the past we would relay the email, but now we will be forwarding it (and encapsulating is as a RFC822 attachment.)

People who only mildly use their apricot.com email may not care and do not want the sudden deluge of spam so may opt to just filter out entirely spam mail.

Need to have a set of decent looking pages for login/logout/password change/reset

Postmark can tell us, on a bounce, if it has marked a from email address as inactive. When this happens we need to mark the email accout as deactivated (and send appropriate notifications)

The bounce and spam webhooks can tell us if a destination email address is "Inactive" which means that postmark will no longer send emails to that address. When this happens we need to create an InactiveEmail object for that.

Make a django model form for all the attributes of the email account that can be updated from the email-account card.

Submit the contents of the form via PUT/PATCH to the server via the REST endpoint for the email account.

Reset pulls the data from the server via GET and updates the data in the card (including the filled in values for the forms)