Allow git-multimail to run as a webhook for GitHub about git-multimail

[MikeMcQuaid]

Hi @rubys! I work on GitHub's webhook team and @mhagger asked me if I could take a look at your approach. I just wanted to chime in and say that the approach seemed sane and thorough from my perspective. Do shout if we can help at all.

Some more general comments here (my personal open-source opinions rather than GitHub's 😉):

Note that this is not as straightforward as it seems to be, since git-multimail would run on a machine that is not where the repository is hosted. The simplest way to deal with this would be to keep a mirror of the repository on the machine running git-multimail.py: the hook would trigger a fetch, and then run git-multimail more or less normally on the mirrored repository. One difficulty is that running git-multimail after the check would break the "new commits detection".

@moy I've found the Heroku deploy button to be great for making it easy for end-users to get up-and-running with stuff like this: https://devcenter.heroku.com/articles/heroku-button

For an example, see https://github.com/mikemcquaid/HookHand#usage and https://github.com/mikemcquaid/HookHand/blob/master/app.json

You and @rubys may find the https://github.com/mikemcquaid/HookHand repo generally interesting (if you'll excuse the self-plug) as a way of receiving webhooks and calling arbitrary scripts like this.

Hope some of this helps; please ask more questions if it does not! Good luck!

[rubys]

One difficulty is that running git-multimail after the check would break the "new commits detection".

I'll note that at least in the case of GitHub, a list of commits is provided by the pushEvent.

[rubys]

This looks to be pretty straightforward. Unless I'm missing something as the github webhook seems to have all of the necessary information. Proof of concept.

[moy]

The difficulty would probably not be the management of the GitHub hook itself (never worked with it, but your proof of concept is rather convincing that this can be done reasonably easily), but the integration with git-multimail. Existing environments for git-multimail just obtain the information about the initial and new commit pointed to by revisions and the user, and the rest of git_multimail.py works exactly the same way regardless of the origin of the information, on the local repository (calling git log and git diff to generate the content of the emails). There's nothing really hard, but it doesn't fit very well in the current code.

No time to work on the feature for now, but pull-requests are obviously welcome ;-).

[rubys]

That's essentially what I'm working on: a GitHub hook that will obtain the information about the initial and new commit; fetch the relevant refspec from GitHub so that the relevant content is local; and then call the post-receive hook on the local repository who won't be aware of any of this. Along the way, I'll stash other useful information provided by the GitHub webhook into environment variables so that it is available should a customized post-receive git hook desire to make use of it.

I intend to provide a pull-request with both code and docs.

[rubys]

Not ready yet for a pull request, but ready for feedback.

Docs: https://github.com/rubys/git-multimail/blob/github-webhook/doc/github.rst
Code: https://github.com/rubys/git-multimail/tree/github-webhook/git-multimail

I imagine that a multimailhook.environment of github could go a long way towards reducing the amount of configuration that projects will require (for example, extracting the identity of the pusher from the WEBHOOK_PUSHER environment variable).

I haven't looked at your test setup, or even thought much yet about how this code can be tested.

[moy]

a GitHub hook that will obtain the information about the initial and new commit; fetch the relevant refspec from GitHub so that the relevant content is local; and then call the post-receive hook on the local repository

Actually, I didn't realize this when I opened the issue, but your approach would be completely independent from git_multimail.py. It would just be a mirroring tool plus a GitHub hooks -> Git hook tool, and could be helpful to hooks other than git_multimail.py. That's cool.

I'm actually wondering whether this already exists.

No time to work on this before mid-december or later, but thanks @rubys for working on this, and thanks @MikeMcQuaid for the feedback.

[rubys]

@MikeMcQuaid:

Do shout if we can help at all

The larger topic I'd like to explore (over time) is true bi-directional synchronization. I'm not looking for bullet proof (after all, networks and/or servers do go down from time to time), but able to detect the times when manual intervention is required and notify the right people to take corrective action.

@moy:

Actually, I didn't realize this when I opened the issue

To be fair, I didn't know where I would end up when I started looking into this either. :-)

[MikeMcQuaid]

The larger topic I'd like to explore (over time) is true bi-directional synchronization. I'm not looking for bullet proof (after all, networks and/or servers do go down from time to time), but able to detect the times when manual intervention is required and notify the right people to take corrective action.

@rubys Probably being stupid here but I'm not totally sure I understand what you're wanting here. Can you elaborate a bit? Note I'm not familiar with git-multimail so please talk to me like an idiot 😀

[rubys]

Apache Software Foundation experiment: http://s.apache.org/2Ky