git-multimail / git-multimail Goto Github PK

git_multimail used to work fine with Python 2.4, but I just upgraded and now get:

[moy@ensiens hooks]$ python2.4 git_multimail.py
File "git_multimail.py", line 2101
class IncrementalDateTime():
^
SyntaxError: invalid syntax

I also have python 2.7 on the same server, so I've just modified the shell bang to use it, but if it's not too hard, it could be nice to restore 2.4 compatibility.

I spent some time figuring out got I can get to the actual email contents for a failed test - until finally figured out that I should comment out "cleanup" "test". Docs could mention that, or even better, don't delete test repo in case of failures (or if it's too hard, don't delete test repo at the end of test at all).

Also, what about providing more "unit" tests? For example, for #118, what I really need to test is that for both commitList='' and commitList=none result of Config.get_recipients() is the same.

Whereas what's available now are integration tests, which are, well, PITA to work with, as there're many restrictions, etc. For example, test repo has announceList set, add using command-line option -c just adds to it, so I cannot really test "empty string or single 'none'" behavior without tricks, which you (and whoever novice will look into it later) would certainly find dirty tricks. End result is that I can't really contribute a testcase with the changes for #118.

It would be nice to allow some configuration keys to have effect only on some refs, like

[mutimailhook]
    var = val
[multimailhook "refs/heads/foo-*"]
    var = otherVal

See #15 for a discussion on the subject.

I very much like how git-multimail sends individual commits as replies to a top-level "refchange" mail... except when the change is a simple update adding one commit. In that case I have two mails when one would have been more than enough.

Please change this behaviour or at least add an option to disable the "refchange" mail when the change is limited to a ref update adding a single commit. This option would only have an effect when commitlist / refchangelist are not used (or do not differ).

Thank you for your work on git-multimail, it was really needed!

When used on Windows system, the $USER environment is empty. Instead it should use
'$USERNAME' or something like '$USERNAME@$USERDOMAIN'.

For me replacing

class GenericEnvironmentMixin(Environment):
def get_pusher(self):
return self.osenv.get('USER', 'unknown user')
<<<<
with

class GenericEnvironmentMixin(Environment):
def get_pusher(self):
return self.osenv.get('USERNAME', 'unknown user')

did the trick.

It currently just says

Copyright (c) 2012,2013 Michael Haggerty

At least the Zimbra webmail considers that

Date: <Anything>

in the body of a message corresponds to the beginning of a quoted message, and hides this line and everything below by default (including the patch, which is the point of sending a commit email ...). Changing the default format to something like

AuthorDate: ...

(Or Authored on: ...)

would work around this behavior of Zimbra. May also benefit some other users.

Even after 46ba7f6, when git_multimail.py is invoked with a fixed USER, I still mange to get "artagnon" (my $USER) in some of the test output. What is going on?

Hello,

I would like it if I could configure certain paths in the git repository that would not be sent as diffs in email. In particular, I'm wanting to keep certain secret material (keys, passwords) in a specific directory in a git repository, but when I push to the repository, I don't want the normal commit diff email to be sent out because it will include copies of those keys. I do want the commit diffs for the rest of the repository, as those are quite useful to see, but just omit certain paths (like .gitignore).

Another option would be to add openpgp encryption to the mails, but that sounds a lot harder :)

See #118 for background. We tried to use multimailhook.refFilterInclusionRegex option, and had gitolite barf about suspicious chars in option value (a regex). So, we had to adjust UNSAFE_PATT setting in gitolite.rc (see patch below). Again, maybe worth mentioned that in gitolite.rc (just mention that UNSAFE_PATT may need to be adjusted in that file to give user a direction, and let them research and decide what would be "safe" setting for them).

    gitolite.rc: Override and make more liberal UNSAFE_PATT.

    UNSAFE_PATT appears to be used to validate "config" directives in
    gitolite.conf file (i.e. entries to be added to .git/config of
    gitolite-managed repos). This changes is similar REMOTE_COMMAND_PATT
    override applied previously. Specific cause is the need to specify a
    regex for git-multimail hook.

diff --git a/per-service/git-servers/roles/gitolite/templates/gitolite.rc b/per-service/git-servers/roles/gitolite/templates/gitolite.rc
index 1e74588..d7406e1 100644
--- a/per-service/git-servers/roles/gitolite/templates/gitolite.rc
+++ b/per-service/git-servers/roles/gitolite/templates/gitolite.rc
@@ -190,6 +190,7 @@
 # Allow single quote appear in gitolite commands, useful e.g. with "desc"
 # command.
 $REMOTE_COMMAND_PATT = qr(^[-0-9a-zA-Z._\@/+ :,\%=']*$);
+$UNSAFE_PATT         = qr([`~#\&;<>]);

 # ------------------------------------------------------------------------------
 # per perl rules, this should be the last line in such a file:

The code dealing with strictutf8 is not activated at all in Python 3 so it has to be broken. But the testsuite passes ...

Hello,

I've seen your project and it looks like something I want to include in a project (Submin) that uses a BSD-style License. It won't be fully integrated, because I would like to contribute my changes so other projects can use it too.

However, I could not find any LICENSE information in your project. Would you be willing to add a LICENSE file (preferably BSD/MIT-style, so it's compatible and we can include it)?

Submin is still in Subversion and I would like to convert Submin to git, but one of the things that I would be missing is a commit-email script, so I'm still a bit hesitant to do so. Using your script would help me achieve that goal more easily :-)

Thanks in advance,
Michiel

In recent addition of "quiet" environment variable (commit 64dd424) for suppressing the mailto list, an on existing environment structure is referenced inside Push.send_emails().

~/testing123$ git push origin master
Counting objects: 3, done.
Writing objects: 100% (3/3), 228 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: Traceback (most recent call last):
remote: File "hooks/post-receive", line 90, in
remote: git_multimail.run_as_post_receive_hook(environment, mailer)
remote: File "/home/jge/githook_test/hooks/git_multimail.py", line 2410, in run_as_post_receive_hook
remote: push.send_emails(mailer, body_filter=environment.filter_body)
remote: File "/home/jge/githook_test/hooks/git_multimail.py", line 2361, in send_emails
remote: if not self.environment.quiet:
remote: AttributeError: 'Push' object has no attribute 'environment'
To /home/jge/githook_test/

• [new branch] master -> master

git 1.7.1 does not have "-c" as an option. The doc states "git-multimail is known to work with Git versions back to 1.7.1" which it does not.

Reminder of the discussion in #52 : it would be nice to allow whitespace separated list of regex, and multi-valued variables for multimailhook.refFilter*Regex variables.

Git does not allow git add .git/config (partly for security reason), but it could make sense to have a .git-multimail file tracked within the repository, that git-multimail would read, to allow anyone with write access to change the configuration of git-multimail.

Typical use-case: a server where only the sysadmin has shell access, but where users would want to configure the list of recipients without having to ask the sysadmin.

Obvious drawback: this pollutes the history with configuration changes.

Warning: we should be careful with security and check that no variable read from this file allow executing arbitrary command. It should probably be disabled by default (but can be activated from /etc/gitconfig as needed).

Hello!

At the moment all email addresses listed in multimailhook.mailingList are collected in "To: " header. Is there a way to split it and send one email to each recepient for they not to see all the other recepients in "To: " field?

If there is no built-in way could you please suggest a place in the code where it can be done?

Alexander

Hello,

For my use case, I'd quite like to include two pieces of extra information - a link to the gitweb/cgit interface (which can be useful at times) but more importantly to convert any bug references in commit logs to clickable links.

e.g. I may have fdo#123 or gh#123 or simply just #123 in the log message and I'd like to include full links appended to the bottom of the message, but before the diff.

I figured that simply providing a config option for a filter script that is run after the git log, prior to returning the data to for injection into the email would allow this.

The script would read everything from the start up to the line with three -'s, it would buffer everything up to this point, do some regexp magic and then inject the appropriate extra links appropriately. I figure the filter script is the easiest way to do this without writing specific functionality into multimail itself with complex buglink regexp configs etc (keeping in mind there may be more than one!)

If I were do make such a change would you accept it upstream?

The default mail submission method is sendmail. sendmail defaults to quitting on a line containing only a ".". The -oi flag can be used to suppress that behaviour (it's the "IgnoreDots" sendmail option).

I would suggest to add -oi to the default flags in sendmail mode. I'm not aware of any portability issues with that option.

Hi, if I only set multimailhook.announceList to some email addresses, but left other list-related options unset, always got

remote: The list of recipients for refchangelist is not configured.
remote: Please set one of the following:
remote: "multimailhook.refchangelist"
remote: "multimailhook.mailinglist"

in the case, no email got sent.

but I don't want every commit email being sent, only important tag/release notification.
Did I config anything wrong? Thanks.

FWIW I have setup git-multimail on git.debian.org and many people are now using it. I just got a report of some invalid mail generated by git-multimail. When a user pushed this commit: http://anonscm.debian.org/gitweb/?p=lintian/lintian.git;a=commit;h=3bbaa34ddf253dfdd9aee041e8d571a97e73584e

He got back a bounce saying "550 Message headers fail syntax check" (this is exim giving back this answer) and the only thing that looks weird in the mail is the Reply-to header:

Return-path: <[email protected]>
Received: from nthykier by vasks.debian.org with local (Exim 4.80)
    (envelope-from <[email protected]>)
    id 1V6dV3-0005q4-Qc; Tue, 06 Aug 2013 09:24:17 +0000
To: [email protected]
Subject: [lintian] 02/03: Add generic infrastructure for checking
 common-licenses
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Reply-To: =?utf-8?q?_Bastien_ROUCARI=C3=88S_=3Croucaries=2Ebastien=40gmai?=
 =?utf-8?b?bC5jb20+?=
In-Reply-To: <[email protected]>
References: <[email protected]>
X-Git-Repo: lintian
X-Git-Refname: refs/heads/master
X-Git-Reftype: branch
X-Git-Rev: 3bbaa34ddf253dfdd9aee041e8d571a97e73584e
Auto-Submitted: auto-generated
Message-Id: <[email protected]>
From: Niels Thykier <[email protected]>
Date: Tue, 06 Aug 2013 09:24:17 +0000

Any idea what's going wrong here?

Note that git-format-patch outputs “From: =?UTF-8?q?Bastien=20ROUCARI=C3=88S?= [email protected]” so it definitely encodes the identity/email more wisely.

GitHub already has a builtin hook to send emails, but it is not very convenient and certainly not as flexible as git-multimail. It would be nice to allow a git-multimail service to be hosted outside GitHub, and have it send emails when it receives an event from a webhook.

Once this is done, it would be nice to setup such a service for git-multimail itself, to make it "self-hosted".

Note that this is not as straightforward as it seems to be, since git-multimail would run on a machine that is not where the repository is hosted. The simplest way to deal with this would be to keep a mirror of the repository on the machine running git-multimail.py: the hook would trigger a fetch, and then run git-multimail more or less normally on the mirrored repository. One difficulty is that running git-multimail after the check would break the "new commits detection".

As noted my @mhagger on #59, we should rename it to post-receive.example or find another way to avoid having people use it when they really wanted git_multimail.py.

e.g. for some "reasons" we use our tree for release and dev. developers push their contributions in refs/heads/contrib/*, and we have release branches too (master, and stable). I would like to get email notifications for master and stable only.

Is there an option to have merge requests trigger git-multimail to send out messages? I have noticed that things are not quite right with SourceForge. For the FlightGear project, here is our commitlogs mailing list archive. An example of a commit which did not produce a message is:

• The original commit in a fork (and its branch)
• The merged merge request.
• The merged commit.

It could be that the problem is with the Allura platform SourceForge uses to manage the merging from forks, but I cannot see how this would not trigger git-multimail in a post-receive hook.

Cheers,
Edward

http://permalink.gmane.org/gmane.emacs.diffs/127520 has this From header with an invalid UTF-8 character (it's actually Latin-1 encoded):

From: =?UTF-8?Q?Fabi=E1n_Ezequiel_Gallina?= [email protected]

Note that the Reply-To header has the correct encoding. (You can get the raw article via the NNTP interface.)

It would be nice to have a command like

$ ./git_multimail.py --list-refs
refs/heads/master all-email-sent
refs/heads/branch excluded
refs/heads/long-lived dont-send
...

so that one could debug the multimailhook.refFilter*Regex variables more easily.

Pasted from IRC:

10:37:10 <avar> Trying to do some more hacking on git-multimail
10:37:38 <avar> I tried to roll it out for our main repo at work, but it just hangs on a bunch of git-rev-parse --verify commands
10:37:59 <avar> presumably something to do with the repo having tens of thousands of tags and the behavior trying to detect new commits
10:38:09 <avar> i.e. something like: generate_revision_change_summary
10:38:44 <avar> or something else calling GitObject()
10:38:55 <avar> This was for pushing one commit, ctrl+c'd it after a minute or so.

I haven't debugged this, just noting it here to track it. Seems
git-multimail is really pathological with lots of tags in ways that
the old bash commit script isn't.

The most likely cause is that finding if a commit is "new" is really expensive.

The following construct (at line 2908 as of 7329408) requires python 2.7 or 3.1 to work, and doesn't work under python 2.6:

re_template = r'^\s*#\s*{}\s*$'

The following does work (replace {} with {0}):

re_template = r'^\s*#\s*{0}\s*$'

I'd like to use a login/password to access to my SMTP server.

This may cause issues both in "normal" usage and with gerrit which pass branch names as command-line parameters.

Can anyone help me with this issue?
http://stackoverflow.com/questions/24465137/how-to-enable-git-multimail-hook-in-gitolite

I'm not sure where to look for any error messages.

This should be along the lines of the code proposed in #63: in SMTPmailer,

        elif self.smtpport == 587:
            self.smtp = smtplib.connect(self.smtpserver, self.smtpport)
            self.smtp.ehlo()
            self.smtp.starttls()
            self.smtp.ehlo()

I've noticed that in our group of devs some people love the fact that the diff is emailed, others are only interested in the summary mails and don't want to see the diffs.

Would it be possible to add some kind of mail header that reflects the various types of notifications? People not interested in diffs could then easily filter away those messages:

Following are header suggestions:

X-Git-NotificationType: diff
X-Git-NotificationType: ref_changed_plus_diff
X-Git-NotificationType: ref_changed

ref_changed_plus_diff would cover the new 'fewer emails if single commit' from 18203d0

Reading thru https://github.com/git-multimail/git-multimail/blob/master/doc/gitolite.rst , it might use some editing for clarity. Specific, first line/paragraph is a bit confusing:

git-multimail supports gitolite natively. Setting up git-multimail on a gitolite 3 installation can be done like this:

"supports natively", what does that mean? Perhaps saying "has special support for gitolite (see "custom environment" in README)". Second sentence, ending with colon, seems like stray at all, given the following paragraph.

We use git-multimail on mirror of https://github.com/gcc-mirror/gcc . Commit message there don't follow git guidelines, and are heritage of ChangeLog file format. There're commits like gcc-mirror/gcc@75c3f23 which lead to git-multimail messages like:

      from  75c3f23     * genmatch.c (fprintf_indent): New function.    (operand::gen_transform): Add indent parameter.     (expr::gen_transform, c_expr::gen_transform,    capture::gen_transform): Ditto and use fprintf_indent.  (dt_node::gen, dt_node::gen_kids, dt_node::gen_kids_1): Ditto.  (dt_operand::gen, dt_operand::gen_predicate,    dt_operand::gen_match_op, dt_operand::gen_gimple_expr,  dt_operand::gen_generic_expr, dt_simplify::gen): Ditto.     (decision_tree::gen_gimple): Adjust calls and inde [...]
       new  853a01d   2015-07-09  Vladimir Makarov  <[email protected]>

It would be to have an option to force usage of single first line as summary (vs git convention that summary is separate from details with a blank line, with an extreme case of summary being multi-line).

To add to confusing reports I submitted (#118, #119) here's another one. (And I do understand they're confusing, but working in devops, with bunch of issues and projects in works, I can't get to investigate them in details for days/weeks, so figure I'd submit them as our users face them - I hope that provides an interesting perspective too: users just do everything per docs, and it just doesn't work :-D).

So, we use gitolite (as in the issues above), and have to projects:

1. Uses:

    config multimailhook.refFilterInclusionRegex = "linux-linaro-lsk-v\d\.\d+($|-rt|-android)$"

Notification mails get sent out even for branches not matching the regex.

1. Uses:

     config multimailhook.refFilterExclusionRegex = "/trunk$"

Yes, you guessed - notification mails are still sent for branch named "trunk" (it's SVN-imported git repo).

We have a limited set of committers and we like to see who commited directly in the "From" field. Please make it possible to use that identity in the From field ideally with a simple setting, otherwise at least with wrapper script.

Right now, it's not possible to override get_fromaddr() of the Environment because it doesn't get the details of the commit or from the changes.

We're using git-multimail with gitolite and experiencing few problems. As I'm trying to investigate them further, I'd like to start recording them (as again, there're few (>1) issues). Worst case, it's kind of user error on our side. But even then, it may be worth to mention such issues on git-multmail docs.

So, git-multimail depends on being able to set some options to empty string (e.g. multimailhook.mailingList, and other multimailhook.*List options). We used obvious gitolite.conf syntax:

config multimailhook.commitList = ""

and only later spotted that with the syntax above, gitolite simply doesn't put multimailhook.commitList options to a particular repo's .git/config at all, which of course causes it behave rather different from the expectation.

See discussion here: #56 (comment)

Currently, we allow sending HTML with syntax coloring for patches for commit messages. It would be tempting to allow the template message to include actual HTML too (e.g. <a href="...">View the code online</a>), but currently all the content of the message is HTML-escaped, hence this is not possible.

I am getting the following stack trace when doing git push origin <branchname>, but subsequently running the same command gives no error, tells me Everything up-to-date. The push is not actually happening as I confirmed from the origin server.

Edit: The push is happening, but I get the stack trace nevertheless.

Counting objects: 3, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 312 bytes | 0 bytes/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Traceback (most recent call last):
remote:   File "hooks/post-receive", line 72, in <module>
remote:     environment = git_multimail.GenericEnvironment(config=config)
remote:   File "/var/tmp/git-multimail/git_multimail.py", line 2060, in __init__
remote:     super(ProjectdescEnvironmentMixin, self).__init__(**kw)
remote:   File "/var/tmp/git-multimail/git_multimail.py", line 1918, in __init__
remote:     **kw
remote:   File "/var/tmp/git-multimail/git_multimail.py", line 1959, in __init__
remote:     **kw
remote:   File "/var/tmp/git-multimail/git_multimail.py", line 1926, in __init__
remote:     super(FQDNEnvironmentMixin, self).__init__(**kw)
remote:   File "/var/tmp/git-multimail/git_multimail.py", line 1889, in __init__
remote:     config=config, **kw
remote:   File "/var/tmp/git-multimail/git_multimail.py", line 2031, in __init__
remote:     **kw
remote:   File "/var/tmp/git-multimail/git_multimail.py", line 1967, in __init__
remote:     super(PusherDomainEnvironmentMixin, self).__init__(**kw)
remote:   File "/var/tmp/git-multimail/git_multimail.py", line 1728, in __init__
remote:     config=config, **kw
remote:   File "/var/tmp/git-multimail/git_multimail.py", line 1719, in __init__
remote:     super(ConfigEnvironmentMixin, self).__init__(**kw)
remote:   File "/var/tmp/git-multimail/git_multimail.py", line 1897, in __init__
remote:     super(MaxlinesEnvironmentMixin, self).__init__(**kw)
remote:   File "/var/tmp/git-multimail/git_multimail.py", line 1854, in __init__
remote:     super(FilterLinesEnvironmentMixin, self).__init__(**kw)
remote:   File "/var/tmp/git-multimail/git_multimail.py", line 1998, in __init__
remote:     raise ConfigurationException('No email recipients configured!')
remote: git_multimail.ConfigurationException: No email recipients configured!
To [email protected]:core-services/core-shell.git
   c3b06f2..bf2999b  master -> master

What am I missing? I am using Git 2.3.1 on OS X and the origin is served by Gitlab.

Version 1.2dev-d64e85e93bd68f2cfd4d5896d125e7d450bdd7a1 (testing under Gerrit).

A UnicodeDecodeError is raised with my commits. If I change my name to Sebastien (s/é/e), it works.

Traceback (most recent call last):
  File "/srv/gerrit/hooks/git_multimail.py", line 3506, in <module>
    main(sys.argv[1:])
  File "/srv/gerrit/hooks/git_multimail.py", line 3498, in main
    run_as_update_hook(environment, mailer, refname, oldrev, newrev, options.force_send)
  File "/srv/gerrit/hooks/git_multimail.py", line 3208, in run_as_update_hook
    push.send_emails(mailer, body_filter=environment.filter_body)
  File "/srv/gerrit/hooks/git_multimail.py", line 3130, in send_emails
    rev.recipients,
  File "/srv/gerrit/hooks/git_multimail.py", line 1838, in send
    self.f.writelines(lines)
  File "/srv/gerrit/hooks/git_multimail.py", line 1463, in generate_combined_email
    for line in self.generate_email(push, body_filter, values):
  File "/srv/gerrit/hooks/git_multimail.py", line 795, in generate_email
    for line in self.generate_email_header(**extra_header_values):
  File "/srv/gerrit/hooks/git_multimail.py", line 1072, in generate_email_header
    self.header_template, **extra_values
  File "/srv/gerrit/hooks/git_multimail.py", line 746, in expand_header_lines
    value = addr_header_encode(value, name)
  File "/srv/gerrit/hooks/git_multimail.py", line 450, in addr_header_encode
    header_name="[email protected]"
  File "/usr/lib/python2.7/email/header.py", line 183, in __init__
    self.append(s, charset, errors)
  File "/usr/lib/python2.7/email/header.py", line 267, in append
    ustr = unicode(s, incodec, errors)

Debian maintainers often use Git repository to maintain their packages and they like to get commit notices to follow changes in the package. Unfortunately, they also have to merge the main git repository when they update the package to a new upstream version of the software. At that point, git-multimail will possibly generate hundreds of mail for the upstream commits that got merged.

This is very annoying and it would be nice if git-multimail could be told to only send the summary mail when the number of individual commit emails exceeds a pre-defined limit.

At Git Merge 2015 we had a discussion that Setting up Git to send proper email patches is complicated, and that it would be nice to have a tool that sends [PATCH] emails to a configurable email address when commits are pushed to a repository. I wonder whether doing so could be in the scope of git-multimail. Opinions?

In the best case there also would be an easy way to integrate this into GitHub.

#85 stores them cleartext in the config file.

Is it possible to send a html emails with colored diffs - this would greatly improve readability

Hi,

It seems that the mails that multimail sends out are type text/plain.
I think it would be easier to use the mails if they were type multipart, with the log message and header text in a text/plain part, then the diff as an inline text/x-diff attachment. This enables some MUAs to add syntax highlighting for the diff, makes it easier to extract just the diff, etc. Perhaps this could be optional if some people prefer the text/plain style.

Thanks.

First of all, great project. I've been using it for submin (github.com/mjholtkamp/submin, it's a web admin frontend for git and subversion) and I've been very happy with it!

I've noticed that if you push a large set of commits, most e-mail clients (I've tested Thunderbird and Mail.app) will not sort the mails sent by git-multimail correctly. This is probably due to their sorting by date. (They are correctly grouped together in 'threads' or 'conversations').

Reading the commits/changesets in a-chronological order is confusing, especially if the 'summary' email is not the first e-mail in the 'conversation' (e.g. in Mail.app).

Chaining references (In-Reply-To points to one changeset before, instead of to the summary) would probably fix some e-mail clients, but not all (otherwise Mail.app would show the summary e-mail as the first one in the 'conversation').

How about forcing the sort order by artificially changing the sent date slightly? Since the resolution of the date/time is in seconds, we could increase each sequential message by 1 second (faking a 'delay' in when they were sent).

Possible problems: e-mails are sent in the future.

Is this a problem? Even if you send 59 changesets (+ 1 summary), the difference with the real sending time is only one minute. It's not correct of course, but I think a couple of seconds of difference is a small price to pay for readability.

What do you think?