svnmailer - documentation

en

• svnmailer

• Code Documentation
• Change Log

General Comments

The svnmailer is an extensible subversion commit notification tool. Its purpose in the first place is to create human readable commit mails and correctly encoded other notifications. In order to accomplish this, the content may be recoded to fit the requirements of the supported standards like MIME or XML 1.0. Note that this does not change the meaning of the actual content. It's more like encoding a < character as < in an HTML or XML document.

That means that you always get valid and readable mails, but you SHOULD NOT expect to be able to simply copy the diffs from the generated mails and apply them with the patch program. Use svn diff or ViewCVS diffs for such tasks. The svnmailer is able to generate the proper ViewCVS urls and place them quite handy near the diffs in the notification mails.

Nevertheless, there are always people doing weird things, so the svnmailer provides the possibility to generate multipart mails, which contain the opaque diff data. Note however, that the using the diffs directly from the multipart tracking mails has a number of disadvantages, including the following:

1. The mails may be badly modified by gateways, because they are not protected by a proper transfer encoding
2. The mails may not contain the whole change. There are options to omit diffs for special actions (often diffs for deletions are turned off). Further diffs of binary files are always omitted.
3. Keyword and newline expansion in subversion happens client side, so the mail diff may be another one than the stuff svn diff generates.
4. Last but not least email is an unreliable medium. The typical case to lose tracking mails is that the mailing list rejects mails that are greater than a certain limit. So how would you apply a diff, which you don't even have received? Sure, one could truncate the mails properly, but then issue 2 applies...

Installation

Before installing the svnmailer package make sure, that you meet the following requirements:

• Python 2.3 or later
• A POSIX compatible platform or Win32
• The subversion bindings for python

The svnmailer is a pure python package, which is packed using distutils. So the installation on your system is fairly simple. First download the package and make sure you've checked the integrity of the downloaded file. There are a detached GPG signature (*.asc) and signed hash files (*.md5 and *.sha1), which you can use for this task.

After unpacking the archive file change into the svnmailer-1.0.9 directory and follow the typical python procedure:

Now there should be two things installed, the svn-mailer command line script and svnmailer package itself. The package is copied to the "proper" location, where python finds it by default. The location of the script depends on the OS and the python installation. For example, on linux it typically installs into /usr/bin or /usr/local/bin. For customizations please refer to the related python documentation.

The next step is to create a configuration file. After you've done that, you can configure your repository to let the svnmailer do its work.

Config File Design

In short: the configuration file controls, who gets the notifications for which path in which repository. Further it defines the basic script parameters like how to send the notification mail, which diff program to use (if any) and so on. It is supposed to be compatible to the config of the original mailer.py script.

Syntactical Elements

The file is a plain text file in an INI like format as defined by the standard python ConfigParser module. Basically it consists of several sections, that are started with a line containing:

and finished at the next [section] or the end of the file. Values are defined that way:

As you might have guessed, comments are preceded by the hash character (#). Empty lines (or lines containing only whitespaces) are ignored. Note that section headings, comments and value definitions have to start at the first column, because the configuration parser treats (non-empty) lines starting with spaces or tabs as continuations of the previous line (like in mail headers).

Semantics

The sections in your config file are processed by the svnmailer as follows: The [general] section contains the basic script parameters. The [defaults] section contains default values for the group sections. All values that are not defined in a selected notification group are taken from the defaults. [defaults] is optional. Further you can have a [maps] section, which specifies value mapping tables (which may define further sections, too). All sections not recognized otherwise define notification groups. The names of these sections don't care (except for debugging purposes). They just have to be unique within the config and may not be named defaults, general or maps, of course. If there is no separate group configuration, the defaults will be applied. So a minimal config is:

If you call the svnmailer with this config, it will generate diffs for every possible action at any path for any supplied repository and write a notification message to stdout.

Group Selection

When the svnmailer is called, it reads the config file and selects all groups, which should be notified of that particular event (commit or revprop change). Which groups are selected, is determined by the for_repos, for_paths, exclude_paths and ignore_if_other_matches options. Since the svnmailer tries to minimize the number of notification mails, each of these selected groups gets one mail at maximum per event (except they are configured for huge mail splitting). Furthermore, if the notifications generated for different selected groups are detected to be equal, these groups are merged and just one mail is sent to all of those groups. Currently this detection compares the list of modified paths (that were matched by each group) and several configuration options, which are in particular: show_nonmatching_paths, commit_subject_template, propchange_subject_template, lock_subject_template, unlock_subject_template, commit_subject_prefix, propchange_subject_prefix, lock_subject_prefix, unlock_subject_prefix, max_subject_length, reply_to_addr, diff_command, generate_diffs, browser_base_url, viewcvs_base_url, extract_x509_author, long_mail_action, long_news_action, mail_transfer_encoding, news_transfer_encoding, mail_type, apply_charset_property, show_applied_charset.

Note that there's a subtle difference to the original mailer script. If there are group configurations defined, the svnmailer will never consider the [defaults] section as an additional group to be notified.

[maps] Configuration Section

The [maps] section allows you to specify mapping tables for values that you can't create directly by substitutions or just want to abbreviate via a nick name. For example, a typical configuration of from_addr is:

This is fine as long all authors really do have a mail address at example.org, which local part matches the authentication user. However, a more complex mapping from author to mail address is not possible with this method. In this case, maps can provide a solution. Different map types are planned; for now, the svnmailer only supports plain config maps, which are described in the next section.

Note that not all options are mappable. The exceptions are all options that specify boolean or integer options in addition to the following list: generate_diffs, mail_transfer_encoding, news_transfer_encoding, config_charset, include_config.

Plain Config Maps

All you need is a translation table, which maps the authenticated author to his real address. Something like this:

Further you need to associate the [author table] with the revision author and the from_addr option:

In the above config the [maps] section defines, that the resulting values of from_addr should be mapped using the [author table] section. Now the svnmailer takes the specified from_addr (%(author)s), substitutes the author with, say, john and maps john to doe@example.org. If it doesn't find the value (john) in the mapping table, it is passed through unchanged.

On list options like to_addr, every item is mapped separately. For example:

This excerpt sends all commits to the author itself and an archive account.

[general] Configuration Section

The possible configuration options in [general]

Option Name	Type	Description
config_charset	string	The character encoding of the config file
include_config	List	A list of config files to include
diff_command (deprecated)	command line	The external diff program to use.
sendmail_command	command line	The sendmail compatible command line template
smtp_host	string	The SMTP host[:port] to use
smtp_user	quoted literal	The user used for SMTP authentication
smtp_pass	quoted literal	The password used for SMTP authentication
nntp_host	string	The NNTP host[:port] to use
nntp_user	quoted literal	The user used for NNTP authentication
nntp_pass	quoted literal	The password used for NNTP authentication
debug_all_mails_to	mail addresses	Mails should go only to these fixed addresses
cia_rpc_server (deprecated)	string	CIA XML-RPC tracking server
tempdir	string	The directory to use for temporary files

Though the [general] section defines such basic parameters, it may be finally empty, because there are more or less useful defaults given for each option. In order to make sure, that you didn't forget it, the svnmailer requires at least the section heading to be present in the config.

The different options play together as follows:

• If the diff_command option is given and not empty, it is used to generate the diff output. Otherwise an internal differ is used. The internal differ always generates the unified diff format.
• If the sendmail_command option is given, it is used to send the mail.
• If the smtp_host option is given and sendmail_command is not given or empty, the former is used to open an smtp session to the specified host and deliver the mail via SMTP. If the server requires authentication, you have to supply both smtp_user and smtp_pass. If the server doesn't support authentication, you may not supply the smtp_user option (alternatively just leave it empty).
• If neither sendmail_command nor smtp_host is specified, the notification message is written to stdout (with a small header, which configuration groups were selected). This can be used for debugging purposes.

config_charset

The config_charset option defines, how the svnmailer should interpret the (bytes read as) option values in your config files. It defaults to us-ascii, which means, normally you can leave it just out. But note that there are exceptions. The options of type "quoted literal" are not charset decoded, but taken literally (though possibly unquoted). Furthermore the first item of command line option values (the command itself) may be interpreted literally, too. This depends on the OS and the --path-encoding option. See there for details.

There's a subtle limitation: config_charset should be an US-ASCII based encoding (for example, UTF-16 is a bad choice).

include_config

The include_config option specifies further config files to append to the current one. The config files can be defined using either relative or absolute paths in the filesystem. Relative paths are taken relative to the config file which includes them. The option value is parsed like a command line -- it is split on whitespaces, except when they are quoted. Have a look at the description of diff_command for a detailed explanation of the rules.

One typical use of include_config is to exclude mappings from the main config for maintenance and/or security reasons.

Note that it is not possible (yet?) to override a section by including another file. Each and every section still can only be defined once.

And the following could be the passwords.txt file:

diff_command

The diff_command option in [general] is deprecated. diff_command options defined in group sections are taken in preference of the one defined in [general].

sendmail_command

The sendmail_command option defines the command line template of the program that should be called for sending a mail. The program should expect the mail body on stdin. The stdout channel of the program is closed by the svnmailer, but stderr is passed through the caller of the svnmailer.

In contrast to diff_command there are no substitutions made on the arguments. Instead the final command line is constructed as follows:

1. The command template is split into its arguments following the rules described at diff_command.
2. To specifiy the mail sender the arguments -f and the sender address is attached to the argument list
3. The argument list is further extended with all recipient addresses.

This calling convention is compatible to, for example, sendmail or qmail's sendmail wrapper, hence the name sendmail_command.

As with diff_command no shell metacharacters are interpreted.

For compatibility reasons the sendmail_command option can also be written as mail_command.

smtp_host

The smtp_host option defines the SMTP server to connect in order to send a mail. This option is ignored if sendmail_command is defined and not empty. The option value is a hostname, optionally followed by a colon and a port. If the server supports authentication, you can supply the required credentials via the smtp_user and smtp_pass options.

smtp_user and smtp_pass

These two options are only used if the smtp_host is used by the svnmailer. They define the credentials to be used in the SMTP session when attemting to send mail. If you supply smtp_user, you have to define smtp_pass as well. However, svnmailer just checks for the presence of smtp_user to know, if any credentials should be used.

The utilized smtp library supports the CRAM-MD5, PLAIN and LOGIN authentication mechanisms.

Because of the nature of those two options, the values are not considered to be charset encoded. They are sent literally to the SMTP server. In order to make sure, that spaces and other possibly weird characters are taken literally, you can enclose the actual string in double quotes ("). For double quotes and backslashes inside the quoted string apply the same rules as for command line arguments. Have a look at the description of diff_command for details. Of course, the surrounding quotes and backslash escape characters are stripped before submitting the string to the SMTP server.

nntp_host

The nntp_host option defines the NNTP server to connect in order to post the notification as a news article. The option value is a hostname, optionally followed by a colon and a port. If the server requires authentication, you can supply the credentials via the nntp_user and nntp_pass options.

nntp_user and nntp_pass

These two options are used if a news notification is submitted via an nntp_host. They define the credentials to be used in the NNTP session when attemting to post the article. If you supply nntp_user, you have to define nntp_pass as well.

Similar to smtp_user and smtp_pass the values are not considered to be charset encoded. They are sent literally to the NNTP server. See there for a further explanation of the argument format.

debug_all_mails_to

This one is a real debugging option. It specifies a fixed list of mail addresses, where all notification mails should be sent to -- regardless of the to_addr or bcc_addr templates of the selected groups. The addresses of the overridden recipients are sent along with the mail using the X-Supposed-Recipients header.

cia_rpc_server

The cia_rpc_server in [general] if deprecated. cia_rpc_server options defined in group sections are preferred to the ones defined in [general].

tempdir

The tempdir option defines a directory to use for temporary files. By default or if the specified directory is empty this temp directory is chosen by python's tempfile module.

Group Configuration Sections

The possible configuration options in group sections and [defaults]

Option Name	Type	Description
for_repos	regex	Matches the repository file path
for_paths	regex	Matches the virtual path inside a/the repository
exclude_paths	regex	Excludes paths that might be matched with for_paths
ignore_if_other_matches	boolean	Determines, whether the group should be ignored, if any other group matches this path
show_nonmatching_paths	token	Defines how to deal with changed paths that don't belong to the group
commit_subject_template	template	The mail subject template for normal commits
propchange_subject_template	template	The mail subject template for revprop change notifications
lock_subject_template	template	The mail subject template for lock notifications
unlock_subject_template	template	The mail subject template for unlock notifications
commit_subject_prefix	string	The mail subject prefix for normal commits
propchange_subject_prefix	string	The mail subject prefix for revision property notifications
lock_subject_prefix	string	The mail subject prefix for lock notifications
unlock_subject_prefix	string	The mail subject prefix for unlock notifications
max_subject_length	number	The maximum subject length
from_addr	template	The sender addresses
to_addr	template	The receiver addresses
reply_to_addr	template	The reply-to address
to_newsgroup	template	The newsgroups to post to
diff_command	command line	The diff command to use
generate_diffs	token list	The list of actions, which generate diffs
browser_base_url	string	Base URL and type of the repository browser installation
viewcvs_base_url	string	(DEPRECATED) Base URL of the ViewCVS installation
long_mail_action	tuple	Action to take on overlong notification mails
long_news_action	tuple	Action to take on overlong notification news postings
mail_transfer_encoding	token	The content transfer encoding used for mails
news_transfer_encoding	token	The content transfer encoding used for news postings
mail_type	token	How to construct the mail (multipart/single)
apply_charset_property	boolean	Resolves the content charsets from svnmailer:content-charset properties.
show_applied_charset	token	Specifies whether the content charset (configured or default) of the should be written into the diff header.
custom_header	tuple	Name and Value format string for a custom header, which is included in the mail
extract_x509_author	boolean	Treats the author as x509 subject string and tries to extract the author's real name and email address
cia_rpc_server	string	CIA XML-RPC tracking server
cia_project_name	template	The project name submitted to the CIA tracker
cia_project_module	template	The project module submitted to the CIA tracker
cia_project_branch	template	The project branch submitted to the CIA tracker
cia_project_submodule	template	The project submodule submitted to the CIA tracker
cia_project_path	template	The project path stripped from the absolute file paths before submitting to the CIA tracker

The options described here are all valid both in group configurations and in the [defaults] section. If a option in a normal group configuration is missing, its value is taken from [defaults]. If there is nothing defined, a hardcoded default is applied.

for_repos

The for_repos option defines a regular expression, which is used to match against the file path of repository, for example /var/svn/my-repository. The file-path, which is matched against is guaranteed to not have a directory separator at the end (slash or backslash). Note that the regular expression always matches from the beginning of the path, so your regex typically will begin with .*. This is, because the svnmailer uses the re.match function - see the python docs for further information.

If the for_repos option is not defined or empty, the particular group matches for any repository (which is the default). Named matches of this group are stored for later substitutions.

In the following example the group "sample group" will be selected only if the script is called for the "public" repository (e.g. /var/svn/repositories/public):

for_paths

The for_paths option defines a regular expression, which is used to match against one of the modified paths stored in the repository, but only if the group was preselected by repository (see for_repos). If the path matched against is a directory, it is guaranteed to end with a slash, so that matching by directory paths results in more simple regular expressions. As with for_repos, the match always starts at the beginning of the path, but without a leading slash.

If the for_paths option is not defined or empty, the particular group matches for any path inside the repository (which is the default). Named matches of this group are stored for later substitutions.

In the following example the group "sample group" will be selected only if the script is called for the "public" repository (e.g. /var/svn/repositories/public) and everything under the /site/ directory (e.g. /site/images/foo.gif, but not for /site-tools/buildsite.sh):

exclude_paths

Since regular expressions usually match positive, it's from time to time helpful (and better readable) to exclude substrings with a separate match. The exclude_paths option exists for that purpose. It matches exactly like for_paths, but the group is selected only if the supplied regex does not match (and has been preselected by for_repos and for_paths).

If the exclude_paths option is not defined or empty, nothing will be excluded (which is the default). Of course, named groups of the match will not be stored for substitution, because the group is not selected, if there is a match of exclude_paths.

In the following example the group "sample group 1" will be selected only if the script is called for the "public" repository (e.g. /var/svn/repositories/public) and everything under the /site/ directory (e.g. /site/images/foo.gif), but not for stuff under /site/tools/. For every change under the site/tools/ directory the group "sample group 2" will be notified:

Note that if the exclude_paths option was not given, every change under site/tools/ would generate a notification for both groups.

ignore_if_other_matches

Consider a main project, which consists of several subprojects. Every subproject has its own notification group:

The exclude_paths option could be matched easier, if the sub projects really would be named subdigit. But usually this is not the case. As you see, maintaining the exclude_paths regex grows to a nightmare the more projects are added. The ignore_if_other_matches option is supposed to help out of this ugly situation. If set to a positive value (e.g. yes), the group will not be selected for the matched path if there are any other groups that match the same path / repository. The above config could be rewritten as:

Note that there is a border case. If you use this feature for more than one group, it can happen, that finally the list of selected groups per path consists only of more than one ignorable groups. Theoretically these would unselect each other. Practically all those groups are selected, so that the notification is not lost.

The "boolean" values accepted by this option are yes, on, true and 1 for the "true" case and no, off, false, 0, none and the empty string for the "false" case. The default is false.

For compatibility reasons, convenience and better readability this option can also be written as suppress_if_match or fallback.

show_nonmatching_paths

The show_nonmatching_paths option specifies how the svnmailer should deal with situations where the paths matched by for_paths are only a subset of all paths affected by the commit. For instance, consider the following changeset:

Further consider a notification group that matches for all paths beginning with foo/:

The show_nonmatching_paths option provides for three values, which solve the conflict differently:

yes

The additional changes are included in the notification (but after the ones belonging to the group). CIA notices for this group also include all changed paths then. If you have different customers that may not see each other's projects, be careful with this solution.

no

It will be stated in the notification, that there are additional changes (after the path list), but neither the paths nor their diffs are included

ignore

The additional paths will be just ignored. In the notification there will be no sign of changes not belonging to the group.

If show_nonmatching_paths is unset or empty, it defaults to no. Note that this default differs from the less safe yes default used in the subversion 1.2 mailer.py script.

For convenience reasons and better readability this option can also be written as nonmatching_paths, nongroup_paths and show_nongroup_paths.

commit_subject_template, propchange_subject_template, lock_subject_template and unlock_subject_template

These options define the subject templates to be used for the particular notification type (commit, revprop change, lock, unlock). In addition to the normal substitution record the following substitutions are available:

Additional subject template substitutions

Name	Value Description
prefix	This is the subject prefix as configured.
part	If mails are split, this contains the description of the current part ([x/y])
files*	This contains the paths affected by the event. Despite the name files this also contains affected directories (consider them as special files).
dirs*	Well, this only contains the directories affected by the event (in contrast to file).
files/dirs*	The content of files/dirs is determined dynamically. It chooses the value of files by default. If the subject gets too long then, it takes dirs. The length parameter is max_subject_length or 255 if max_subject_length is unset.

* All items are space separated. Further if the path items have a common prefix, it is extracted and the paths shortened respectively. It looks about "in /prefix: foo bar/baz" then.

After the template was filled in, all whitespaces are normalized, that is, leading and trailing spaces are stripped and multiple adjacent whitespaces of any favor are compressed to one space. So you don't have to worry about strange-looking subjects, because one of the substitutions is empty (part is a good candidate).

The svnmailer defines the following default templates in case of unset or empty options:

The default subject templates

commit	%(prefix)s r%(revision)s %(part)s - %(files/dirs)s
revprop change	%(prefix)s r%(revision)s - %(property)s
lock	%(prefix)s %(files/dirs)s
unlock	%(prefix)s %(files/dirs)s

A typical use case of a customized subject template is a mailing list, where the svn authors are not allowed to post, but just one mail address, which represents the notification mailer itself. Or you don't want to expose the mail addresses of the committers. However, it would be still desirable to get the author of the commit in the mail client overview. Just put into the subject:

commit_subject_prefix, propchange_subject_prefix, lock_subject_prefix and unlock_subject_prefix

These options define the subject prefix of the generated mails depending on the described event. If a string is supplied, it's provided as prefix substitution in the subject template. commit_subject_prefix defines the prefix for normal subversion commits (files, directories and versioned properties). propchange_subject_prefix defines the subject prefix for unversioned property change notifications. lock_subject_prefix defines the subject prefix for lock notifications (SVN 1.2 and later) and unlock_subject_prefix for unlock notifications (you knew that, huh?).

The default prefixes are empty.

max_subject_length

The max_subject_length option specifies the maximum length of the generated mail or news subject line. If the generated subject is longer than the defined limit, it is cut and three dots are appended (hence the minimum subject length is 3). If the max_subject_length option is not specified, empty or defines 0, no limit is applied.

For compatibility reasons and convenience the max_subject_length option can also be written as truncate_subject or subject_length.

from_addr, reply_to_addr, to_addr, to_fake and bcc_addr

from_addr, reply_to_addr, to_addr and bcc_addr define address templates for the mails to be sent. from_addr, to_addr and bcc_addr accept space or tab separated lists of address templates, while reply_to_addr takes just one address. The basic semantics should be quite clear: from_addr defines the sender addresses (but usually just one), to_addr the recipient addresses which should show up in the To header, bcc_addr the undisclosed recipients and reply_to_addr the address, where answers to the commit mail should be sent to. If groups are merged during the selection process, there can be any number of senders, receivers and even reply-to addresses in the mail (which conforms to RFC 2822, if you care about such things). Duplicates in the address lists are filtered away. In the case of more than one final sender address, the svnmailer generates an additional Sender: header with the first item of the sender address list (which is more or less random, but they should be all valid, right?).

The to_fake option comes into play if you have bcc_addr defined but no to_addr. By default, the svnmailer would just omit the To header and send the mail to the bcc recipients without this header. You might want to fill the header with a dummy address in this case. to_fake will be just written into the header but not treated as valid recipient address.

If there are no sender addresses given, it uses the string no_author (as the original script does). That may lead to an error while mail sending, so the best is to supply a valid from_addr in the [defaults] section.

If there are no recipients, the svnmailer simply doesn't send the mail (this is useful, if there are more notifier types like news or XML-RPC active). If this is not, what you want, you have to supply functioning to_addr options.

All those options may contain substitution patterns in the form %(name)s. The list of values to substitute is determined for each notification group dynamically using the for_repos and for_paths regular expressions. The substitution names author and group are always defined. If not overridden by one of the regular expressions, author contains the author of the change (or the string no_author if no author could be determined) and group the name of the notification group (the section heading without the braces).

All address templates described here are empty by default. For compatibility reasons the reply_to_addr option can also be written as reply_to.

bcc_addr and to_fake are available in version 1.0.8 and later.

to_newsgroup

The to_newsgroup option specifies a space or tab separated list of newsgroups where the notification should be posted to. This parameter is a substitution template, so you can extract information from the commit information.

Note that you need to define the nntp_host option (in [general]) in order to submit news postings.

diff_command

The diff_command option defines, that you want to use an external diff program (instead of python's difflib module), where to find and how to call it. The option value is a template for the command line to call. The program has to write the diff information to stdout. Stdout and stderr are caught by the svnmailer and dumped into the mail.

As said, the value describes a template. There is a fixed number of substitutions defined for the diff_command option, in particular: label_from, label_to, from and to. from and to define the actual files to process (typically some scrambled temporary file names). label_from and label_to define, how the files should be labeled by the diff program. These substitutions are written as %(name)s and replaced by the svnmailer script. If you want a literal percent character somewhere in the command line, you have to duplicate it (i.e., write %% instead).

The command line template is split on spaces or tabs to separate the different arguments. If you want to have an argument contain such space characters, you have to enclose it in double quotes ("). Further if you want such a quoted argument to contain a double quote character, you have to escape it with a backslash (i.e., write \" instead). To complete the escaping mechanism, you have to duplicate backslashes inside a quoted argument. The surrounding quotes and the backslash escape characters are stripped by the svnmailer to get the final argument string.

Note that on POSIX systems no shell is called to execute the program, so shell metacharacters are not interpreted. However, on Win32 the shell (cmd.exe & Co.) is called, but the arguments are properly escaped.

For compatibility and convenience the diff_command option can also be written as diff.

generate_diffs

The generate_diffs option defines which actions diffs are generated for. It takes a space or tab separated list of one or more of the following tokens: add, modify, copy, delete, propchange and none.

If the add token is given and a new file is added to the repository, the svnmailer generates a diff between an empty file and the newly added one. If the modify token is given and the content of an already existing file is changed, a diff between the old revision and the new revision of that file is generated. The copy token only worries about files, that are copied and modified during one commit. The delete token generates a diff between the previous revision of the file and an empty file, if a file was deleted.

If the propchange token is given, the svnmailer also takes care of changes in versioned properties. Whether it should actually generate diffs for the property change action depends on the other tokens of the generate_diffs list. The same rules as for files apply, except that the svnmailer never generates property diffs for deleted files. For example:

Now svnmailer generates diffs, if:

• A file is added, modified or copied (and modified)
• A property is added or modified (properties cannot be copied)

If a file or property is deleted, it's written as action information into the mail, but no content diff is generated. The default value for generate_diffs contains all possible actions. Mistyped tokens are ignored. If the resulting token list is empty, svnmailer falls back to the default. If you really don't want diffs to be generated, configure explicitly an empty generate_diff option or use:

browser_base_url

If the browser_base_url option is defined and not empty, the svnmailer generates URLs for the specified repository browser. One URL for the whole revision is placed on top and for every generated file diff a conrete URL is written before the actual diff output. The default value is empty and no urls are generated.

The option takes two parameters. The first one specifies the browser type. The second parameter determines the base url of the browser installation. The base url is interpreted by the semantics specified by the type, which can be one of:

viewcvs

The base url specifies the root URL of the ViewCVS installation. The svnmailer tries to keep most query parameters you provide (such as root). The following query parameters are always overridden: view, rev, p1, p2, r1 and r2.

websvn

The base url specifies the repository's WebSVN root listing URL. Depending on the WebSVN configuration this looks different (see the examples below) and produces different resulting URLs. The svnmailer tries to autodetect the installation type on the basis of the supplied URL. If it contains a query parameter called repname, a non-PATH_INFO installation is assumed. In this case the svnmailer may choke if the last path element ends with a slash (i.e. does not represent a file). This is just a sanity check, because the file will be replaced by the svnmailer, when constructing the final URLs.

As you might have guessed, if the URL doesn't contain this repname query parameter, a PATH_INFO installation is assumed. However, the best way to get the proper base URL is to open the repository root directory in your browser and just copy the URL and paste it into the svnmailer config.

And here is an output of a change in a sample repository (revision 5):

viewcvs_base_url

The viewcvs_base_url option is deprecated. Use the browser_base_url option instead.

long_mail_action

From time to time it happens, that commits are huge and so are the accompanying notification mails. They may hit limits of mailing lists or mailboxes they are sent to or simply crash some mail client. The long_mail_action option specifies, what "huge" means (in number of bytes) and what action the svnmailer should take to prevent such mails.

long_mail_action takes at least two space or tab separated parameters. The first one defines the number of bytes, one notification mail body should contain at maximum. Note that this number is applied, before the content is MIME encoded, so the actual mail may be slightly greater than the number of bytes specified here. The second parameter defines the action to take. It can be one of the following tokens:

truncate

This advises the svnmailer to truncate the notification mail starting with the first line reaching the limit. It will add a note about the number of lines truncated.

showurls

If you have a browser_base_url configured and you hit the limit, the svnmailer will just leave out all the diffs and supply the browser URLs only. If you have no browser_base_url configured, you will get neither diffs nor the URLs. If the final mail still gets too long, the showurls action doesn't care about, but showurls/truncate does.

showurls/truncate

This action token behaves mostly like showurls, except that if the single mail still gets too long, it is truncated after reaching the limit. A note about the number of truncated lines is appended then.

split[/number]

This action splits the notification mails into several ones. The mail subject will contain an enumeration string ([1/3], [2/3] etc) so you can read the mails in the proper order later and know if one is missing. In order to keep the mails readable the svnmailer never splits in the middle of a particular diff, but between whole diff blocks (including their meta information). That means, certain mails of the generated sequence may be still too long (containing one single huge diff). The split action doesn't care about this. If you want to truncate such huge single mails, use the split/truncate token instead.

Experience shows, that if the number of mails grows (huge imports easily lead to 30, 40 or more mails), less people actually read those mails. In order to not waste bandwidth and people's good humor, you may want to limit the number of mails per commit. This is done by extending the token by a slash and the maximum number of mails. If this limit is exceeded then, only one mail will be sent, containg just a short summary and an explanation.

split/truncate[/number]

This action token behaves mostly like split, except that overlong single mails (containing one huge diff) are truncated after reaching the limit. A note about the number of truncated lines is appended then.

The optional final parameters determine, whether the configured behaviour should be applied to revision property changes and/or locks, too. The possible tokens are revprop-changes and locks. Now if the second parameter specifies a truncating action, the speecified notifications are truncated if they would execeed the limit, otherwise they are not touched (They are never split). However, overlong revision property change or lock notifications should happen very rarely.

By default no special action is taken to prevent huge mails. But it is highly recommended to configure one.

long_news_action

The long_news_action option defines, how the svnmailer should deal with huge commit notification news articles. It takes exactly the same parameters like the long_mail_action option. Please have a look there for detailed information.

mail_transfer_encoding

The mail_transfer_encoding option specifies, which transfer encoding should be used when sending mails. It takes one of the following tokens:

quoted-printable

The mail will be encoded using quoted-printable. This has the advantage, that no physical line will be longer than 76 bytes (some mail servers cut or reject lines with more than 1000 byte). Disadvantage: The final size depends on the content. There is currently a known bug with this encoding: The underlying email module (from stdlib) changed its API between minor versions and suddenly encoded the already encoded content again. Therefore svnmailer no longer defaults to quoted-printable but to 8bit for now.

base64

The mail will be encoded using base64. Lines are also broken after 76 bytes and the final size is always 4/3 of the original size. Disadvantage: The encoded content is not human readable.

8bit (default)

The mail won't be transfer encoded. This means, that huge lines may be cut or mails containing weird bytes (e.g. \0) may be misinterpreted or rejected.

Note that the mail_transfer_encoding is not applied to multipart mails.

news_transfer_encoding

The news_transfer_encoding option defines the transfer encoding to use for news postings. It takes the value as mail_transfer_encoding.

mail_type

The mail_type option defines how the notification mail should be constructed. If you set it to multipart, the mail will be of type multipart/mixed, otherwise it's a single text mail.

The generation of multipart mails is discouraged, because is has unclear semantics (the diffs are more or less opaque data) and the client support is not as well as it could be. Further the mail_transfer_encoding option is not applied to multipart mails. The diff parts are designated as Content-Transfer-Encoding: binary. Further note, that regardless of the charset, paths are always encoded as UTF-8.

With svnmailer 1.0.4 and later you can also refine the multimail type with options. These options have the form name=value and just have to be appended to the mail_type command. The following options are possible:

type=content/type

The type option allows to specify the content type of the diff parts. A reasonable value could be, for instance, text/x-diff. Several mail clients are known to provide syntax highlighting for the diff then. If the option is not specified it defaults to text/plain.

disposition=content-disposition

The disposition option determines how the diff parts appear in the mail. Possible values are inline and attachment. If the option is not specified it defaults to inline.

apply_charset_property

The apply_charset_property option determines, whether the svnmailer should retrieve the content charset of the modified files from svn:mime-type and svnmailer:content-charset properties. This charset is used to recode the files, when generating the diff, so you get readable notifications, even if you change files with different charsets during one commit.

At first the svn:mime-type property of the particular file (in the particular revision) is evaluated. If it's set and contains a charset attribute (like text/plain; charset=iso-8859-2), the attribute value (iso-8859-2) is chosen as file encoding. Otherwise the svnmailer continues the encoding lookup by evaluating svnmailer:content-charset properties.

The svnmailer:content-charset property can be set either for the file itself or for one or more of its parent directories (while more specific definitions override less specific ones). If set for a particular file, the property just contains the charset name:

If the svnmailer doesn't find the property set for the changed file, it looks for the svnmailer:content-charset property in the parent directories up to /. These directory properties are the way to specify file charsets in a more general manner. If specified, they are expected to contain shell-style glob/charset definitions, one per line. Such a line looks like:

Leading and trailing spaces, empty lines, lines starting with # and lines without a = character are ignored. Those directory properties can be easily set using the svn propedit command:

Now an editor should be opened and you can add the desired definitions. A sample property could look like:

Note that the globs apply to the whole file path including the leading slash. They are interpreted case sensitive by python's fnmatch module. The content of the property is expected to be UTF-8 encoded.

If apply_charset_property is false or there's no svnmailer:content-charset property defined or the charset cannot be found in python's codec collection, the svnmailer falls back to iso-8859-1, which translates literally to unicode and so just represents the byte stream in character form. If the file content is miscoded (according to the chosen charset), uninterpretable byte sequences are represented by the replace character (U+FFFD). Note that asian encodings are not present in python 2.3. If you want to use such encodings, you need to install the CJKCodecs (which are included in python 2.4 and later).

The "boolean" values accepted by this option are yes, on, true and 1 for the "true" case and no, off, false, 0, none and the empty string for the "false" case. The default is false.

For convenience the apply_charset_property option can also be written as charset_property.

show_applied_charset

The apply_charset_property option specifies whether the svnmailer should write the (either assumed or configured) charset of a file into the diff header. This looks like this:

The value in the square brackets represents the content charset before it is recoded to UTF-8. That way you have all information to recode it back if you need to. The following configurations are possible:

yes

Always show the applied charset

no

Never show the applied charset

nondefault

Show the applied charset only if it was explicitly configured

If the option is unset or empty the svnmailer picks nondefault as default ;-).

custom_header

The custom_header option defines a custom header line that is attached to each mail sent for the particular group (or all groups, if you make it default). This allows for better filtering of the mails than just by subject. If the mails are sent to a mailing list, you probably don't need a custom header, because most mailing list software defines its own (and probably better) headers.

The supplied option value consists of two space or tab separated parts -- the header name and the header value template. The header name is always prepended with X- and invalid characters (according to RFC 2822) are stripped. The header value is a substitution template. Note that if the final header value contains characters, that are not contained by the us-ascii character set, it is encoded according to RFC 2047. If you supply the header name only, the header will be attached empty.

If groups are merged during the selection process and multiple custom headers are found, they are all attached to the mail. If multiple values for the same header name are found, they are merged with a comma and a space between. By default custom_header is empty.

In the above example the svnmailer sends all mails for the group some group with an additional header named X-SVN-Repository containing the value public.

extract_x509_author

The extract_x509_author option is useful, if you're using SSL client certificates in conjunction with SSL Options +FakeBasicAuth for repository authentication. If set to a positive value (e.g. yes), the svnmailer tries to extract the author's real name and email address from the supplied x509 Subject Distinguished Name (DN). In case of successful extraction it adds the following values to the substitution parameters:

x509_CN

The contents of the Common Name field (which is assumed to be UTF-8 encoded). This value also will be taken by the message generator for the meta data on top of the message.

x509_emailAddress

The contents of the email address field

x509_address

A string consisting of CN and email address usable as full email address. It has the form Common Name <email address>, where the Common Name is encoded according to RFC 2047.

The "boolean" values accepted by this option are yes, on, true and 1 for the "true" case and no, off, false, 0, none and the empty string for the "false" case. The default is false.

For convenience the extract_x509_author option can also be written as x509_author.

cia_rpc_server

The cia_rpc_server option defines a CIA XML-RPC handler, where your commits or some of them) should be tracked in real time. If this option is defined and not empty, the cia_xmlrpc notifier is considered to be run, but it will be activated only if the selected notification group defines a cia_project_name option. Note that it's possible for every group to run more than one notifier (e.g., mail and cia_xmlrpc) per commit.

The cia_rpc_server option takes a http or https URL. Note that the trailing slash is important. If there's no URL path given (i.e., just http://server), the svnmailer (or better, the utilized XML-RPC library) assumes it to be /RPC2.

The CIA notifier will not send any information if it runs in the post-revprop-change hook. By default there's no RPC server defined.

The cia_rpc_server option is allowed in group section in version 1.0.8 and later.

cia_project_name

The cia_project_name specifies the project name under which the changes are tracked in a CIA real time tracker. In order to track your projects using CIA notifiations you need to define the cia_rpc_server option properly at least and a cia_project_name for the notification groups that should be tracked. However, you should consider to set cia_project_path to a reasonable value.

The cia_project_name option value is a substitution template, so it is possible to extract the project name from the path or the like. By default there's no project name defined.

cia_project_module, cia_project_branch and cia_project_submodule

These options further refine the project description submitted to the CIA tracker. Note that you still need to specify a project name to submit the notification to the CIA server at all.

Like cia_project_name the option values are substitution templates, so it is possible to extract some information from the path. By default all of the descriptions are empty.

cia_project_path

The cia_project_path option defines the path in the repository that should be stripped from the beginning of the paths changed when submitting the file list to the CIA tracker. This lets tracked projects look more self containing. A real life example is the Apache HTTP Server. Its position in the ASF repository is /httpd/httpd/{trunk,branches,tags}. A reasonable cia_project_path value would be httpd/httpd/. Note that the specified path is normalized before being stripped. That means, any leading slash is removed and a trailing slash is appended if needed.

Like cia_project_name the option value is a substitution template, so it is possible to extract this information from the path. By default no path is stripped.

Substitutions

Substitutions are a powerful feature that can simplify some configurations very much. They base on named python format strings. The format strings are similar to the ones you may already know from the printf() function of C or perl, except that the particular formats are not determined by order, but by name. To give a particular format a name, you just write %(name)s instead of %s (for a string). The name can be any sequence of characters.

As usual, when dealing with such format strings, if you want to express a literal %, you need to duplicate it (%%). Note that you should limit your formats to strings (i.e. the s format), because that's what the svnmailer always supplies. Here is a real life example:

Well, where are the format names and values taken from? Depending on the option they are either fixed or determined dynamically. The former -- simpler -- variant is used by the diff_command option. For this option the svnmailer defines, which format names and values are used (label_from, label_to, from and to). For the actual meaning of these format names have a look at the diff_command description.

The dynamic definition of format names and values is a bit more complex. This is used by the address templates, the the subject templates, the CIA project descriptions and the custom_header option. There the list of format names (and values) is taken from previously matched regular expressions (default for_repos, default for_paths, group for_repos and group for_paths -- in that order, later definitions override earlier ones). This relies on another python feature: named matching groups. These are like normal storing parentheses, but you can give them a name. Instead of (to-match), you write (?P<name>to-match) in your regular expression. Such a name has to look like a valid python identifier. After a matching regular expression is executed, the svnmailer stores such named groups for later use separatly for each notification group. A typical use case of this feature is:

Besides the regular expression matches the substitution value record initially contains some fixed values: revision, property, author and group. revision contains the revision number if there is one available (e.g. locks are not tied to a particular revision). In the case of revision property change notifications property contains the name of the propery modified. author contains the author of the particular event (or no_author if no author could be determined). group contains the name of the notification group (that is the section heading without the braces). Further the record may contain the values described at the extract_x509_author option. All initial values can be overridden by one of the regular expressions described above. Additionally the subject templates provide more predefined values, but have a look there for details.

Configuring the Repository

Now, after you've created a config file, you can hook the svnmailer into the repository. Subversion uses so called hook scripts to perform customized actions at certain stages of a change event. (Note that the following descriptions don't apply directly to Windows.)

Configuring For Commit Messages

In order to run the svnmailer for normal commits you need to call it at the post-commit stage. If your post-commit hook isn't customized already, change into the repository/hooks directory and copy the post-commit.tmpl template to post-commit It's generally a good idea, to keep the template file for later reference.

After you've copied the template open the newly created hook script with your favorite editor. You will see a lot of comments describing the purpose of the hook and giving some hints about file system permissions etc. After you've read these comments, there follows a small shell script. Typcially there are sample scripts activated, you may not want to run. If so, comment them out or delete the entries. Finally to activate the svnmailer, add the following:

Of course, the REPOS and REV variables are only available if you left the definitions below the comments. Anyway, that's it. After you saved the file and closed your editor, you only have to make it executable and subversion will execute it after every commit. It is, however, a good idea to test the hook script as the user who runs it, before doing the next commit (That is only possible, if there are already revision stored in the repository):

Now you should receive a notification. If not, you should get a descriptive error message, what went wrong. By the way: a typical error is a non-readable configuration file.

Configuring For Revision Property Changes

Configuring for a revprop change message is mostly equal to commit messages. If you do not allow to modify revision properties, you don't need to care and can stop here. If you want to allow for revision property changes, you should read the related subversion documentation regarding the pre-revprop-change and post-revprop-change hooks first.

You need to hook the svnmailer into post-revprop-change to get a notification containing the new property value.

After opening the hook script the same warnings as for post-commits apply: remove all stuff, you don't want there. After that you can add the svnmailer command to the script. Depending on the subversion release, the command line may differ. Starting with subversion 1.2, the action taken on the property and the old property value (via STDIN) are supplied to the hook script.

The REPOS, REV, USER, PROPNAME and ACTION variables should be still defined, of course. Close the file, make it executable and test it:

Configuring For Lock Notifications

Locking notifications can be sent out when paths are either locked or unlocked. Locking is a new feature of subversion 1.2. In order to activate those notifications you need to hook the svnmailer into the post-lock hook and the post-unlock hook respectively. Similar to the other hooks there should be templates in your hooks/ directory. However, if you've just upgraded your subversion installation to 1.2, they may be not there, because the templates are generated only when you're creating a new repository. So if you don't find them, just create files in your hook directory named post-lock and post-unlock.

Well, when you're editing the template copies, make sure there's nothing called you do not want. And that's what you have to put into the hook scripts to let the svnmailer do its work:

The REPOS and USER variables should be still defined. If you've created the files from the scratch, the variable definitions are as follows:

When you're done, close the file, make it exectuable and test it:

Congratulations, you're done. Happy Hacking!

The svnmailer Command Line

The command line parameters

Parameter Name	Type	Description
--version	action	Shows the version and exits
--help	action	Shows a short help and exits
--debug	flag	Turns on debugging mode
--path-encoding (-e)	string	The encoding used for file and path names
--config (-f)	filepath	The location of the main config file
--revision (-r)	int	The revision number to process
--repository (-d)	filepath	The repository to process
--commit (-c)	flag	Turns on the commit mode
--propchange (-p)	flag	Turns on the revpropchange mode
--lock (-p)	flag	Turns on the lock mode. SVN 1.2 and later
--unlock (-p)	flag	Turns on the unlock mode. SVN 1.2 and later
--author (-a)	string	The author of the change (usually for revprop changes only)
--propname (-n)	string	The name of the modified property (for revprop changes only)
--action (-o)	character	The revprop change action (A, D or M). SVN 1.2 and later

The svnmailer is usually invoked via a small script called svn-mailer. It's located, whereever python installs it (have a look at the installation procedure for details). On my box it was installed to /usr/bin/svn-mailer. The script needs a number of command line parameters, which control its behaviour.

svn-mailer supports two command line styles: the "old-style" command line, which is compatible to the original mailer.py script and its own "new-style" command line, which takes named parameters only. The latter is recommended, however.

The "old-style" variant is derived from svn itself. It starts with a subcommand (either commit or propchange) and continues with the necessary parameters. Further it is fixed. No further options are allowed. Here are the old-style variants (optional parameters are enclosed in square brackets):

These lines, translated into the new style, look about:

The following sections describe all available "new-style" command line parameters in detail.

--version

If you supply the --version parameter, the svnmailer writes its own name and version plus the version of the subversion bindings it uses to stdout and exits immediately.

--help

If you supply the --help parameter, the svnmailer writes a short usage information to stdout and exits immediately.

--debug

The --debug option turns the svnmailer into debug mode. This means, that no mails are sent, but written to stdout, so you can test your installation. Additionally a header called X-Config-Groups is attached, which contains the selected notification groups.

--path-encoding (-e)

The --path-encoding option can be used to override the assumptions regarding the path name encodings the svnmailer makes based on the locale environment. This affects the paths given both via command line and config. If you're using file and path names consisting of ASCII characters only, you can (and probably should) safely skip the following paragraphs.

Normally this option should not be necessary. However, the POSIX filesystem doesn't know anything about encodings at all, so it can easily happen, that the bytes that happen to represent the file name are not interpretable as characters in the locale used. --path-encoding provides a workaround for this kind of problem (at least it tries to). You can specify the encoding of filenames independet from the locale. It just should be one of the codecs recognized by python.

Unfortunately there may be still a problem. The function to open the repository of underlying subversion bindings expects an unicode path and recodes it back to the locale representation (possibly using a slightly different algortithm than python). That way, the svnmailer has to recognize the encoding of the path given in the command line. The conversion back to the locale (inside svn) is not influencable by the --path-encoding option. If you get a crash caused by the subversion library, that describes a recoding error, you can try setting the LC_CTYPE environment variable to some ISO-8859-1 locale (like en_US.iso-8859-1), and just treat the octets of the file names as characters (the reason for this is, that the codepoints of ISO-8859-1 are compatible to unicode, so it should be all recoded smoothly) -- or use --path-encoding to supply it in a readable form to the svnmailer.

Anyway, the best solution is either to use just ASCII names or to use consistent encodings for file names and locales. UTF-8 is a good choice.

--config (-f)

The --config option is used to pass the path and name of a configuration file to the svnmailer. If you omit this option, the svnmailer looks at some default locations for a config, which are in order: repos/conf/mailer.conf, a file called mailer.conf in the same directory the svn-mailer script is located in and /etc/svn-mailer.conf. The file found first wins. If you didn't supply a --config option and no file is found at the default locations, the svnmailer exits with an error. It is recommended, to supply always the full path to the config file, so that no misunderstandings can happen.

If the supplied config file is a single dash (-), the config is read from STDIN. Note that this interferes with the property value supplied via stdin on revision property changes in subversion 1.2 and later.

For convenience the --config option can also be written as -f.

--revision (-r)

The --revision parameter defines the revision to process. It has to be a number and a valid revision of the supplied repository, otherwise the svnmailer will exit with an error. Typically you don't need to worry about this, since the proper revision number is passed to the hook script by subversion itself.

For convenience the --revision parameter can also be written as -r.

--repository (-d)

The --repository parameter specifies the full filesystem location of the repository. This is the path matched against by the for_repos configuration option. If it does not point to a subversion repository, the svnmailer will exit with an error. Typically you don't need to worry about this, since the proper repository path is passed to the hook script by subversion itself.

For convenience the --repository parameter can also be written as -d.

--commit (-c), --propchange (-p), --lock (-l) and --unlock (-u)

These options tell the svnmailer whether it is running in the post-commit, the post-revprop-change, the post-lock or the post-unlock hook. They are mutually exclusive (in fact, if you supply more of one, the most right option wins). The --commit option is default and can be omitted. The --lock and --unlock switches are only available if you're using subversion 1.2 or later. Have a look at the repository configuration section for further information.

The options also can be abbreviated, of course. --commit can be written as -c, --propchange as -p. --lock as -l and --unlock as -u.

--author (-a)

The --author option defines the author of the change. This is typically used in the post-revprop-change hook, where the author maybe different from the original revision author. The change author is passed to the hook script by subversion.

It is also possible to pass the --author option from within the post-commit hook. In that case it overrides the revision author noted in the repository. This is useful, if there is no revision author stored.

For convenience the --author parameter can also be written as -a.

--propname (-p)

The --propname parameter is only used for revprop-change notifications. It specifies the name of the property being modified. The name is passed to the hook script by subversion.

For convenience the --propname parameter can also be written as -n.

--action (-o)

The --action parameter is only used for revprop-change notifications. It specifies the action that was taken on the specified property. It contains one of the letters A (for "added"), M (for "modified") or D (for "deleted"). The proper value is passed to the hook script by subversion 1.2 and later. If the svnmailer is in propchange mode and this parameter is specified and valid, the svnmailer reads the old property value from STDIN and generates a diff between the old and the new value.

For convenience the --action parameter can also be written as -o (shortcut for "operation" :-).

Perlig - Projects - Publications

Imprint