This site is now 100% read-only, and retired.

Handling Debian mailing lists easily

Posted by Steve on Mon 29 May 2006 at 12:45

If you're a Debian user it is likely that you're subscribed to several of the mailing lists which the project uses for discussion, development coordination, etc. There are a lot of lists available, some quiet, and some very busy. If you're subscribed to several of them you might be looking for a simple way of organizing them, thankfully procmail makes it easy.

If you're already using procmail then you will already be familiar with the idea of sorting mail into different mailboxes (mbox) or folders (Maildir). For newcomers to procmail there is a simple introduction to filtering mail with procmail available elsewhere on this site.

procmail is configured via the file .procmailrc inside your home directory (and the global file /etc/procmailrc if it exists). This file typically contains several different things:

  • Setup of variables.
  • Rules to match messages and apply actions to them.

The rules are the important parts. Using rules you can match almost all aspects of incoming mail. Typically this means that you'll ensure that all mails sent to a particular email address get filed into a single folder - or make sure that all emails sent from a particular email address get sent into another.

For example this is a rule which I use to make sure that all mails sent to the bugtraq mailing list get stored into a mailbox called bugtraq:

:0:
*(Delivered-To:) mailing list bugtraq@securityfocus.com
bugtraq

I have other, similar, rules for filing messages sent to particular lists, or from particular individuals. What I used to have was a number of rules for each Debian mailing list I was subscribed to which looked like this:

...
...

:0:
*(X-Mailing-List:).*(<debian-devel-announce@lists.debian.org>)
debian-devel-announce

:0:
*(X-Mailing-List:).*(<debian-user@lists.debian.org>)
debian-user

:0:
*(X-Mailing-List:).*(<debian-devel@lists.debian.org>)
debian-devel

...
...

Each of these rules was very simple, and stored the mail from a particular list to a mailbox of the identical name. However these needed updating every time I subscribed to a new mailing list, and took up a lot of space making my ~/.procmailrc file look more crowded than it needed to be.

Thankfully there is a better approach. Included within the devscripts package there are some sample rules which can be used to filter Debian mailing lists in a much simpler manner.

The examples are installed to the directory /usr/share/doc/devscripts/examples/, and can be read at your leisure.

Keeping my mail in folders which I was using previously ~/Mail/debian-devel, ~/Mail/debian-user, etc, just needed the addition of this single rule:

# Sort debian mailing lists into mailboxes.
# This dynamic style means when subscribing to new Debian mailing lists
# no changes need be made to this file.
# Even as yet none existant mailing lists are catered for.
:0:
* ^X-Mailing-List: <debian-.+@lists.debian.org>
* ^X-Mailing-List: <debian-\/[-a-zA-Z0-9]+
debian-$MATCH

This takes advantage of procmails matching abilities to find mails which have a X-Mailing-List header matching "debian-*@list.debian.org". The second line is responsible for matching the actual name of the mailing list - and this result is stored in a variable named "$MATCH".

Using variables in procmail rules, or recipes, is explained in the manpage for procmailrc. Reading that we see the following:

\/ splits the expression in two parts. Everything matching the right part will be assigned to the MATCH environment variable.

This explains why the second line sets $MATCH up with the name of the mailing list which we use when saving to a mailbox.

(You might wonder why I manually prefix the match with "debian-", that is only done this way because I adopted the example from the devscripts file - there matchers are saved into a subfolder...)

I've been using procmail for years and have never used the \/ operator, or the $MATCH variable, but this is a perfect example of where using it can be very simple and useful.

 

 


Re: Handling Debian mailing lists easily
Posted by fsateler (201.214.xx.xx) on Mon 29 May 2006 at 16:25
[ View Weblogs ]
I read the mailing lists through gmane. This way it is a lot easier, and it also doesn't clutter my mailbox (gmail unfortunately doesn't handle lists very well).

PD: Seems like the article parser ate some parts of the procmailrcs.
--------
Felipe Sateler

[ Parent ]

Re: Handling Debian mailing lists easily
Posted by Steve (62.30.xx.xx) on Mon 29 May 2006 at 16:30
[ View Weblogs ]

I've tried reading lists online in the past, but I alway end up reverting to subscriptions. Partly because I like to be able to save messages of interest and partly because I never have troubles remembering where I finished reading the previous day!

As for the parser, Ooops! I've fixed it now.

Steve

[ Parent ]

Re: Handling Debian mailing lists easily
Posted by fsateler (201.214.xx.xx) on Mon 29 May 2006 at 17:04
[ View Weblogs ]
Well, since gmane doesn't expire articles, you don't need to save any messages. As for where did you end reading... that seems like a problem with your news reader, since all I've had remember which articles are read/unread/new. Also, you can ignore threads of no interest (which is good if the list is high traffic)
--------
Felipe Sateler

[ Parent ]

Re: Handling Debian mailing lists easily
Posted by Steve (62.30.xx.xx) on Mon 29 May 2006 at 20:40
[ View Weblogs ]

My bad. I was talking about reading the mailing lists through the online browser interface - rather than through a newsreader.

So if I switched browser/computer I'd lose track of which messages I'd read. (Think of me using something like Google Groups.)

To be honest ignoring long threads, and people, which aren't of interest is another reason for using my subscription by mail - since I can setup killfiles quickly and easily.

(Although I'm sure I could do the same with a newsreader; just not with a web-based system.)

Steve

[ Parent ]

Re: Handling Debian mailing lists easily
Posted by reluctant (65.78.xx.xx) on Mon 29 May 2006 at 18:22
For browsing, the linux.debian.* newsgroups do it for me. Thunderbird handles it, as do the other newsreaders. But you can't reply to the newsgroup; must post to the list directly.

[ Parent ]

Re: Handling Debian mailing lists easily
Posted by paulgear (203.206.xx.xx) on Tue 30 May 2006 at 03:20
If you read the lists with Thunderbird's news reader via gmane.org, it handles posting for you automatically.

One of the big advantages of doing it this way is that you can save bandwidth by killing conversations and the like.

[ Parent ]

Alioth lists
Posted by Anonymous (200.115.xx.xx) on Tue 30 May 2006 at 10:32
I use this procmailrc from devscripts, altough i added the following snip for alioths mailing lists:


:0:
* ^X-BeenThere: .+@lists.alioth.debian.org
* ^X-BeenThere: \/[-a-zA-Z0-9]+
$DEBIAN/$MATCH/


Also, I put this and the other debian-specific rules in a .procmailrc_debian and include it in my main .procmailrc with:


INCLUDERC=$HOME/.procmailrc_debian


Note: I do use the $DEBIAN folder and create mailboxes under that directory stripping the debian- part of the mailing lists, but if you dont, you may want to ad a debian- before $MATCH (and hope there is no clashing in mailinglists names... there suldn't be...)

[ Parent ]

Re: Handling Debian mailing lists easily
Posted by marki (82.119.xx.xx) on Tue 30 May 2006 at 11:41
How can I make complex rules?
When I just type rules one after another, they are ANDed. Is there a possibility for OR?

For example I have a rule to forward all emails except defined to my mobile phone as sms. Now I have about 10 general exceptions (emails from mailer-daemon, mailing lists, ...). But I want to add more complex filtering - for example to exclude also emails from user@example.com which have Subject "Fwd: .*".

How to do this?

[ Parent ]

Re: Handling Debian mailing lists easily
Posted by Steve (62.30.xx.xx) on Tue 30 May 2006 at 11:48
[ View Weblogs ]

See the procmail documentation is the short answer. Here is a good example.

Steve

[ Parent ]

Re: Handling Debian mailing lists easily
Posted by dom (217.147.xx.xx) on Tue 30 May 2006 at 18:24
I use List-ID rather than X-anything for identifying mailing lists, because it's an RFC-defined header and is fairly universal. Never heard of the X-Mailing-List header before...

[ Parent ]

Re: Handling Debian mailing lists easily
Posted by Steve (62.30.xx.xx) on Fri 25 Aug 2006 at 21:58
[ View Weblogs ]

This technique can also be applied to Debian bug report mails.

Steve

[ Parent ]