procmail will, on occasion, add a '>From ' header to mail that it is delivering. when Header.pm sees this line (in sub read), it barfs and quits processing header lines.
At 11:18 am, Tuesday, January 8 2002, lantz moore mumbled: Are you able to _reillably_ reproduce the problem? Are can you say how it barfs, exactly?
Steve Kowalik <stevenk@debian.org> writes:
yes, the relevant bit from the procmail manpage says:
When in explicit delivery mode, procmail will generate a leading
`From ' line if none is present. If one is already present
procmail will leave it intact. If procmail is not invoked with one
of the following user or group ids: root, daemon, uucp, mail, x400,
network, list, slist, lists or news, but still has to generate or
accept a new `From ' line, it will generate an additional `>From '
line to help distinguish fake mails.
when the function 'read' in Header.pm attempts to process the escaped
mail-header '>From ', it simply stops processing headers. '>From ' does
not look like a valid mail header to Header.pm. i could probably whip up
some example code and test inputs too if you need them.
i really don't know if this is a bug in Header.pm, if Header.pm is just
being pedantic, or if procmail is doing things non-standardly. however,
it seems to be something that procmail does explictly, and on purpose.
also, i've seen patches for mutt that address the same issue.
thanks!
* Steve Kowalik (stevenk@debian.org) [020109 08:59]: in a header. Lines which start with From, but are in the body of the message, will be escaped into >From to distinguish them from the message-separator line which is used in mbox folder types. That's a known defect from mbox system, and that may have been buggy in old versions of mutt (never saw that problem with mutt myself). Why do you want to use 'explicit delivery mode'? Write the author of procmail for support when you still think it behaves wrong. But please include a demonstration script, not just a plain text question. Otherwise it is too hard for us to understand what happens.------------------------------------------------------------------------ drs Mark A.C.J. Overmeer markov@ATComputing.nl AT Computing, UNIX Training and Consultancy http://www.ATComputing.nl http://Mark.Overmeer.net Mark@Overmeer.net
Hello. The following bug report from Lantz Moore has been reassigned to the Debian procmail package:---------------------------------------------------------------------------- Package: libmailtools-perl Version: 1.42-2 Severity: normal procmail will, on occasion, add a '>From ' header to mail that it is delivering. when Header.pm sees this line (in sub read), it barfs and quits processing header lines.---------------------------------------------------------------------------- [ More discussion about this bug at http://bugs.debian.org/128160 ] I can reproduce this by misconfiguring my MTA, but in either case it seems problematic that procmail transforms rfc822-compliant messages into non-rfc822-compliant ones. Thanks.
This bug with 'procmail -d <user>' also causes 'formail -i "..."' to further corrupt the message, but the bug is dependent on the headers present before the non-RFC compliant '>From ' header. If we start with a message whose headers begin: From athan@dsl.miggy.org Tue Aug 28 20:14:03 2007 Return-path: <athan@dsl.miggy.org> Envelope-to: athan@miggy.org Delivery-date: Tue, 28 Aug 2007 20:14:03 +0100 And then deliver it using 'procmail -d athan', we get: From athan@dsl.miggy.org Tue Aug 28 20:14:03 2007 >From athan Tue Aug 28 20:14:03 2007 Return-path: <athan@dsl.miggy.org> Envelope-to: athan@miggy.org Delivery-date: Tue, 28 Aug 2007 20:14:03 +0100 Now, within procmail we run it through, bogofilter -e -p -u, and get: From athan@dsl.miggy.org Tue Aug 28 20:14:03 2007 >From athan Tue Aug 28 20:14:03 2007 Return-path: <athan@dsl.miggy.org> Envelope-to: athan@miggy.org Delivery-date: Tue, 28 Aug 2007 20:14:03 +0100 Then, as bogofilter isn't catching everything, we run it through, spamassassin, and get: From athan@dsl.miggy.org Tue Aug 28 20:14:03 2007 X-Spam-Checker-Version: SpamAssassin 3.1.7-deb (2006-10-05) on bowl.fysh.org X-Spam-Level: X-Spam-Status: No, score=-1.6 required=3.0 tests=AWL,BAYES_00,UPPERCASE_25_50 autolearn=ham version=3.1.7-deb >From athan Tue Aug 28 20:14:03 2007 Return-path: <athan@dsl.miggy.org> Envelope-to: athan@miggy.org Delivery-date: Tue, 28 Aug 2007 20:14:03 +0100 Note how SpamAssassin has inserted some headers after the properly formed 'From ' header, but before the improperly formed '>From ' header. Now we run it hits a procmailrc rule to set Content-Type correctly for inline PGP signed messages, which filters it through: formail -i "Content-Type: application/pgp; format=text; x-action=sign" and the headers end up as: From athan@dsl.miggy.org Tue Aug 28 20:14:03 2007 X-Spam-Checker-Version: SpamAssassin 3.1.7-deb (2006-10-05) on bowl.fysh.org X-Spam-Level: X-Spam-Status: No, score=-1.6 required=3.0 tests=AWL,BAYES_00,UPPERCASE_25_50 autolearn=ham version=3.1.7-deb Content-Type: application/pgp; format=text; x-action=sign >From athan Tue Aug 28 20:14:03 2007 Return-path: <athan@dsl.miggy.org> Envelope-to: athan@miggy.org Delivery-date: Tue, 28 Aug 2007 20:14:03 +0100 Note the addition of a blank line, causing premature end of headers. In fact formail has now not REPLACED the Content-Type header, it's added a new one. If the '>From ' line is filtered out before calling formail then things work properly. It would seem that 'From ' should be turned into something RFC-compliant, not '>From ', maybe 'Old-From_: ' ? Now to see if I can make procmail strip/change that badly formed header....
Another user on my affected machine found out that procmail's !<address> recipe target is also adversely affected by the ">From ..." header that "-d <localpart>" adds. Of course it also turns out that the little voice at the back of my head was correct that with exim I don't actually need to add the -d argument at all. It was cruft left over in our config from when we originally switched to exim in the late 90s! But adding that header is still a definite bug.
Am 2007-08-28 20:52:49, schrieb Athanasius:
Not right, since it does NOT affect "formail". "formail" is right by
adding a blanc line since ">From" can only be in a Body NOT THE HEADER.
I have never seen such "From" headers.
Question:
When do you get a Message with a "From " Header? -- I do not
know a singel MTA or fetchmail/getmail which add a "From " Header.
Do you try to deliver the content of a Mailbox to "-d <user>"?
If yes, it is the wrong way because
procmail -d ${USER} <mail_file
works only IF the "mail_file" was from a Maildir. Otherwise you
should use:
formail +1 <mail_file -ds procmail -d ${USER}
The behaviour of "formail" is right.
Maybe you should correct the WRONG "From " header before you try to
change working programs...
Thanks, Greetings and nice Day
Michelle Konzack
Systemadministrator
Tamay Dogan Network
Debian GNU/Linux Consultant
formail, and any other program, may indeed be correct to treat email
with this '>From ' header in it in this manner, but that is not the
point.
The point is that delivery with 'procmail -d <user>' inserts that very
header in the first place.
As for 'From ...' headers. That's the mbox format. A line which
begins 'From ...' delimits each message in an mbox format file. I don't
recall if a blank line (so "\n\n") is necessary before this.
Perhaps you should understand mbox format before you comment further.
The testing I did, which I simplified the results of was to put a
quick recipe at the very start of my .procmailrc to spit out a copy of
the email as it was at that point. This already had the '>From ...'
header in it and I now know exim wasn't responsible for this as I've
since changed its config to not use -d with procmail and now the header
no longer appears. I also at this point had VERBOSE=yes in my .procmailrc.
I then took the file this first recipe produced and by hand fed it
through any other filter it would have hit. First spamassassin, then
bogofilter, then the formail recipe. I saved each stage of this.
THAT is what I was describing in my original addition to this bug
report.
Note that the procmail manpage says:
-d recipient ...
This turns on explicit delivery mode, delivery will be to the
...
When in explicit delivery mode, procmail will generate a leading `From
' line if none is present. If one is already present procmail will
leave it intact. If procmail is not invoked with one of the following
user or group ids : root, daemon, uucp, mail, x400, network, list,
slist, lists or news, but still has to generate or accept a new `From '
line, it will generate an additional `>From ' line to help distinguish
fake mails.
It's that very last described behaviour which is the bug. That
'escaping' with a > before 'From' is what is used in the *BODY* of an
email if it begins with 'From', so as to not falsely think the email has
finished and move onto the next. This is NOT what should happen in the
headers.
Look at the code in from.c in the procmail source, the makeFrom()
function. It uses FAKE_FIELD, which is defined to ">From ". So if
Debian wanted to fix this problem locally all it needs is that changing
to "X-Procmail-From: " or somesuch.
What procmail does (the extra >From upfront) is a reaction to a badly configured mail system. I have to admit that I'm not quite sure anymore why I picked that way to communicate this fact. The intention definitely was not to destroy information, just add. Maybe I followed someone else's lead back then, or maybe I merely tried to establish new ground. I admit adding this into procmail, then not make formail treat it accordingly is dubious at best. The best way to deal with this probably is to add a different header like "Fake-From_: " or something similar. I'll see what I can do.