Dear Maintainer,
RT changes the original order of email headers when forwarding attached emails.
This is seen in case the original message was offered to RT with a leading unix
style From_ header. This behaviour causes havoc in case the full mime message
later gets analysed by strict spam scanners who interpret mime-included message
headers.
Both the full headers of a message as offered to RT (headers-incoming.txt)
and the full headers of message as forwarded in mime attachemnt
by RT (headers-forwarded.txt) are attached to this bugreport.
The first line in headers-incoming.txt is
From redacted@tilburguniversity.edu mon apr 4 20: 01:10 2022
. (This was likely added by postfix doing local(8postfix) mail delivery, and
alias expansion.) However, in headers-forwarded.txt it ends up at line 19.
This breaks the email headers. (And our spam scanner RSPAMD agrees and
reports it as BROKEN_HEADERS.)
Below we elaborate on the details of our setup.
* What led up to the situation?
There is no specific trigger for this situation, presumably the bug existed in
the system eversince it was installed. What triggered our investigation was a
helpdesk call from one of our customers telling us that "forwarded tickets from
RT4 were bounced by our mail gateway".
A little bit of background on our setup:
- Tickets are received through SMTP and added to the system by rt-mailgate. Our
configuration is based on
https://rt-wiki.bestpractical.com/wiki/ManualEmailConfig.
- The RT4 view used to forward the original email
(https://<rt_host>/rt/Ticket/Display.html?id=<ticket_number>)
- We're using Postfix (3.1.15-0+deb9u) as MTA to send and receive emails from
RT4. Outgoing mail is relayed through our mail gateway.
- On our mail gateway we use RSPAMD (version 3.1-1~buster from rspamd.com)
Other related packages:
- apache2 (2.4.25-3+deb9u12)
- postgresql-9.6 (9.6.24-0+deb9u1)
- request-tracker4 rt4-clients rt4-db-postgresql rt4-db-sqlite rt4-fcgi
(all version 4.4.1-3+deb9u3)
The reason for these bounces: RSPAMD marked all of these messages to have
BROKEN_HEADERS. This ultimately results in bouncing the message as the SPAM
score increased beyond the configured limit. Investigation into these
BROKEN_HEADERS showed that when piping emails to rt-mailgate (from the aliases
file), postfix adds something like: "From REMOVED@tilburguniversity.edu Tue Mar
29 12:52:50 2022" before the original email header (so this becomes the first
line parsed by rt-mailgate). This behaviour can be explained from the local(8)
manpage that comes with postfix: "In the case of UNIX-style mailbox delivery,
the local(8) daemon prepends a "From sender time_stamp" envelope header to each
message". This can be confirmed by piping the email alias to a file using 'cat'
instead of rt-mailgate.
--- See mail-delivered-to-cat.txt
Next, we have tested rt-mailgate to "deliver" to netcat: 'nc -l -p 81 > nc.log
2>&1'. The nc.log still contains the first line added by postfix. Feeding only
the SMTP header (including the line added by postfix against RSPAMD did NOT
give a BROKEN_HEADER classification. Command: 'cat mail-delivered-to-cat.txt |
/usr/bin/rt-mailgate --queue 'General' --action correspond --url
http://localhost:81/rt'
--- See nc.log
After this we actually delivered the ticket to the RT queue: 'cat
mail-delivered-to-cat.txt | /usr/bin/rt-mailgate --queue 'General' --action
correspond --url http://localhost/rt'
Last step is to stop postfix on the local system and forward the first entry in
the history (from view:
https://<rt_host>/rt/Ticket/Display.html?id=<ticket_number>) to a random email
address. When running postcat of that forward from the local mailqueue we get
the results showed in postcat-forwarded-message.txt. When pasting the header in
the MIME attached message into RSPAMD, RSPAMD reports BROKEN_HEADERS.
Changing the following entry in the SMTP header of the MIME attachment:
"From REMOVED@tilburguniversity.edu Tue Mar 29 12:52:50 2022"
To (not the colon):
"From: REMOVED@tilburguniversity.edu Tue Mar 29 12:52:50 2022"
RSPAMD no longer reports BROKEN_HEADERS
We have been able to reproduce the behaviour by setting up a clean installation
of RT4, with the same versions as specified below.
* Workaround
We have been able to work around the problem by filtering the first line (added
by postfix) on delivery to rt-mailgate in the aliases configuration. An
example:
Change:
rt: "| /usr/bin/rt-mailgate --queue 'Test' --action correspond -- url http://localhost/rt"
To:
rt: "| /usr/bin/awk 'NR > 1 || !/^From /' | /usr/bin/rt-mailgate -- queue 'Test' --action correspond --url http://localhost/rt"
In /your/aliases/file and run 'postalias /your/aliases/file'
* Thanks to:
Lars Biemans and Wessel Dankers for the collaboration and
effort on finding and working around this issue!
Bye,
Sander van Dinten and Joost van Baal-Ilić