#507465 mb2md: Please provide an option to tweak mail separation detection.

#507465#5
Date:
2008-12-01 16:32:56 UTC
From:
To:
This script intentionally looks for a blank line in between messages
in the mbox file.  There is no such requirement that I know of;
certainly all the mbox data I have has tons of messages with no blank
line before the '^From ' line.

As is, I got 30% fewer messages in the new Maildir, and lots of
messages were actually two or three messages run together.  The result
is so garbled I wonder if anyone else has ever used this script...

This tweak seems to have worked for me:
--- /usr/bin/mb2md	2005-07-04 16:38:47.000000000 -0400
+++ mb2md	2008-12-01 11:18:42.080177429 -0500
@@ -979,8 +979,6 @@
 	# The subject of the message
 	my $subject = '';

-	my $previous_line_was_empty = 1;
-
                     # We record the message start line here, for error
                     # reporting.
         my $startline;
@@ -1003,7 +1001,6 @@
             $_ =~ s/\r\n$/\n/;

             if ( /^From /
-		&& $previous_line_was_empty
 		&& (!defined $contentlength)
 	       )
             {
@@ -1419,8 +1416,6 @@
                             # End of the if statement dealing with message body.
             }

-	    $previous_line_was_empty = ( $_ eq "\n" );
-
                             # End of while (MBOX) loop.
         }
                             # Close the input file.

#507465#12
Date:
2008-12-03 08:05:34 UTC
From:
To:
Hello,

There *is* certainly mention of a blank line in the first few
references that turn up when looking for an mbox file specification:

http://www.qmail.org/man/man5/mbox.html
http://en.wikipedia.org/wiki/Mbox

mb2md has worked perfectly well for me with mbox files coming from
Procmail and Dovecot. Are you certain your mailboxes are not ill-
formatted to begin with?

Bye,
Bruno

#507465#17
Date:
2008-12-07 12:45:53 UTC
From:
To:
severity 507465 wishlist
retitle 507465 mb2md: Please provide an option to tweak mail separation detection.
thanks

Putting back the submitter in the loop, please Cc submitters, or use the
NNNNNN-submitter@bugs.debian.org alias…

Bruno De Fraine <bruno@defraine.net> (03/12/2008):

Err, quoting qmail as a reference looks bogus to me… Anyway, looking at
e.g. RFC 4155 (which points to qmail's site too, sigh, with typo, sigh
again), we have:

,--[ Appendix A. The "default" mbox Database Format ]--
|    The "default" mbox database format uses a linear sequence of Internet
|    messages, with each message being immediately prefaced by a separator
|    line, and being terminated by an empty line.  More specifically:
|
|    …
|
|       o Each message in the database MUST be terminated by an empty
|         line, containing a single end-of-line marker.
`--

So you're right.

Requesting an option to make the newline between two sucessive mails
optional makes anyway sense to me, adjusting bug severity and title
accordingly.

Mraw,
KiBi.

#507465#26
Date:
2008-12-07 16:49:08 UTC
From:
To:
Bruno De Fraine <bruno@defraine.net> writes:

Yes, they do seem to be badly formed.  My dud mail files were from
thunderbird/icedove.  Evidently its local mail storage is ad-hoc mbox-like
non-mbox files.  Sigh...

Anyway, thanks, all, for the investigation.