#1079229 cyrus-replication: Replication stopped working after upgrading from 3.8.4-1 to 3.10.0-1

Package:
cyrus-replication
Source:
cyrus-replication
Description:
Cyrus mail system - replication
Submitter:
Gábor Gombás
Date:
2025-05-28 16:25:02 UTC
Severity:
normal
Tags:
#1079229#5
Date:
2024-08-21 18:02:25 UTC
From:
To:
Hi,

After upgrading, syc_client can no longer authenticate. These are the
messages from sync_client:

Aug 21 19:55:14 primary.lan cyrus/sync_client[1331228]: couldn't authenticate to backend server: bad protocol / cancel
Aug 21 19:55:14 primary.lan cyrus/sync_client[1331228]: getaddrinfo(replica.lan) failed: Servname not supported for ai_socktype

While these are the errors messages on the other side:

Aug 21 19:55:14 replica.lan cyrus/imap[34041]: login: [xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx] imap/primary.lan GSSAPI User logged in SESSIONID=<cyrus-XXXX>
Aug 21 19:55:14 replica.lan cyrus/imap[34041]: decoding error: generic failure; SASL(-1): generic failure: , closing connection
Aug 21 19:55:14 replica.lan cyrus/imap[34041]: session ended: sessionid=<cyrus-XXXX> userid=<imap/primary.lan> id.name=<(null)>

Regards,
Gabor

#1079229#10
Date:
2024-08-22 05:21:10 UTC
From:
To:
Hi,

I found another message, which for some reason does not go to the
default log:

Aug 22 06:15:38 replica.lan cyrus/imap[51641]: encoded packet size too big (705497681 > 4096)

It's always the same number, and 705497681 is 2A0D0A51 in hex, which is
the '* CR LF )' sequence, so I suspect sync_client sends an unencrypted
command which imapd on the other side tries to interpret as an encrypted
packet.

Regards,
Gabor

#1079229#17
Date:
2025-05-26 05:19:48 UTC
From:
To:
Hi,

upstream team answered (see
https://github.com/cyrusimap/cyrus-imapd/issues/5452#issuecomment-2908229941):

The two errors from sync_client confused me for a minute, but I think
what happens here is:

     It tries to connect to the replica via imap (sync_try_imap: yes is
the default). The authentication fails with the bad protocol / cancel error.
     It wants to try again via csync, but can't work out how (probably
there's no "csync" record in /etc/services), so this fails too. I think
this is a red herring and the real problem is the first attempt having
failed.

The errors on the other side show:

     imap service receives a valid login using GSSAPI
     something unknown goes wrong in SASL
     the session is dropped

The "decoding error:" on line 2 looks like it comes from
prot_sasldecode(), which "Decode(s) data sent by a SASL security layer".
Here it is again, trimmed:
decoding error: generic failure; SASL(-1): generic failure: , closing
connection. I think this means:

     generic failure is how sasl_errstring() described the issue
     SASL(-1): generic failure: is how sasl_errdetail() described the issue
     , closing connection is imapd describing how it's handling the
error. Looks like it was trying to parse a command from the client at
the time the error happened. That might be the failed client connection
attempt sending a Logout before closing its end.

I think something is going wrong in SASL here. I think it looks like this:

C: tries to login
S: login succeeds, sends success response
C: fails to parse the success response due to SASL failure, decides the
login failed
C: sends logout
S: fails to parse the logout due to SASL failure, drops the connection

The other log message, "which for some reason does not go to the default
log", isn't from cyrus-imapd because it's in the wrong place. I guess
it's probably from SASL, describing the problem it encountered. It's
hard to be certain since the timestamp is completely unrelated to the
other log lines, but I suppose the reporter sees a lot of this error and
just picked one as an example, rather than trying to correlate it with
the other log lines.

I don't know much about SASL nor GSSAPI, but it looks like there's been
a few changes in cyrus-sasl related to GSSAPI over the last few years.
Maybe something changed in there.

Does the problem still occur if they use something other than GSSAPI?

#1079229#24
Date:
2025-05-28 10:01:25 UTC
From:
To:
Another question from upstream team:

   Are all GSSAPI connections to the 3.10.0 server broken, or only
   replication?

   * if all GSSAPI connections are broken, that may suggest something
     wrong with the GSSAPI support in SASL
   * if GSSAPI works otherwise and is only broken for replication, that
     may suggest sync_client is doing something wrong

#1079229#29
Date:
2025-05-28 10:04:24 UTC
From:
To:
It seems that the reporter address doesn't exist...


This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

   gombasg@digikabel.hu
     host inmx.digicable.hu [92.249.128.162]
     SMTP error from remote mail server after RCPT
TO:<gombasg@digikabel.hu>:
     550 Sorry, no mailbox here by that name.

#1079229#34
Date:
2025-05-28 10:58:41 UTC
From:
To:
can you update it?

i.

#1079229#39
Date:
2025-05-28 16:22:31 UTC
From:
To:
Hi,

no I can't change reporter mail address. He has now to follow manually
the responses and questions pushed to this bug and answer