vsftpd 3.0.5: s_children unsigned underflow causes false "too many connected users" errors
Hi Maintainers,
Title
vsftpd 3.0.5: s_children unsigned underflow causes false "too many connected users" errors in standalone mode
Product
vsftpd 3.0.5
Component
Standalone mode (standalone.c)
Severity
Major
Summary
During FTPS testing in standalone mode, the server intermittently began rejecting new connections with:
421 There are too many connected users, please try later.
even though:
no user sessions remained active
no worker processes remained running
max_clients was configured to a value far higher than the actual connection count
Investigation showed that the session count had wrapped to:
4294967295
which indicates an unsigned integer underflow.
Environment
vsftpd 3.0.5
Standalone mode
FTPS enabled
PAM authentication
Client certificate authentication
TLS 1.3
Configuration included:
max_clients=300
Symptoms
After repeated connect/login/list/quit operations, new connections were rejected:
421 There are too many connected users, please try later.
Diagnostic logging showed:
Connection refused: too many sessions. num_clients=4294967295
The value:
4294967295
is:
0xffffffff
which corresponds to unsigned underflow from zero.
Reproduction Steps
Start vsftpd 3.0.5 in standalone mode.
Configure FTPS.
Configure PAM authentication.
Connect using an FTPS client.
Repeatedly perform:
CONNECT
LOGIN
LIST
QUIT
Example test command:
curl -v --insecure \
--cert client.crt \
--key client.key \
--user username:password \
ftps://server:990/
Repeat several times.
Eventually the server begins refusing new sessions despite there being no active users.
Evidence
Session limit reached unexpectedly
Server log:
CONNECT: Client "x.x.x.x"
Connection refused: too many sessions.
421 There are too many connected users, please try later.
Additional diagnostic logging showed:
num_clients=4294967295
No active worker processes
At the time of failure:
ps -ef | grep vsftpd | grep -v grep
returned only the listener process:
root 1 0 ... /path/to/vsftpd ...
No child session processes remained.
Root Cause Analysis
The issue appears to originate in:
static void
handle_sigchld(void* duff)
within standalone.c.
Current code:
/* Account total number of instances */
--s_children;
If s_children is already zero when this code executes, the value wraps:
0 -> 4294967295
Subsequent connection accounting then reports an impossibly large client count and the server rejects connections as if the configured limit has been exceeded.
Proposed Fix
Guard the decrement operation.
Existing code
/* Account total number of instances */
--s_children;
Proposed change
/* Account total number of instances */
if (s_children > 0)
{
--s_children;
}
Test Results After Patch
After applying the guard:
if (s_children > 0)
{
--s_children;
}
the following tests completed successfully:
CONNECT
LOGIN
LIST
QUIT
performed repeatedly over FTPS using client certificates and PAM authentication.
Observed results:
No further 4294967295 values observed
No further "too many connected users" errors
No accumulation of worker processes
Login, directory listing and logout continued to function normally
Additional Question
Is there any expected code path where handle_sigchld() can execute when s_children == 0, or is the underflow indicative of another accounting path that may also require investigation?
Minimal Patch
--- a/standalone.c
+++ b/standalone.c
@@
- --s_children;
+ if (s_children > 0)
+ {
+ --s_children;
+ }
Log Excerpt
... FTP response: "226 Directory send OK."
... FTP response: "221 Goodbye."
CONNECT: Client "x.x.x.x"
Connection refused: too many sessions.
num_clients=4294967295
FTP response:
421 There are too many connected users, please try later.
Expected Behaviour
Once all child processes have exited and no active sessions remain, new connections should continue to be accepted and internal session accounting should never underflow below zero.
minimal relevant configuration excerpt
listen=YES
ssl_enable=YES
guest_enable=YES
pam_service_name=vsftpd.virtualmax_clients=300require_cert=YES
force_local_logins_ssl=YES
force_local_data_ssl=YES
Authentication: PAM
FTPS: enabled
Client certificates: enabled
Mode: standalone
Configuration omitted for brevity. The issue was reproduced in standalone mode with FTPS, PAM authentication and client certificate authentication enabled. A full redacted configuration can be provided if required.
Kind regards
Robert Jennings
Marine Software Limited