The size limitation specificed in the cache_dir statement in the squid
configuration seem to be ignored, as squid uses more than the provided
space on the disk an exits when it is unable to write files.

I discovered this in Debian Edu while installing a client with squid
as a http proxy.  We use a script
/usr/share/debian-edu-config/tools/squid-update-cachedir to update the
squid configuration and tell squid to use 80% of the available space
in the partition.  But squid uses more than that and end up failing
after a while.

See here from a run when a client is using the proxy and filling up
the disk:

root@tjener:/etc# cat /etc/default/squid3
CONFIG=/etc/squid3/squid-debian-edu.conf
SQUID_ARGS="-YC -f /etc/squid3/squid-debian-edu.conf"
root@tjener:/etc# grep cache_dir squid3/squid-debian-edu.conf
# maximum_object_size, append_domain and cache_dir settings.
cache_dir ufs /var/spool/squid3 567 16 256
root@tjener:/etc# df -h /var/spool/squid3
Filesystem                              Size  Used Avail Use% Mounted on
/dev/mapper/vg_system-var+spool+squid3  709M  519M  148M  78% /var/spool/squid3
root@tjener:/etc# df -h /var/spool/squid3
Filesystem                              Size  Used Avail Use% Mounted on
/dev/mapper/vg_system-var+spool+squid3  709M  544M  122M  82% /var/spool/squid3
root@tjener:/etc# df -h /var/spool/squid3
Filesystem                              Size  Used Avail Use% Mounted on
/dev/mapper/vg_system-var+spool+squid3  709M  604M   62M  91% /var/spool/squid3
root@tjener:/etc# service squid3 status
* squid3.service - LSB: Squid HTTP Proxy version 3.x
   Loaded: loaded (/etc/init.d/squid3)
   Active: active (exited) since Wed 2014-10-15 15:01:06 CEST; 12min ago
  Process: 8323 ExecStop=/etc/init.d/squid3 stop (code=exited, status=0/SUCCESS)
  Process: 7440 ExecReload=/etc/init.d/squid3 reload (code=exited, status=0/SUCCESS)
  Process: 8416 ExecStart=/etc/init.d/squid3 start (code=exited, status=0/SUCCESS)

Oct 15 15:06:45 tjener.intern (squid-1)[8731]: Write failure -- check your disk space and cache.log
Oct 15 15:06:45 tjener.intern squid[8428]: Squid Parent: (squid-1) process 8731 exited with status 1
Oct 15 15:06:48 tjener.intern squid[8428]: Squid Parent: (squid-1) process 8739 started
Oct 15 15:06:49 tjener.intern (squid-1)[8739]: Write failure -- check your disk space and cache.log
Oct 15 15:06:49 tjener.intern squid[8428]: Squid Parent: (squid-1) process 8739 exited with status 1
Oct 15 15:06:52 tjener.intern squid[8428]: Squid Parent: (squid-1) process 8743 started
Oct 15 15:06:52 tjener.intern (squid-1)[8743]: Write failure -- check your disk space and cache.log
Oct 15 15:06:52 tjener.intern squid[8428]: Squid Parent: (squid-1) process 8743 exited with status 1
Oct 15 15:06:55 tjener.intern squid[8428]: Squid Parent: (squid-1) process 8755 started
Oct 15 15:06:55 tjener.intern (squid-1)[8755]: Write failure -- check your disk space and cache.log
You have new mail in /var/mail/root
root@tjener:/etc#

Hi Petter,

Thanks for your bug report.

Can you please provide some more details so when can track down where the issue comes from?

  * Is there any other file in /var/spool/squid3?
  * Can you provide the output of "squidclient mgr:storedir”?

Best regards,

L

Hello,

I can confirm this behavior. It occurs simply during normal usage.

I am using:

  cache_dir aufs /var/spool/squid3 10000 16 32

However the cache dir on one host was way over that, close to 30 GB. Luckily
that partition still had a considerable amount of free space.

What filesystem type is in use for the device the cache_dir is stored on?

Upstream have known issues with ZFS and XFS where the underlying FS uses
different amounts of storage space for files than gets reported to
Squid. This usually results in smaller on-disk than Squid is aware, but
it could as easily result in much larger sizes.

Amos

Upstream work recently indicates that this may also be related to
cache_swap_high garbage collection issues.

BUT we still need information about what HDD format has been used, and
now also what traffic rates are happening at the time of overflow.

Amos