#377697 Provide runtime selectable comm plugin framework

#377697#5
Date:
2006-07-10 18:50:31 UTC
From:
To:
 	Today, I updated (on unstable) squid and the proxy never start
again...

 	At first, I try to take a look on configurations (becase some
syntax variables have been changed), but next, I use the clean default
configuration and fail too, so I taked a look on logs...

 	On syslog there is the next message:

Jul 10 20:44:52 springfield (squid): comm_select_init: epoll_create():
(38) Function not implemented
Jul 10 20:44:52 springfield squid[3873]: Squid Parent: child process 3878
started
Jul 10 20:44:52 springfield squid[3873]: Squid Parent: child process 3878
exited due to signal 6
Jul 10 20:44:55 springfield squid[3873]: Squid Parent: child process 3881
started
Jul 10 20:44:55 springfield (squid): comm_select_init: epoll_create():
(38) Function not implemented
Jul 10 20:44:55 springfield squid[3873]: Squid Parent: child process 3881
exited due to signal 6
Jul 10 20:44:58 springfield squid[3873]: Squid Parent: child process 3884
started
Jul 10 20:45:04 springfield squid[3873]: Exiting due to repeated, frequent
failures


 	Some information:

ii  libc6                   2.3.6-15              GNU C Library: Shared
libraries
ii  libc6-dev               2.3.6-15              GNU C Library:
Development Libraries and Hea
ii  squid                   2.6.1-2               Internet Object Cache
(WWW proxy cache)
ii  squid-common            2.6.1-2               Internet Object Cache
(WWW proxy cache) - co

Linux springfield 2.4.27 #1 Wed Oct 27 13:20:00 CEST 2004 i586 GNU/Linux

#377697#10
Date:
2006-07-10 23:43:43 UTC
From:
To:
severity 377697 important
thanks

[ I'm taking this to debian-devel seeking for broader consensus ]

Latest versions of squid (2.6.x and 3.0.x) support the epoll()
function that is provided by 2.6 kernels. This speeds up seek
operations on disk and thus squid performance, but is not supported
by older 2.4.x kernels.

Since this is a compile time choice and kernel 2.4.27 is still in the
archive we have the following options:

1. drop epoll() support
2. build multiple versions of squid (with and w/o epoll())
3. drop support for older kernels (will etch release with a 2.4
default kernel?)

Please give your advice.

Regards,

L

Il giorno 10/lug/06, alle ore 20:50, alex ha scritto:

#377697#17
Date:
2006-07-11 08:16:08 UTC
From:
To:
No, so it's reasonable not to care.
#377697#22
Date:
2006-07-13 01:26:06 UTC
From:
To:
Il giorno 11/lug/06, alle ore 17:58, Moritz Muehlenhoff ha scritto:

Marco d'Itri:

So in the end since 2.6 will not be supported in etch a note in
README.Debian about failing to support 2.4 is enough.

Thanks to all that shared their opinions.

#377697#31
Date:
2006-08-01 22:34:16 UTC
From:
To:
It shouldn't have to be; the consequences of installing a package depending
on 2.6-specific features onto a sarge system running a 2.4 kernel are clear,
and any package maintainer should be ashamed to ship a package with etch
that silently breaks the package on upgrade instead of providing a proper
upgrade path.

That's not appropriate, but I'm not sure if I'll treat it as RC.  I may be
beyond caring at this point, seeing the poor condition that so many Debian
packages are in today.

The upgrade path from sarge 2.6.8 to etch is *worse* than the upgrade path
from sarge 2.4.27 to etch, a fact that Marco d'Itri should be well aware of
given that udev is at the center of the problem.  To recommend that other
developers drop support for 2.4 in etch and ensure the upgrade path from 2.4
is just as bad is appalling.

#377697#36
Date:
2006-08-02 01:31:54 UTC
From:
To:
Hi Steve,

Il giorno 02/ago/06, alle ore 00:34, Steve Langasek ha scritto:

I could not find any statement on what would be the upgrade process
from sarge to etch. This is why I asked on debian-devel for advise on
how to resolve this issue.

The answer (not only the one from Marco), was that etch will not
support 2.4 anymore.

If this is the case etch will require an upgrade process involving
the kernel update before any user-space update.

If this is not the case, I'm left with no option to include the much-
better performing epoll() support in squid, since upstream is not
willing to integrate a runtime check in the short term and I cannot
support such an intrusive unofficial patch (which, BTW, does not
exist at all ATM).

Since you are the one that has the last word on this, should I revert
this change until etch+1?

Regards,

#377697#41
Date:
2006-08-02 01:31:54 UTC
From:
To:
Hi Steve,

Il giorno 02/ago/06, alle ore 00:34, Steve Langasek ha scritto:

I could not find any statement on what would be the upgrade process
from sarge to etch. This is why I asked on debian-devel for advise on
how to resolve this issue.

The answer (not only the one from Marco), was that etch will not
support 2.4 anymore.

If this is the case etch will require an upgrade process involving
the kernel update before any user-space update.

If this is not the case, I'm left with no option to include the much-
better performing epoll() support in squid, since upstream is not
willing to integrate a runtime check in the short term and I cannot
support such an intrusive unofficial patch (which, BTW, does not
exist at all ATM).

Since you are the one that has the last word on this, should I revert
this change until etch+1?

Regards,

#377697#46
Date:
2006-08-02 04:18:00 UTC
From:
To:
  "Kernel version 2.4 is deprecated and won't be shipped as part of etch,
  but user space applications will have to deal somehow with 2.4 kernels
  (because of upgrade path, local installations, ...)."

<http://lists.debian.org/debian-devel-announce/2006/07/msg00005.html>

I realize this particular message was posted to d-d-a after the thread in
question, but I don't see that it should come as a surprise to anyone --
least of all Marco, who as I said is certainly in a position to know the
problems we face with the sarge->etch upgrade path.  (The only other comment
in that thread saying it was ok to drop 2.4 support was from Moritz
Muehlenhoff; but you also got feedback from Stephen Gran and Mark Brown,
saying that a runtime check would be preferable.)

That's absolutely out of the question.  Most systems will be unbootable when
trying to upgrade to a 2.6 kernel before upgrading any of the userspace.
And while it's *possible* that we could say "install busybox-cvs-static,
upgrade glibc, and install udev and initramfs-tools, but don't touch slapd,
squid, or rageircd; install the 2.6 kernel; reboot and continue the
upgrade", I have two problems with that: first, it's much more complicated
than I think any Debian upgrade should be considering that users don't
actually read the release notes before upgrading, and second, I have no
faith in this upgrade path remaining intact until the release of etch given
the current approach of our kernel team.

I realize this isn't an answer that's likely to make you happy, but yes,
given those choices I think it's better to disable epoll() entirely.  It may
not be the most whizz-bang option, but with the current release cycles
Debian can't win at whizz-bang anyway.  Where it *can* win is on being
solid, and solid upgrade support is part of that.

Why do you say that it would be intrusive?  It looks to me like a simple
change to support building more than one select interface at a time, and
using the best one that works.  If such a patch existed, would you consider
applying it?

I only have the last word on it if I make it a release critical bug, which
as I've said, I'm not sure I want to do.

Thanks,

#377697#51
Date:
2006-08-02 05:43:07 UTC
From:
To:
I'd surely be willing to do so for rageircd.

Greetings
Marc

#377697#56
Date:
2006-08-02 09:56:45 UTC
From:
To:
Il giorno 02/ago/06, alle ore 06:18, Steve Langasek ha scritto:

Could user application 'deal somehow' with 2.4 kernels making the
administrator aware that they will need a 2.6 kernel to work at all?

Squid has a comm interface with different modules (poll, select and
epoll on linux, kqueue on freebsd, etc) choosen at build time. I
agree with upstream that compiling more than one comm module takes a
big change in one of the critical section and they are not willing to
make such a change to a stable release.

In alternative a new comm module based on libevent could be added.
libevent would support both epoll() and poll() and is in debian as of
now. I don't have the skills to create such a patch, but would
happily propose it upstream and include in the debian package.

Thanks,

#377697#61
Date:
2006-08-02 10:35:16 UTC
From:
To:
Same with rageircd, but with the addition, that rageircd will detect
libepoll at build time and automatically use it. libepool, however,
does not seem to be in Debian.

Greetings
Marc

#377697#66
Date:
2006-08-02 20:48:01 UTC
From:
To:
When will you make the administrator aware of this that won't make the
upgrade path ugly?

- if you error out in the preinst, the old version of squid continues
  running, but the dist-upgrade will break in the middle
- if you error out in the postinst, the dist-upgrade completes, but the
  administrator may then be looking at unexpected downtime related to trying
  to upgrade the kernel before squid will run again

I dunno, the 538 line patch I have here to support runtime selection between
kqueue, epoll, and select (as enabled) doesn't seem like such a big change
to me. :)

Well, unlike the patch I just created, writing this would require
understanding the APIs of both libevent and squid's internal comm interface.
That would take a bit longer to write & debug, compared with just providing
a basic plugin interface. <shrug>

#377697#71
Date:
2006-08-02 23:45:44 UTC
From:
To:
Il giorno 02/ago/06, alle ore 22:48, Steve Langasek ha scritto:

Could it be stated in the Release Notes? Something like 'Upgrading
from sarge with a 2.4 kernel expect some daemons to stop working
until a new kernel is installed'.

I simply gave you the answer I got from upstream when asking for the
same question. libevent was easier and preferred.

Can I have a look at your patch and, if you will, permission to
submit it upstream?

Regards,

#377697#76
Date:
2006-08-04 11:22:19 UTC
From:
To:
Is there a suitable 2.4 based test system accessible to developers
somewhere?

What reasons did upstream give for this decision?  I've often found
people are much more flexible about this sort of thing when presented
with a patch but it sounds like there was a more fundamental objection
here.

#377697#81
Date:
2006-08-03 22:27:54 UTC
From:
To:
As mentioned, the release notes are not a reliable method of communicating
with our users because users take a perverse pleasure in not looking at the
documentation until *after* something breaks.

Absolutely.  A preliminary patch is attached; if upstream is interested in
this approach, please give me the opportunity to clean it up a bit more and
port all of the comm implementations to it, but otherwise you may consider
it available under the same license as squid itself.

Cheers,

#377697#86
Date:
2006-08-14 20:27:39 UTC
From:
To:
/usr/sbin/squid -D -sYCNX
gives
FATAL: comm_select_init: epoll_create(): (38) Function not implemented

#377697#91
Date:
2006-08-16 13:59:40 UTC
From:
To:
Hi,

the very same problem occurs here with a machine running kernel 2.6.17.3!

As far as I can see, EPOLL should be available with this kernel:

However, squid 2.6.2-1 (as well as 2.6.1) won't start up and logs the same
error message:

Version 2.6.9-10sarge2 of squid works fine, by the way.

Two important points to note:
- the kernel is not a debian standard kernel, but home-made
- the machine is a 64bit Alpha

Yours,
Andreas.

#377697#96
Date:
2006-08-18 21:00:43 UTC
From:
To:
I need squid with support for 2.4 kernel because the 2.6 kernels (for
686-smp) do not support AVM B1 ISDN cards (and similar active cards)


Bodo

#377697#101
Date:
2006-08-22 18:21:00 UTC
From:
To:
forwarded 377697 http://www.squid-cache.org/bugs/show_bug.cgi?id=1739
thanks

Hi Steve,

Thanks for your patch. I forwarded it to upstream and am waiting for
directions. I will return to you as soon as I get an answer.

Regards,

#377697#106
Date:
2006-12-13 18:17:44 UTC
From:
To:
I am getting the same problem with a system I updated from sarge to etch
yesterday.

Dec 13 12:12:37 huey squid[13918]: Squid Parent: child process 13924 started
Dec 13 12:12:37 huey (squid): comm_select_init: epoll_create(): (38)
Function not implemented
Dec 13 12:12:37 huey squid[13918]: Squid Parent: child process 13924
exited due to signal 6
Dec 13 12:12:40 huey squid[13918]: Squid Parent: child process 13926 started
Dec 13 12:12:40 huey (squid): comm_select_init: epoll_create(): (38)
Function not implemented
Dec 13 12:12:40 huey squid[13918]: Squid Parent: child process 13926
exited due to signal 6
Dec 13 12:12:43 huey squid[13918]: Squid Parent: child process 13930 started
Dec 13 12:12:43 huey (squid): comm_select_init: epoll_create(): (38)
Function not implemented
Dec 13 12:12:43 huey squid[13918]: Squid Parent: child process 13930
exited due to signal 6
Dec 13 12:12:46 huey squid[13918]: Squid Parent: child process 13932 started
Dec 13 12:12:46 huey (squid): comm_select_init: epoll_create(): (38)
Function not implemented
Dec 13 12:12:46 huey squid[13918]: Squid Parent: child process 13932
exited due to signal 6
Dec 13 12:12:49 huey squid[13918]: Squid Parent: child process 13934 started
Dec 13 12:12:49 huey (squid): comm_select_init: epoll_create(): (38)
Function not implemented
Dec 13 12:12:49 huey squid[13918]: Squid Parent: child process 13934
exited due to signal 6
Dec 13 12:12:49 huey squid[13918]: Exiting due to repeated, frequent
failures

david@huey:~$ uname -a
Linux huey 2.4.26 #1 Wed Jul 7 23:13:45 CDT 2004 i686 GNU/Linux
david@huey:~$ dpkg -p squid
...
Version: 2.6.5-2

I will try updating the kernel, but this will be a definite problem for
people upgrading from sarge to etch once etch releases.

#377697#111
Date:
2007-04-10 14:09:28 UTC
From:
To:
severity 377697 wishlist
severity 380652 wishlist
retitle 377697 Provide runtime selectable comm plugin framework
retitle 380652 Provide runtime selectable comm plugin framework
forward 377697 http://www.squid-cache.org/bugs/show_bug.cgi?id=1739
forward 380652 http://www.squid-cache.org/bugs/show_bug.cgi?id=1739
thanks

Hi,
Etch has been released with a notice in the Release Notes that kernel
2.6 is required to run Squid. Since Debian doesn't support < 2.6
anymore, these bugs are being converted to wishlist bug for a new
comm framework allowing for selectable plugins and forwarded to the
upstream bug record.

A major rework of the storage framework is going on upstream, se
expect this process to take a long time.

Regards,