#555735 resolvconf: Please don't use bash

#555735#5
Date:
2009-11-11 13:31:50 UTC
From:
To:
If I read correct, the only reason the script
/etc/resolvconf/update.d/bind uses bash is line:

  bind:   [ "$RSLT" ] && NMSRVRS="${RSLT// /; }; "

Please rewrite this like with "echo ... | sed ..." and the bash
requirement can be lifted.

#555735#12
Date:
2009-11-12 13:27:41 UTC
From:
To:
Thanks for the bug report.  Merging with #519364.  Please read the log
of #519364 for more information.

This wish will not be implemented until the package acquires a new
maintainer.

#555735#15
Date:
2011-09-22 21:54:23 UTC
From:
To:
(Revisiting this old report.)  Thanks again for your suggestion.

You wrote:

What's the advantage of lifting the bash requirement?

(I assume that by "bash requirement" you mean that the script
has #!/bin/bash" at the top rather than "#!/bin/sh".)

#555735#20
Date:
2011-09-23 05:09:59 UTC
From:
To:
On 2011-09-22 23:54, Thomas Hood wrote:
| (Revisiting this old report.)  Thanks again for your suggestion.
|
| You wrote:
| > Please rewrite this like with "echo ... | sed ..." and the bash
| > requirement can be lifted.
|
| (I assume that by "bash requirement" you mean that the script
| has #!/bin/bash" at the top rather than "#!/bin/sh".)

Correct.

| What's the advantage of lifting the bash requirement?

* If feature can be done in POSIX, there is no need to revert to
  big hammer "bash".
* POSIX compliant programs can be run also under
  dash, posh, mksh etc. E.g. Bare Bones Debian does
  not install Bash, but uses Dash only.
* Simplifying dependencies (to "bash" in this case).

Jari

#555735#23
Date:
2011-09-23 05:09:59 UTC
From:
To:
On 2011-09-22 23:54, Thomas Hood wrote:
| (Revisiting this old report.)  Thanks again for your suggestion.
|
| You wrote:
| > Please rewrite this like with "echo ... | sed ..." and the bash
| > requirement can be lifted.
|
| (I assume that by "bash requirement" you mean that the script
| has #!/bin/bash" at the top rather than "#!/bin/sh".)

Correct.

| What's the advantage of lifting the bash requirement?

* If feature can be done in POSIX, there is no need to revert to
  big hammer "bash".
* POSIX compliant programs can be run also under
  dash, posh, mksh etc. E.g. Bare Bones Debian does
  not install Bash, but uses Dash only.
* Simplifying dependencies (to "bash" in this case).

Jari

#555735#26
Date:
2011-09-23 09:28:36 UTC
From:
To:
This is not a good argument.  I can just as easily say:
if a feature can be implemented in bash then I don't
need to resort to sed.

People think that bash is slow, but it's much faster than
alternatives when its advanced features are needed.

For example, bash's ${foo/x/y} string substitution is
many times[*] faster than dash plus sed -e 's/x/y/' because
it avoids the overhead of pipes and forks.


If there were a stripped down version of Debian without
bash where resolvconf would be useful then that might
be a good reason to try to eliminate the dependency on bash.

Stripped down systems usually use busybox.  I notice
that busybox's ash shell does implement ${//}.  Hmm.

I can't find any information about Bare Bones Debian.
Do you have a URL for me?


This advantage is a rather academic considering that the
bash package is Essential in Debian.

The bash binary is less than one megabyte in size.  So the
disk-space saving is rather trivial.

[*] To see the difference in speed, consider the following script
"b", written in bash, and the result, "d", of translating it to dash.
Now observe the difference in running time.  The bash script
runs in less than one second; the dash script takes ten
minutes --- roughly seven hundred times slower.

$ echo === /tmp/b === ; cat /tmp/b ; echo ; echo === /tmp/d === ; cat
/tmp/d ; echo
=== /tmp/b ===
#!/bin/bash

let count=100000
aaa="foobarbaz"
while (( count-- )); do
	bbb=${aaa/bar/xxx}
done
echo $bbb

=== /tmp/d ===
#!/bin/dash

count=100000
aaa="foobarbaz"
while [ "$count" != 0 ] ; do
	count=$((count-1))
	bbb="$(echo "$aaa" | sed -e 's/bar/xxx/')"
done
echo $bbb

$ time /tmp/b
fooxxxbaz

real	0m0.844s
user	0m0.840s
sys	0m0.000s
$ time /tmp/d
fooxxxbaz

real	9m58.534s
user	0m8.260s
sys	0m16.340s

#555735#31
Date:
2011-09-23 10:45:27 UTC
From:
To:
On 2011-09-23 11:28, Thomas Hood wrote:
| On Fri, Sep 23, 2011 at 07:09, jari <jari.aalto@cante.net> wrote:
| > | What's the advantage of lifting the bash requirement?
| >
| > * If feature can be done in POSIX, there is no need to revert to
| >  big hammer "bash".
|
| This is not a good argument.  I can just as easily say:
| if a feature can be implemented in bash then I don't
| need to resort to sed.

Not correct. Every minimalistc system has standard tools: /bin/sh and
sed. But not bash. The benefit of minimalistic tools:

- Have less bugs
- Are portable
- Are rock solid
- Are understood by all (POSIX; that is).

Bash syntax is not something everyone groks. But Basic sed(1)
susbstitution commands are (exclusing the memory commands). Besides
sed is more versatile with regexps.

| Stripped down systems usually use busybox.

Not at all. It refers to plain /bin/sh too.

| I can't find any information about Bare Bones Debian.  Do you have a
| URL for me?

When you install Debian, you can install it without any extra
packages. That is bare bones Debian. It's shell is dash(1). See:

  Someone commented: "A faster and smaller default system shell is
  important to a lot of our users."
http://lists.debian.org/debian-devel/2009/07/msg00741.html

  "Switching the default /bin/sh to dash"
http://lists.debian.org/debian-devel/2009/06/thrd2.html#00767

  "Proposed release goal: Switch to dash as /bin/sh to speed up the boot"
http://lists.debian.org/debian-release/2007/07/threads.html#00027

  "Debian 6.0 release notes"
http://www.debian.org/releases/stable/i386/release-notes/ch-information.en.html#shell-diversions

| This advantage is a rather academic considering that the
| bash package is Essential in Debian.

I think that has been questioned in favor of Dash. It's only for
historical reasons that Bash were initially marked essential.

| The bash binary is less than one megabyte in size.  So the
| disk-space saving is rather trivial.
|
| [*] To see the difference in speed

In minimalistic systems it more question about CPU and memory than
speed. Bash is harder on system resources than Dash in old systems[*]

Jari

[*] We must always remember to think wider. Old PCs in africa, is
schools etc. Even my own /home directory server is running PII/64M and
to conserve memory it rund /bin/mksh, not bash as user shell.

#555735#34
Date:
2011-09-23 10:45:27 UTC
From:
To:
On 2011-09-23 11:28, Thomas Hood wrote:
| On Fri, Sep 23, 2011 at 07:09, jari <jari.aalto@cante.net> wrote:
| > | What's the advantage of lifting the bash requirement?
| >
| > * If feature can be done in POSIX, there is no need to revert to
| >  big hammer "bash".
|
| This is not a good argument.  I can just as easily say:
| if a feature can be implemented in bash then I don't
| need to resort to sed.

Not correct. Every minimalistc system has standard tools: /bin/sh and
sed. But not bash. The benefit of minimalistic tools:

- Have less bugs
- Are portable
- Are rock solid
- Are understood by all (POSIX; that is).

Bash syntax is not something everyone groks. But Basic sed(1)
susbstitution commands are (exclusing the memory commands). Besides
sed is more versatile with regexps.

| Stripped down systems usually use busybox.

Not at all. It refers to plain /bin/sh too.

| I can't find any information about Bare Bones Debian.  Do you have a
| URL for me?

When you install Debian, you can install it without any extra
packages. That is bare bones Debian. It's shell is dash(1). See:

  Someone commented: "A faster and smaller default system shell is
  important to a lot of our users."
http://lists.debian.org/debian-devel/2009/07/msg00741.html

  "Switching the default /bin/sh to dash"
http://lists.debian.org/debian-devel/2009/06/thrd2.html#00767

  "Proposed release goal: Switch to dash as /bin/sh to speed up the boot"
http://lists.debian.org/debian-release/2007/07/threads.html#00027

  "Debian 6.0 release notes"
http://www.debian.org/releases/stable/i386/release-notes/ch-information.en.html#shell-diversions

| This advantage is a rather academic considering that the
| bash package is Essential in Debian.

I think that has been questioned in favor of Dash. It's only for
historical reasons that Bash were initially marked essential.

| The bash binary is less than one megabyte in size.  So the
| disk-space saving is rather trivial.
|
| [*] To see the difference in speed

In minimalistic systems it more question about CPU and memory than
speed. Bash is harder on system resources than Dash in old systems[*]

Jari

[*] We must always remember to think wider. Old PCs in africa, is
schools etc. Even my own /home directory server is running PII/64M and
to conserve memory it rund /bin/mksh, not bash as user shell.

#555735#37
Date:
2011-09-23 12:04:11 UTC
From:
To:


OK.  I didn't know that your argument was really "It's best not to use
bash so that people can have the resolvconf package installed without
having the bash package installed (on a bare bones system).

Your bare bones Debian system is, you say, just a Debian system
without extra packages.

But the bash package is Essential, and so is also included on a
bare bones Debian system.

So it turns out that even if resolvconf doesn't use bash, the bash
package still has to be installed.  So there is no disk space saving.
What, then, was your argument?


I agree with the truism that superfluous complexity is to be avoided.

As for "less bugs" and "rock solid", there are factors other than code
complexity to be considered, including the degree to which the code has been
tested.  I have encountered several bugs in dash, posh and zsh over the
years but I have never hit a bug in bash.  Probably because bash is so
widely used and, thus, tested.

Bash syntax is better understood than POSIX syntax.  Bash has a decent man
page, more complete than dash's man page.  It actually requires quite a bit
of effort to find out which features of sh are POSIX and which ones aren't.

Bash patterns are as powerful as regexps when extglob is turned on.


Systems that are stripped down because of severe CPU, disk and memory
constraints, including every embedded or handheld Linux system I have ever
closely examined, use busybox.

For systems without such severe CPU, disk and memory constraints there is
simply no advantage to omitting bash from the system.

And bash shouldn't be omitted, since, as I showed, bash is so much better
than dash for some purposes.

Very interesting threads.  But they all have to do with switching the
/bin/sh symlink from bash to dash.  Great, I support that.  That means that
the vast majority of (simple) sh scripts will run faster.

But that is an entirely different question from the one here.  The question
here is: should resolvconf and programs like it (i.e., programs that
purposely specify #!/bin/bash and use non-POSIX bash features) stop using
bash?

My answer is no.


I'd be interested in URLs of discussions about this.  Who is suggesting that
bash cease to be Essential?  Currently it is a bug for a package (such as
resolvconf) to Depend on bash.  The moment bash ceases to be Essential, all
such packages become RC buggy because they fail to Depends on bash.  Such a
transition will take a lot of time and effort.

And for what benefit?  To make it possible for a few obsessives to save 1 MB
of disk space?


In minimalistic systems it more question about CPU and memory than


I prove to you that bash completes a task 700 times faster than a dash + sed
equivalent, and your reply is that speed doesn't matter, but CPU does?  I
hope you realize that that makes no sense.

Perhaps dash + sed uses less memory.  But every system that Debian has ever
run on had enough memory to load a bash image.

Anyway, thanks for the interesting discussion.

#555735#44
Date:
2011-09-23 15:55:33 UTC
From:
To:
On 2011-09-23 14:04, Thomas Hood wrote:
| > Not correct. Every minimalistc system has standard tools: /bin/sh and
| > sed. But not bash.
| But the bash package is Essential, and so is also included on a
| bare bones Debian system.

As said, bash is there for historical reasons only. The future is not
bash, it has switched to dash. Standard practices like POSIX are
always desireable over specialized ones.

| > The benefit of minimalistic tools:
| >
| > - Have less bugs
| > - Are portable
| > - Are rock solid
| > - Are understood by all (POSIX; that is).
|
| As for "less bugs" and "rock solid", there are factors other than code

In addition, the above implies:

- Smaller code base leads to better security
  (that's due to simplicity vs. bash's complex code base)

| I have encountered several bugs in dash, posh and zsh over the
| years but I have never hit a bug in bash.

I'm sure there are numerous bugs in Bash development mailing lists. In
much greater numbers in bash's whole life time compared to number of
total bugs found in dash.

| Bash syntax is better understood than POSIX syntax.  Bash has a decent man
| page, more complete than dash's man page.  It actually requires quite a bit
| of effort to find out which features of sh are POSIX and which ones aren't.

POSIX is the standard. Not Bash. POSIX shells work the same
everywhere.

You don't need to know what features exist in POSIX, as this is basic
shell programming. But you need to consult bash man pages if you want
to go beyond it.

| Bash patterns are as powerful as regexps when extglob is turned on.

We're drifting here. The question is not higher level language agaist
lower level. Naturally Perl has also more features than sh.

The bug in question is about a program, which with minimal efforts,
can be turned into POSIX compliant. My understanding is that the
program does not do anyting that specifically would require huge
amount of Bash features.

So why not make it POSIXly compliant? Small effort, big gain.

Jari

#555735#47
Date:
2011-09-23 15:55:33 UTC
From:
To:
On 2011-09-23 14:04, Thomas Hood wrote:
| > Not correct. Every minimalistc system has standard tools: /bin/sh and
| > sed. But not bash.
| But the bash package is Essential, and so is also included on a
| bare bones Debian system.

As said, bash is there for historical reasons only. The future is not
bash, it has switched to dash. Standard practices like POSIX are
always desireable over specialized ones.

| > The benefit of minimalistic tools:
| >
| > - Have less bugs
| > - Are portable
| > - Are rock solid
| > - Are understood by all (POSIX; that is).
|
| As for "less bugs" and "rock solid", there are factors other than code

In addition, the above implies:

- Smaller code base leads to better security
  (that's due to simplicity vs. bash's complex code base)

| I have encountered several bugs in dash, posh and zsh over the
| years but I have never hit a bug in bash.

I'm sure there are numerous bugs in Bash development mailing lists. In
much greater numbers in bash's whole life time compared to number of
total bugs found in dash.

| Bash syntax is better understood than POSIX syntax.  Bash has a decent man
| page, more complete than dash's man page.  It actually requires quite a bit
| of effort to find out which features of sh are POSIX and which ones aren't.

POSIX is the standard. Not Bash. POSIX shells work the same
everywhere.

You don't need to know what features exist in POSIX, as this is basic
shell programming. But you need to consult bash man pages if you want
to go beyond it.

| Bash patterns are as powerful as regexps when extglob is turned on.

We're drifting here. The question is not higher level language agaist
lower level. Naturally Perl has also more features than sh.

The bug in question is about a program, which with minimal efforts,
can be turned into POSIX compliant. My understanding is that the
program does not do anyting that specifically would require huge
amount of Bash features.

So why not make it POSIXly compliant? Small effort, big gain.

Jari

#555735#52
Date:
2011-09-23 20:37:42 UTC
From:
To:
The reason we are drifting is that you continue to introduce arguments that
have to be addressed because IMHO they are defective.

In your latest message, for example, you write that "standard practices like
POSIX are always desireable over specialized ones".  But you seem to admit
later that it's sometimes better to use Perl than sh.  Obviously it is.  In
the same way it is sometimes better to use bash than sh, e.g., when it saves
having to make expensive command pipelines.

I think that your various shifting claims amount to a prejudice against
bash. I don't happen to share this prejudice. I have worked on projects
where scripts were massively speeded up through use of bash's extended
features. That some people want to deprive themselves of that benefit just
seems bizarre to me.

I do agree that if a script can be rewritten to be POSIX-compliant *at no
cost* then it may be advantageous to do so, if only because dash runs pure
POSIX sh code about 25% faster than bash does. In some cases that speed
increase may be worth the effort.

As it happens, it looks as if it will be possible to make /sbin/resolvconf
POSIX-compliant at no significant cost (other than my time) so I will
implement this.  The list-records script will continue to use bash (so that
extended globs can continue to be used in the interface-order file).

Best wishes,

#555735#55
Date:
2011-09-23 20:37:42 UTC
From:
To:
The reason we are drifting is that you continue to introduce arguments that
have to be addressed because IMHO they are defective.

In your latest message, for example, you write that "standard practices like
POSIX are always desireable over specialized ones".  But you seem to admit
later that it's sometimes better to use Perl than sh.  Obviously it is.  In
the same way it is sometimes better to use bash than sh, e.g., when it saves
having to make expensive command pipelines.

I think that your various shifting claims amount to a prejudice against
bash. I don't happen to share this prejudice. I have worked on projects
where scripts were massively speeded up through use of bash's extended
features. That some people want to deprive themselves of that benefit just
seems bizarre to me.

I do agree that if a script can be rewritten to be POSIX-compliant *at no
cost* then it may be advantageous to do so, if only because dash runs pure
POSIX sh code about 25% faster than bash does. In some cases that speed
increase may be worth the effort.

As it happens, it looks as if it will be possible to make /sbin/resolvconf
POSIX-compliant at no significant cost (other than my time) so I will
implement this.  The list-records script will continue to use bash (so that
extended globs can continue to be used in the interface-order file).

Best wishes,