Dear Maintainer,

I have just upgraded mariadb-client (and -server) from version 10.3.x to
version 10.5.8-3. As far as I can tell, no versions between those made
it into testing. If I'm not mistaken, the last version I was running
pre-upgrade was 10.3.24-2.

Prior to the upgrade, in an interactive session within the 'mysql'
terminal-based client, Ctrl+W would kill everything to the left of the
cursor up to the first word boundary (which in practice appeared to mean
"whitespace"), but nothing to the left of that point.

This is apparently not the default, but I have it configured in
~/.editrc. The contents of that file are as follows:
---8<---
bind "^U" vi-kill-line-prev
bind "^W" ed-delete-prev-word
---8<---

Following the upgrade and a restart of the client, in such an
interactive session, Ctrl+W now kills everything to the left of the
cursor, all the way to the beginning of the line. This appears to be the
default behavior, used when no alternate configuration has been applied.

It thus appears as if the settings I have in place in ~/.editrc are
simply being ignored. The environment variable EDITRC is not set.
Launching with it explicitly set does not appear to change anything.


I've found this behavior mentioned in a few places online. Nearly all of
them seem to advise resolving it by adding that Ctrl+W bind in
~/.editrc. The only one I've found thus far which indicates that doing
so does not change the behavior is

https://bugs.mysql.com/bug.php?id=95287

in which the solution found is to build with the internal libedit rather
than the system copy, which is obviously not suitable for Debian.

The only mentions of 'edit' in the relevant changelogs that I've found
are in those for mariadb-client-core-10.5. Version 1:10.4.12-1~exp2 has
an entry for "Link with libedit instead of readline5", and version
1:10.5.5-2 has one for "Add native dependencies on gnutls, libedit and
ncurses" (in relation to cross-compiling). I'd guess that the former is
the more relevant of the two, but I don't know for sure.

The installed version of libedit2 is 3.1-20191231-1. Based on the date,
I doubt that it was recently upgraded from an earlier version, but I
cannot testify to that for certain.


The only mention of ~/.editrc being ignored that I've managed to find
yet is a question on superuser.com from 2010, regarding ghci, which got
no answer.


I'm not sure whether the bug here is in mariadb-client or in libedit, or
even somewhere else, but this is clearly a behavior regression.

If the existing configuration through this file is supposed to continue
to be respected after the upgrade, please make appropriate adjustments
so that that happens.

If it is not, please explain and document what the correct method is for
applying such configuration.

If there is anything I can do to help track this down and get it
working, please let me know. I've tried strace, but failed to find
anything relevant thus far in the result; in particular, I find no
mention of the string 'editrc', although libedit.so.2 is apparently
read.

On further investigation, this might not be the full explanation after
all.

According to 'man editline', the ed-delete-prev-word function is bound
by default to Ctrl+Meta+H. When I execute that key combination, it kills
not up the preceding whitespace (as Ctrl+W used to do), but to the
preceding word boundary (using what seems to be the same criteria as
would be used by '\b' in a regular expression).

https://certif.com/cplot_print/libedit.html and
https://www.certif.com/spec_print/editline.html document the "kill
until previous whitespace" functionality as being provided by
em_kill_region. 'man editline' refers to em-kill-region, but states that
it kills everything between the cursor and "the mark"; since the mark
must be explicitly placed and omitting it means invoking this function
is an error, that doesn't seem to be equivalent to "kill until previous
whitespace".

The latter of those pages documents the readline equivalent of
ed-delete-prev-word as being backward-kill-word, and the one for "kill
until previous whitspace" as being unix-word-rubout. I haven't found a
way to readily test the behavior of these on my system yet, so I don't
know for sure which if any of them provide the behavior I expected.


I'm coming to suspect that there may be no way to regain access to the
previous behavior without either severely patching libedit (either
locally or upstream) or reverting to have mariadb-client use readline
again instead. Either of those things would be drastic, but for this
regression to be permanent would be quite unfortunate.

Either way, the fact that ~/.editrc does not seem to be being respected
is still an issue, which may or may not reside in mariadb-client. This
additional discovery may simply mean that it was being ignored
previously as well, I just hadn't noticed.


I filed this as a bug report on account of regression, but it's starting
to look like it might belong more in more general discussion channels
for the time being, until the actual root nature of the regression can
be figured out...

Thanks for reporting and debugging this. There are no libedit related
customizations in the Debian packaging, so I suggest you report this
upstream at jira.mariadb.org, where upstream devs might have some input
into this.

Otto

Hi,
upstream at jira.mariadb.org, where upstream devs might have some input
into this.

Could this be related to:

This issue has not seen any progress.

If you want to help improve MariaDB in Debian in the open source way,
you could for example:

- submit your suggestion for a fix as a Merge Request at
https://salsa.debian.org/mariadb-team/mariadb-10.5
- help with documentation/testing to improve our understanding on what
exactly the bug is about
- triage the other bugs filed against MariaDB in Debian so there is
time freed up to work on this bug

Thanks!

On Thu, 26 Nov 2020 12:50:26 -0500 The Wanderer <wanderer@fastmail.fm> wrote:

Hi, this is also bugging me on Fedora 32.  Digging deep into the
library chain and source code, it almost certainly is a libedit issue.
Linux doesn't have issetugid(), and in src/el.c in libedit
(2019-something thru 20210419 versions) it clearly only reads editrc if
issetugid is available.

The upstream here is http://thrysoee.dk/editline/, and it has this
changelog:

====
2013-07-10 Jess Thrysoee

   * version-info: 0:46:0

   * configure.ac, src/el.c: ~/.editrc was never sourced on Linux.
	On Linux issetugid is not available. When unable to determine if
	the current process is tainted, we did not trust the HOME
     environment variable and therefore could not load ~/.editrc.
	Now instead use secure_getenv or a issetugid based
     implementation of secure_getenv. Patch by Paolo Tosco.
====

Which means at one time this was fixed.  That's why this used to work
for us.  But something must have regressed in the source, and the fix
code which I think is the stuff dealing with HAVE_SECURE_GETENV defines
near the top of el.c doesn't actually help at all anymore.

A quick fix would be to fudge the source having to do with issetugid to
just not check the ifdef nor call the function.  Of course, that could
be a security hole if somehow it gets run in a setuid/gid context.

I have contacted the thrysoee.dk maintainer about getting this fixed in
his version, which will automatically trickle down to fix this for us
in deb & fedora.

I confirmed I was correct by making a small patch to libedit and
recompiling libedit and installing the custom rpm.  It indeed fixes
this bug, without any changes to mysql.  The bug is for sure in
libedit.  The bug is for sure the issetugid-related code.

My patch to libedit-20210419-3.1.tar.gz is attached.

Note: this patch is almost certainly a security hole as I don't address
the reason that setuid stuff is in there in the first place.  Consider
it a PoC and not something you should use unless you know what you are
doing.  I would expect the upstream to come up with a real fix shortly.

Op za 22 mei 2021 om 04:27 schreef Trevor Cordes <debian@tecnopolis.ca>:

MariaDB 10.3 uses libreadline-gplv2
MariaDB 10.5 uses libedit
I think this explains the changes in behavior.

libreadline or libedit?  In any event, I experienced these symptoms a
few years ago with MariaDB, and at that time (going from memory) it was
because Fedora changed from compiling with readline to libedit.  And at
that time the solution was to make an .editrc with the bindings you
wanted.  Since I made an .editrc, and that fixed the symptoms, that
proves it was using libedit before this current fiasco.  That's my
theory, anyhow.

It's not two states we're dealing with, it's three:
1. MariaDB uses libreadline
2. MariaDB uses a working libedit that reads .editrc
3. MariaDB uses a broken libedit that doesn't read .editrc

We're in state 3 right now, not state 2.

That explains why the "fix" of using the included-with-MaraiDB libedit
solves the symptoms: because that's a non-broken libedit that reads
editrc.  But it's always better (IMHO) to use the system libraries, so
it seems prudent to get libedit fixed upstream.  For instance, if
libedit fixes a CVE, does MariaDB remember to patch or update their
in-tree version??

Upstream Jess Thrysoee has already gotten back to me with a better patch
and we're testing now.  So it seems like this won't be an issue soon
enough, once distros take on the upstream changes.

Upstream, Jess and I did work out a fix and I made a "real" patch that
should fix the problem without a security hole.

http://thrysoee.dk/editline/

As projects pull from this upstream (as Fedora does, not sure about
Deb) they will get this fix.  If you don't want to wait, download the
source and compile/install or use your package manager build system
with this new source.  Restart your mysql client and the bug should be
gone: libedit should read your editrc, and from there you can customize
^W to do the "sane" thing.

So now mariadb should allow sane ^W settings now matter how it is
compiled:
- with readline: always worked
- with libedit that comes with maria-db: always worked
- with your system libedit: now works once you add settings to .editrc
  and use this latest libedit verion (libedit-20210522-3.1 or
  newer)

And hopefully that will be the last we ever see of this bug, which has
cropped up every few years since 2009, 2013, 2019, 2021?  :-)

Cheers.

Op za 22 mei 2021 om 23:54 schreef Trevor Cordes <trevor@tecnopolis.ca>:

It'd be nice if ^W worked by default.

I support idea that ctrl-w should retain behaviour that it had all
those years in libreadline5, until it was removed in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=980504
and libedit2 made an drop-in replacement.

Could we please change the default keybindings in Debian libedit2 itself,
instead of having to do same as having to create following ~/.editrc
for all users on all systems in order for seamless transition?

bind "^W" ed-delete-prev-word
bind "^_" vi-undo

Those two would at least (mostly) restore behaviour of two most
missing/changed features.

Modifying .editrc seems like a change that could have a lot of side
effects. Maybe "backwards compatibility" or migration from readline to
libedit should be handled by libedit in some more generic way so we
don't have to adapt in MariaDB packaging?

Or should we maybe go back to readline if it is better?

Hello!

If somebody wants to continue to work on this issue[1], please submit
your packaging improvement suggestion as a Merge Request on Salsa[2].
I promise to review them promptly.

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=975911
[2] https://salsa.debian.org/mariadb-team/mariadb-server/-/wikis/Contributing-to-MariaDB-packaging-in-Debian

From https://thrysoee.dk/editline/:

2021-05-22 Jess Thrysoee
* version-info: 0:66:0
* all: sync with upstream source
* src/el.c: editrc not read on systems without issetugid
Patch by Trevor Cordes

bookworm (and later) defaults to old behavior, so does buster with the
libedit2 from bookworm backported.

Also, seems to be a libedit2 issue rather than mariadb-client.

Regards,
     Gerardo

Hi!
If there is a consensus that this is not a mariadb-client bug, I vote
for either:
- closing this bug report;
- tagging "wontfix".

@Otto, I'll let the final decision to you.

Faustin

Hi!

Latest version of MariaDB in Debian unstable/testing/Bookworm is
10.11.2. You might want to consider testing it.

If you want to contribute in the open source way to fix this or any
other issue, see
https://salsa.debian.org/mariadb-team/mariadb-server/-/wikis/Contributing-to-MariaDB-packaging-in-Debian
on how to submit a Merge Request!


If you have time to help, please consider these (in order of importance):

1. Review current open MRs at
https://salsa.debian.org/mariadb-team/mariadb-server/-/merge_requests

2. Review items highlighted by Debian QA systems (Lintian, builds etc)
and submit a fix to improve the quality:
https://tracker.debian.org/pkg/mariadb

3. Review what testing we have at
https://salsa.debian.org/mariadb-team/mariadb-server/-/pipelines and
think about potential gaps - CI is very important as it is the only
way we can prevent regressions in a scalable way

4. Review/follow-up on existing bugs that currently need more
information: https://bugs.debian.org/cgi-bin/pkgreport.cgi?repeatmerged=no&src=mariadb&src=mariadb-10.6&src=mariadb-10.5&src=mariadb-10.3&src=mariadb-10.1

MariaDB and C++ skills are useful, but not required. For example
reviewing the NEWS for 10.11 requires no coding skills:
https://salsa.debian.org/mariadb-team/mariadb-server/-/merge_requests/37


My request for help from debian-devel in
https://lists.debian.org/debian-devel/2023/02/msg00272.html did not
get many responses, so the future of this package depends on how
active the users and people who previously reported bugs are in
participating in the maintenance of the package.

- Otto

Hi!

This issue is still unresolved.

Just to recap,  the switch from readline to libedit was due to a
forced change across all of Debian, see:
- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=940879
- https://salsa.debian.org/mariadb-team/mariadb-server/-/commit/d8796a0e3b5fc4450f4dc80fd159599a7f03997a
(by Bastian, CCd)

As a result people reported
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=975911 and
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=992560

We can go back to readline or prepopulate libedit with the same
keybindings as readline used to have if there is somebody who wants to
contribute that change and do the required testing.

The optimal fix is probably to contribute to
https://salsa.debian.org/debian/libedit (maintainer Sylvestre CC'd)

- Otto

Unfortunately, mariadb's GPLv2-only license is not compatible with readline's GPLv3-or-later.
The old readline v5 was deliberately removed. Please do not reintroduce it.

That leaves us with prepopulating libedit.