Dear Maintainer,

  (This is a resubmission; the previous bug may have been lost to spam
filtering.)

  As of the upgrade to 29.1 on 2023-09-06, Emacs seems to be
disregarding some user preferences for the font for the "default"
face.

* What led up to the situation?

The upgrade to Emacs 29.1 on 2023-09-06.

* What exactly did you do (or not do) that was effective (or ineffective)?

Executing the following steps:

- (customize-face 'default)
  - Font Family: setting "Terminus (TTF)"
  - Font Foundry: setting "PfEd"
  - Optionally (does not affect outcome):
    - Weight: setting "medium"
    - Disabling any font attributes (inlcuding Weight)
  - [Apply]

Equivalently:
(custom-set-faces
 '(default ((t (:inherit nil :extend nil :stipple nil :background "black" :foreground "white" :inverse-video nil :box nil :strike-through nil :overline nil :underline nil :slant normal :weight medium :height 120 :width normal :foundry "PfEd" :family "Terminus (TTF)"))))
 )

* What was the outcome of this action?

Emacs used the "Purisa" font as default font.  This font has the same Font
Foundry as "Terminus (TTF)" but is a "Comic Sans"-like special-purpose font
and unsuitable for normal operations.

* What outcome did you expect instead?

I expected Emacs 29.1 to honour my configuration settings, as Emacs 28 did
before.

* Observations

- This only affects the default face.  Based on my attempte to debug the
  problem (see below), the bug is caused by special treatment for the
  default face.
- This bug is likely related to #1029710, which was reported at the same
  time as my original (regrettably spam-filtered?) report.
- I have narrowed down the bug somewhat (see below) and am using a
  local workaround.
- This issue affects emacs-gtk and emacs-pgtk equally.
- This issue affects X11 and wayland equally.

* Debugging results

** Why this seems to happen

Here is the flow of events that leads to the problem, to the best of
my (very limited) understanding:

1. At some point, the font spec for the default face is set up
2. I set my preferences for the default face
3. My preferences are applied to the default face spec in some order
4. During "Font Family" selection, the following happens at some point:
  a) While searching for suitable fonts, Emacs calls font_list_entities(f, spec)
  b) font_list_entities(f, spec) (font.c, L2540) does some
   preprocessing and then:
     1. asks the driver for a list `vec' of suitable fonts (L2585)
     2. and filters out unsuitable fonts (L2602), calling
  c) font_delete_unmatched(vec, spec, size) in turn scans the `vec' to
   remove elements that don't match `spec'.
    1. Specifically, for properties like font weight
       (FONT_WEIGHT_INDEX), L2486 checks if the requested property is
       an exact match and otherwise removes the candidate (L2502).
    2. I assume that L2477 skips checks for properties that are left
       unconstrained (nil) in the spec but have not verified that.

The unexpected behaviour happens at 4.c:

4.c.2 always seems to allow filtering by font weight, no matter what I
      select in the font face

4.c.1 always seems to require a font weigth of 80 ("regular"), no
      matter what I select in the font face interface.

I have tried to investigate further to check:

- Does the wrong weight come from a hardcoded value?
- Does the wrong weight come from a stale font spec attribute?

*** Hardcoded value?

I tried modifying the `font_weights' table in font.c, but was only
able to "fix" the behaviour by entirely removing the entry for
weight 80.  My interpretation is that the value "80" probably comes
from one of the default fallback fonts and is accidentally retained in
the font spec (i.e., the weight doesn't seem to be hardcoded
anywhere in Emacs).

*** Stale font attribute?

I traced the calls to `internal-set-lisp-face-attribute' immediately
after updating the font.  Below is the order in which the attributes
are set (some appear more than once):

  :underline
  :overline
  :strike-through
  :box
  :inverse-video
  :stipple
  :inherit
  :extend
  :family
  :foundry
  :inherit
  :extend
  :stipple
  :background
  :foreground
  :inverse-video
  :box
  :strike-through
  :overline
  :underline
  :slant
  :weight
  :height
  :width

Note how :family is set before :weight is updated (to the correct
value):

- After setting :family, the font spec is associated with a font
  object for the "Purisa" font, but at least retains :family
  (of the font that I requested).

- Once :weight is updated, internal-set-lisp-face-attribute calls
  set_font_frame_param(), at which point :family becomes "nil".


My current interpretation (based on the above) is that
font_list_entities() gets called after the "Font Family" entry has
been updated in the `spec' but before the weight is updated; instead,
the spec probably inherits the weight of the current font for the
default face, as associated with the current frame.  Since that weight
is different from the weight of the font I requested,
font_delete_unmatched() kicks that font out.

*** How should this be fixed?

I don't understand enough of the rationale behind the font selection
logic to propose a fix.  Changing the order in which font spec entries
are updated _might_ fix the problem, but there is some special-case
handling for the default font that I haven't wrapped my head around.

*** Threats to validity

The above is based on some manual tracing, and I may have made
mistakes while logging.  I am also unfamiliar with the Emacs internals
and font selection algorithm.

** Why this didn't happen before

I previously used the "Terminus" bitmap font, which is no longer
supported due to limitations in the GTK-based font renderer (ftcrhb).
Older Emacs versions might still have behaved "more correctly" due to
the presence of the X font renderer (which is no longer included, to
my understanding).

This bug report is unrelated to the question of whether Emacs should
or should not support bitmap fonts.

** My current workaround

My current workaround replicates a pre-existing Windows NT kludge and
allows fonts to be considered "matching" if the differences in font
weight are small:

=====================================================================
--- emacs-29.1+1/src/font.c	2023-07-30 15:32:46.000000000 +0000
+++ emacs-kludge/src/font.c	2023-09-08 21:03:11.584666482 +0000
@@ -2484,6 +2484,8 @@
 		 int candidate = XFIXNUM (AREF (entity, prop)) >> 8;

 		 if (candidate != required
+			&& (prop != FONT_WEIGHT_INDEX
+			   || eabs (candidate - required) > 25)
 #ifdef HAVE_NTGUI
 		     /* A kludge for w32 font search, where listing a
 			family returns only 4 standard weights: regular,
=====================================================================

(The value of 25 should allow both "semi-light" and "medium" fonts to
be used in place of "regular" fonts.)

I don't consider this patch to be a solution to the problem, since I
strongly suspect that it has some undesirable side effects during font
selection (possibly only on specific platforms or for specific fonts /
font weights), but it works well enough for me as a stop-gap fix, and
if this patch also fixes #1029710, I would propose including it in
Debian/sid as a workaround until there is a "proper" fix.

** Other possible workarounds

My reading of `xfns.c' is that Emacs tries to inherit fonts from the
environment, so it may be possible to work around this bug by changing
Gtk defaults.  (This is not something that I have tried, since I am
not very familiar with Gtk.)

Christoph Reichenbach <creichen@gmail.com> writes:
[snip]

For me, it is Fantasque Sans Mono, but I agree it is not selecting the correct
font. The selected font seems non-deterministic, possibly state
dependent. After some various choices of Mono font, I ended up with

Can you duplicate the problem for other fonts?  I tried 4 or 5 other
monospaced fonts and they all seemed to work, at least in a fresh "emacs
-Q".

I observed that choosing some non-existing font name (e.g. FooBar) had
more or less the same effect. So maybe the issue is just that emacs
cannot find "Terminus (TTF)". A wild guess would be the parens in the
name causing the problem.

Apologies if this is covered already in your extensive report.

Dear David,

  my sincere apologies for missing your mail-- fortunately the most
recent update breakage reminded me to check the bug tracker again.

  I had no trouble with the other monospaced fonts that I tried.

  My working assumption at the time of my investigation was that default
font selection searches for fonts with some hardcoded default values and
only applies some user-requested properties after it has already
selected the font.

  Basically, font selection (for the default font only-- other fonts
take a different code path) seems to follow the following steps, as far
as I understood it:

- Set hardcoded default font settings, including:
  - weight = regular
  - slant = ...?  (probably also regular)
- Apply user preferences for:
  - font foundry
  - font family
- Search for a font in the current font foundry that satisfies the
  current font family, weight, and slant requirements
- Apply user preferences for:
  - weight
  - slant
- Return the font

  This is greatly simplified-- the search step returns a list of
possible matches, for instance, and there are several selection
heuristics involved that I don't remember much about.


  If my interpretation is correct, then another workaround could be to
change "Terminus (TTF)" to report a "regular" weight that is identical
to its "medium" weight, but I have not had the time to figure out how to
try this.

  Apologies for the verbosity of my earlier report; had I understood the
problem fully, I would have kept it shorter...

  All the best,

Hello again,

  just to keep the records up to date: the bug is still present in
1:29.3+1-2, and the same patch still works around it (for me).

  All the best,