In Debian bookworm, markdown-toc is currently unusable.

Given the following markdown buffer:

```


# Background


## Why migrate?


## Gitolite and GitWeb inventory


# Proposal
```

markdown-toc-generate-toc crashes with:

Debugger entered--Lisp error: (wrong-type-argument consp nil)
  markdown-imenu-create-nested-index()
  funcall(markdown-imenu-create-nested-index)
  (markdown-toc--compute-toc-structure (funcall imenu-create-index-function))
  (funcall markdown-toc-user-toc-structure-manipulation-fn (markdown-toc--compute-toc-structure (funcall imenu-create-index-function)))
  (markdown-toc--generate-toc (funcall markdown-toc-user-toc-structure-manipulation-fn (markdown-toc--compute-toc-structure (funcall imenu-create-index-function))))
  (insert (markdown-toc--generate-toc (funcall markdown-toc-user-toc-structure-manipulation-fn (markdown-toc--compute-toc-structure (funcall imenu-create-index-function)))))
  (save-excursion (if (markdown-toc--toc-already-present-p) (progn (markdown-toc--delete-toc t))) (insert (markdown-toc--generate-toc (funcall markdown-toc-user-toc-structure-manipulation-fn (markdown-toc--compute-toc-structure (funcall imenu-create-index-function))))))
  markdown-toc-generate-toc(nil)
  funcall-interactively(markdown-toc-generate-toc nil)
  command-execute(markdown-toc-generate-toc record)
  execute-extended-command(nil "markdown-toc-generate-toc" nil)
  funcall-interactively(execute-extended-command nil "markdown-toc-generate-toc" nil)
  command-execute(execute-extended-command)

Antoine Beaupre <anarcat@debian.org> writes:

Hi Antoine;

I agree that markdown file looks innocuous, but do you know if it is
just this file or any markdown file?

d

I managed to have it generate a TOC with a simple:

# a

# b

## c

But it also crashed (ultimately) on:

https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy/tpa-rfc-36-gitolite-gitweb-retirement.md

a.

Control: tag -1 confirmed upstream fixed-upstream pending
Antoine Beaupre <anarcat@debian.org> writes:

Thanks for reporting; fix pending!

Oh interesting! Where can I see it? is there a patch?

David Bremner <david@tethera.net> writes:

This bug appears to be native-comp related.  After cherry picking three
commits from upstream, I was able to generate a toc for the md snippet
in the initial but report.  In the time it took to reply to this bug,
native-comp occurred, and generate-toc began to fail again.  I purged
the eln cache and tried again, and it succeeded...

Antoine Beaupré <anarcat@debian.org> writes:

git clone git@salsa.debian.org:emacsen-team/markdown-toc-el.git
git checkout for-testing-before-upload

Nicholas D Steeves <sten@debian.org> writes:

I also confirmed that both the patched version (in the staging branch)
and unpatched version (in bookworm) work correctly with

https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy/tpa-rfc-36-gitolite-gitweb-retirement.md

when one loads markdown-toc and generates the toc before native comp
kicks in.

Nicholas D Steeves <sten@debian.org> writes:

Sorry for not being clear.  What I mean is that I believe that this bug
is triggered by native compilation, and that it's unlikely that I'll
find enough time figure out what the upstream issue is before the
autoremoval.  Also, upstream hasn't seen any activity since 2021...

Feel free to forward this bug and/or adopt this package (I don't use
it).

Cheers,
Nicholas

You seem to have pasted a link to the TPA GitLab wiki here... Did you
mean to paste some other bug report or link there?

I think I'm okay with the package being removed from bookworm if we
can't find a quick fix here. The release date is just too close. We can
always fix this in a point release or get a backport going.

Interestingly, it's not marked as autoremoval here though:

https://tracker.debian.org/pkg/markdown-toc-el

Alternatively, I wonder if there's a way to make a simpler module that
would defer the TOC generation to an external command...

Is there something out there that takes a markdown doc as input and
outputs a TOC?

Cheers,

a.

Answering my own question, this does what I want:

    md_toc github -o .  < tpa-rfc-36-gitolite-gitweb-retirement.md

... and what I want in many cases is actually:

    md_toc github -o .  < tpa-rfc-36-gitolite-gitweb-retirement.md | sed 's/](#.*//;s/ \[/ /'

which is just an ordered list of items, without markdown links.

md_toc is part of `python3-md-toc`.

Antoine Beaupré <anarcat@debian.org> writes:

That's confirmation that I tested with the link to the file that you
were noted (in an earlier post) that you were testing with.

Interesting, maybe someone has faith that we'll fix this in a point
release!

I saw your note on python3-md-doc, and it made me think about more
generic TOC and endnotes functions that call an external parser.

This one might be a viable alternative, with a slightly more active
upstream than markdown-toc:
https://github.com/snosov1/toc-org#markdown-support

Cheers,
Nicholas

Hi,

Here is a way to work around this bug (whether in Emacs or in markdown-toc).

To test:
emacs --eval="(setq native-comp-deferred-compilation-deny-list '(\"markdown-toc\"))"

To make permanent:
(setq native-comp-deferred-compilation-deny-list '("markdown-toc"))

That said, I'm not convinced that a workaround like this should be
inserted into Debian's markdown-toc (or any package)...

Cheers,
Nicholas

That said it doesn't actually work! I have tried both the `--eval` and
the `setq` and neither fix the crash.

Debugger entered--Lisp error: (wrong-type-argument consp nil)
  markdown-imenu-create-nested-index()
  markdown-toc-generate-toc(nil)
  funcall-interactively(markdown-toc-generate-toc nil)
  command-execute(markdown-toc-generate-toc record)
  execute-extended-command(nil "markdown-toc-generate-toc" nil)
  funcall-interactively(execute-extended-command nil "markdown-toc-generate-toc" nil)
  command-execute(execute-extended-command)

I'm not sure this is related to native compilation, is it really?

I thought it was some imenu thing, maybe it doesn't get autoloaded properly?

a.

Antoine Beaupré <anarcat@debian.org> writes:
[snip]

You're welcome!

When I 'rm -rf ~/.emacs.d/eln-cache, and restart Emacs,
markdown-toc-generate-toc works correctly, but will eventually break
again once various things are compiled again (sooner than later on a
fast machine!).

I'm not sure why the deny list hack works with an empty eln-cache, but
not after a 'rm ~/.emacs.d/eln-cache/*/markdown-toc*'.

Based on your hunch, I tested removing each rdep from the eln-cache, and
I found that the (wrong-type-argument consp nil) bug occurs in
markdown-mode-toc when markdown-mode is native compiled.

markdown-toc-generate-toc succeeds for me when

  ~/.emacs.d/eln-cache/*/markdown-mode-*.eln and
  ~/.emacs.d/eln-cache/*/markdown-toc-*.eln and
  ~/.emacs.d/eln-cache/*/imenu-*.eln

are removed, and emacs is started with this minimal workaround:

  emacs --eval="(setq native-comp-deferred-compilation-deny-list '(\"markdown-mode\"))"

Yes, markdown-mode, no "toc".  For this hack to work long-term for most
users, it seems like imenu would probably need to be added to that
list...but here's the strange thing: Logically,
markdown-toc-generate-toc uses imenu, so it seems like it should trigger
native-comp of imenu at L261.  Neither markdown-mode, nor
markdown-mode-toc explicitly (require 'imenu), so yes, I think you're
right that one or both of them is depending on autoloads.  Markdown-toc
ends up native-compiled using this method, markdown-mode and imenu
don't, and for reasons that aren't yet clear, this allows markdown-toc
to function correctly.

I also wonder if there may be a dash.el+native comp bug in play.

'hope this band-aid does the trick until the root of the problem can be
identified.

Nicholas

I can confirm that trashing the eln-cache and restarting emacs with
`markdown` in the block list works.

This package has found an upstrain maintainer and needs a QA upload
and/or a new Debian maintainer.

https://github.com/ardumont/markdown-toc/commits/master/
    and the orphan bug #1040073

Cheers,
Nicholas

Hi Antoine,

Can you still reproduce this?  I can't.  See attached proof.

I filed an issue upstream to ask whether they will make a new release:

https://github.com/ardumont/markdown-toc/issues/68

A.