Hello,

Our buildbot instances has the web interface enabled:

c['www'] = dict(port=8080)

But buildbot doesn't want to start:

2022-01-24 20:36:49+0100 [-] adding 834 new builders, removing 0
2022-01-24 20:37:17+0100 [-] adding 441 new schedulers, removing 0
2022-01-24 20:37:34+0100 [-] while starting BuildMaster
        Traceback (most recent call last):
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
            result = g.send(result)
          File "/usr/lib/python3/dist-packages/buildbot/util/service.py", line 49, in reconfigServiceWithBuildbotConfig
            yield svc.reconfigServiceWithBuildbotConfig(new_config)
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1613, in unwindGenerator
            return _cancellableInlineCallbacks(gen)
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
            _inlineCallbacks(None, g, status)
        --- <exception caught here> ---
          File "/usr/lib/python3/dist-packages/buildbot/master.py", line 304, in startService
            yield self.reconfigServiceWithBuildbotConfig(self.config)
          File "/usr/lib/python3/dist-packages/buildbot/util/service.py", line 49, in reconfigServiceWithBuildbotConfig
            yield svc.reconfigServiceWithBuildbotConfig(new_config)
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
            result = g.send(result)
          File "/usr/lib/python3/dist-packages/buildbot/www/service.py", line 215, in reconfigServiceWithBuildbotConfig
            self.setupSite(new_config)
          File "/usr/lib/python3/dist-packages/buildbot/www/service.py", line 285, in setupSite
            raise RuntimeError("could not find buildbot-www; is it installed? %s" % (str(info())))

The str(info()) part is an addition on our side to know what plugins
were actually detected, and we have: [...] 'www': {}}

so nothing is detected for www. Indeed, looking at the source code in
/usr/lib/python3/dist-packages/buildbot/plugins/db.py that detects
plugins:

    def _tree(self):
        if self._real_tree is None:
            self._real_tree = _NSNode()
            for entry in iter_entry_points(self._group):

This is using pkg_resources.iter_entry_points to get the list of python
classes etc. but that reports nothing, for instance in a plain chroot:

# apt install buildbot
# python3 -c "import pkg_resources ; print(list(pkg_resources.iter_entry_points('buildbot.www')))"
[]

while /usr/lib/python3/dist-packages/buildbot/www/ is really there. One
thing I noticed is that
/usr/lib/python3/dist-packages/buildbot-2.0.1.post2.egg-info/entry_points.txt
doesn't list anything for buildbot-www.

We had to revert to the manual installation of buildbot, the Debian
package was unusable for us.

Samuel

Hi,

this is a known issue. The buildbot-www package cannot be integrated in
Debian due to tons of missing JS dependencies. See buildbot(7) and bug
#883529 for more details.

Since then, buildbot has changed from coffeescript+gulp to plain
JS+webpack. This should make it easier to package the missing
dependencies and support packaging the www plugins. Still far from
a trivial task unfortunately... I am not even sure how to proceed.
I would really appreciate some help on the matter.

In the meantime, you may install buildbot-www via pip (although this is
not a very nice workaround).

Thanks.

Robin Jarry, le mar. 25 janv. 2022 17:34:19 +0100, a ecrit:

Ah, ok, thanks!

For some reason manpages are not getting found on the box, but I could
read it with the complete path.

Samuel

Thanks for the ongoing maintenance of buildbot!  F-Droid is in the process of
moving to buildbot.  We generally try to maintain all of our systems using
Debian packages, so we might be able to contribute to getting those JS deps in
Debian.  Is there a quick way to find out the list of what's missing?  I'm not a
JS dev at all, but I've been able to package Go and Ruby packages without
knowing those languages, so it might also work here.

Quoting Hans-Christoph Steiner <hans@eds.org>:

See https://wiki.debian.org/Javascript/Nodejs/Tasks/

The tool generates the source code for a wiki page, e.g. like this one:

https://wiki.debian.org/Javascript/Nodejs/Tasks/converse.js

Updates need to be performed manually.

HTH!

Martin, Jun 20, 2022 at 15:49:

I ran the script on buildbot-build-common which is not a real
installable package, only a toolkit used by www plugins (it took more
than 2 hours).

https://paste.sr.ht/~rjarry/2197906bc7f706e726dddfe3071a27b0a82c5f80

I cannot subscribe to the wiki to create a new page: "Automatic account
creation disabled to stop spammers signing up."

In any case, there seem to be a lot of missing packages. This does not
look like a trivial task :)

Cheers,
Robin

Thank you! I created a page with your output:

https://wiki.debian.org/Javascript/Nodejs/Tasks/buildbot-build-common

Here and there, the table looks strange. Maybe the Python script is
a little bit buggy.

I saw worse! Also, probably not all packages are really needed for
packaging, but only for development or for optional featues.