Dear Maintainer,
On a user whose AppStream cache for Flathub is missing or older than Flathub's catalog, Discover parses the whole 51 MB appstream.xml inside its own process, and the memory stays in its heap for as long as it runs.
That is every user's first launch and again after every change of the catalog, including the one Discover pulls by itself (it refreshes a remote older than 6 hours at start).
The cache cannot be made for the user by the system: the apt hook /etc/apt/apt.conf.d/50appstream refreshes only `appstreamcli refresh-cache --source=os`, and a user cannot write /var/cache/swcatalog.
Measured with plasma-discover 6.7.4-3 from testing (the package, unpatched) and the same source with the patch below, built in a clean debian:testing container (dpkg-buildpackage, nocheck).
Machine: AMD A10-9600P, 12 GB.
Each launch of plasma-discover in its own nested KWin virtual session with a private HOME, Pss of the process tree sampled for 85 s; "resident" is the last sample, "main anon" is Pss_Anon of the Discover process alone.
6.7.4-3 (Debian) 6.7.4-3 + patch
first launch of a user, no cache 971 MiB main anon 206 MiB main anon
launch after Flathub's catalog changed 1,016 MiB main anon 200 MiB main anon (resident 1,080 against 320 MiB)
launch with the caches current 254 MiB 204 MiB
second remote added, caches current 322 MiB 189 MiB
With the patch the parsing happens in short-lived `appstreamcli refresh-cache` processes (about 500 MiB each, up to three at a time, 7 to 25 s on this machine) that end before Discover loads the compiled cache: the peak of the whole tree is higher for those seconds (1.4 GiB against 1.1 GiB) and what stays resident is a third.
Cause, in the 6.7.4 sources:
- libdiscover/backends/FlatpakBackend/FlatpakBackend.cpp, FlatpakBackend::createPool(): an AppStream::Pool with addExtraDataLocation(<remote>/active) is loaded with loadAsync(); the pool compiles its cache (en-US-<md5 of the path>.xb in ~/.cache/appstream) in Discover's process when the catalog is newer than the cache.
- libdiscover/appstream/OdrsReviewsBackend.cpp, parseRatings(): `AppStream::Pool appstreamData; appstreamData.load();` loads the standard locations, among them the Flatpak catalogs (cache en-US-flatpak-<remote>-<arch>.xb); the system copy is stale after every pull, because only the os source is refreshed by apt.
- libdiscover/backends/PackageKitBackend/PackageKitBackend.cpp, reloadPackageList(): the same for the PackageKit pool.
The cache files an unprivileged `appstreamcli refresh-cache` writes are the very files these pools look for (checked: byte-identical to the ones Discover writes), and it does nothing when they are current (0.02 s).
The patch (also compiled and run against 6.7.4-3):
- AppStreamUtils::compileCache() runs `appstreamcli refresh-cache` with the given arguments and waits for it, with the locale of the pool in LANGUAGE (the cache files are named after the locale);
- the Flatpak backend runs it with `--datapath <remote>/active` in the backend's thread pool before the pool's loadAsync();
- the ratings filter and the PackageKit backend run it without arguments before their pool loads;
- when it cannot run, a warning says why and the pool loads as before;
- when a refresh of a remote's appstream data fails (offline), the log says how old the catalog in use is: "Using the appstream metadata of testrepo last refreshed on 2026-11-03T11:48:11 (8 hours ago)".
Tested the expiry as well: a catalog update published after the cache was built (a local remote) is pulled at the next start, the compiled cache is rebuilt after the pull (new mtime, new content) and the three new entries are found in it; with the remote unreachable the old cache is used and its age is logged.
Not covered: Flatpak remotes of a per-user installation (~/.local/share/flatpak) are standard locations of the user scope, which `appstreamcli refresh-cache` does not compile, so they are parsed in the process as before.
The notifier plugin DiscoverPackageKitNotifier is unchanged.
This may be forwarded to KDE (plasma/discover) if you think it fits there.
The patch and this report were prepared with AI assistance (Claude Sonnet 5.5) for Daniel Ramos, who reviews them before they are sent.
Description: Compile AppStream's catalog caches in a short-lived process, not in Discover
A pool that loads a catalog newer than its cache compiles the cache in the process that loads it,
and the XML stays in that process's heap for as long as Discover runs.
Flathub's catalog is 51 MB of XML: about 1,000 MiB resident in Discover against about 200 MiB when the
pools are filled from compiled caches (measured on an AMD A10-9600P, Debian 6.7.4-3).
It happens for each user's first launch and again after every change of Flathub's catalog, Discover's
own pull included, because Debian's apt hook refreshes the system cache of the operating system only
and a user cannot write the system cache.
appstreamcli refresh-cache writes the very cache files the pools look for and does nothing when they are
current, so run it before the places that load Flatpak catalogs: the per-remote pools of the
Flatpak backend (--datapath), the ratings filter default pool (OdrsReviewsBackend) and the PackageKit
backend pool.
If it cannot run, say so in the log and load as before.
Also say how old the catalog in use is when a refresh fails (offline).
Author: Daniel Campos Ramos <Capitain_Jack@yahoo.com>
Origin: vendor
Forwarded: no
Last-Update: 2026-11-03
Assisted-by: claude-sonnet-5-5 via Claude Code
--- a/libdiscover/appstream/AppStreamUtils.h
+++ b/libdiscover/appstream/AppStreamUtils.h
@@ -40,4 +40,10 @@
componentsByCategoriesTask(AppStream::ConcurrentPool *pool, const std::shared_ptr<Category> &cat, AppStream::Bundle::Kind kind);
DISCOVERCOMMON_EXPORT bool kIconLoaderHasIcon(const QString &name);
+
+/**
+ * Compiles AppStream's cache with appstreamcli refresh-cache and the given arguments, waiting for it.
+ * A pool that loads data newer than its cache compiles the cache in the loading process, and the XML stays in its heap.
+ */
+DISCOVERCOMMON_EXPORT bool compileCache(const QStringList &arguments, const QString &locale = {});
}
--- a/libdiscover/appstream/AppStreamUtils.cpp
+++ b/libdiscover/appstream/AppStreamUtils.cpp
@@ -6,6 +6,7 @@
#include "AppStreamUtils.h"
+#include "libdiscover_debug.h"
#include "utils.h"
#include <AppStreamQt/category.h>
#include <AppStreamQt/component.h>
@@ -22,6 +23,8 @@
#include <QJsonArray>
#include <QJsonObject>
#include <QMetaEnum>
+#include <QProcess>
+#include <QStandardPaths>
#include <QUrlQuery>
#include <QtConcurrentRun>
@@ -235,3 +238,28 @@
}();
return icons.contains(name);
}
+
+bool AppStreamUtils::compileCache(const QStringList &arguments, const QString &locale)
+{
+ const QString appstreamcli = QStandardPaths::findExecutable(QStringLiteral("appstreamcli"));
+ if (appstreamcli.isEmpty()) {
+ qCWarning(LIBDISCOVER_LOG) << "appstreamcli not found, AppStream will compile its cache in this process";
+ return false;
+ }
+
+ QProcess process;
+ process.setProcessChannelMode(QProcess::MergedChannels);
+ if (!locale.isEmpty()) {
+ // the cache files are named after the locale, so the process must use the one of the pool
+ auto environment = QProcessEnvironment::systemEnvironment();
+ environment.insert(QStringLiteral("LANGUAGE"), locale);
+ process.setProcessEnvironment(environment);
+ }
+ process.start(appstreamcli, QStringList{QStringLiteral("refresh-cache")} + arguments);
+ if (!process.waitForFinished(-1) || process.exitStatus() != QProcess::NormalExit || process.exitCode() != 0) {
+ qCWarning(LIBDISCOVER_LOG) << "Could not compile the AppStream cache, AppStream will compile it in this process:" << process.errorString() << process.exitCode()
+ << process.readAll().trimmed();
+ return false;
+ }
+ return true;
+}
--- a/libdiscover/appstream/OdrsReviewsBackend.cpp
+++ b/libdiscover/appstream/OdrsReviewsBackend.cpp
@@ -7,6 +7,7 @@
#include "OdrsReviewsBackend.h"
#include "AppStreamIntegration.h"
+#include "AppStreamUtils.h"
#include "CachedNetworkAccessManager.h"
#include "OdrsReviewsJob.h"
@@ -306,6 +307,7 @@
}
// Filter out non-apps, to match behavior of lists backed by ResourcesProxyModel
+ AppStreamUtils::compileCache({});
AppStream::Pool appstreamData;
appstreamData.load();
--- a/libdiscover/backends/FlatpakBackend/FlatpakBackend.cpp
+++ b/libdiscover/backends/FlatpakBackend/FlatpakBackend.cpp
@@ -1176,7 +1176,15 @@
},
Qt::QueuedConnection);
});
- pool->loadAsync();
+ // loading compiles the catalog's cache when it is newer than the cache, and the XML would stay in our heap
+ auto watcher = new QFutureWatcher<bool>(this);
+ connect(watcher, &QFutureWatcher<bool>::finished, this, [watcher, pool] {
+ watcher->deleteLater();
+ pool->loadAsync();
+ });
+ watcher->setFuture(QtConcurrent::run(&m_threadPool, [appstreamDirPath, locale = pool->locale()] {
+ return AppStreamUtils::compileCache({u"--datapath"_s, appstreamDirPath}, locale);
+ }));
}
QSharedPointer<FlatpakSource> FlatpakBackend::integrateRemote(GLibHolder<FlatpakInstallation> flatpakInstallation, GLibHolder<FlatpakRemote> remote)
--- a/libdiscover/backends/FlatpakBackend/FlatpakRefreshAppstreamMetadataJob.cpp
+++ b/libdiscover/backends/FlatpakBackend/FlatpakRefreshAppstreamMetadataJob.cpp
@@ -8,6 +8,12 @@
#include "FlatpakRefreshAppstreamMetadataJob.h"
#include "libdiscover_backend_flatpak_debug.h"
+#include <KFormat>
+
+#include <QDateTime>
+#include <QFile>
+#include <QFileInfo>
+
FlatpakRefreshAppstreamMetadataJob::FlatpakRefreshAppstreamMetadataJob(FlatpakInstallation *installation, FlatpakRemote *remote)
: QThread()
, m_cancellable(g_cancellable_new())
@@ -52,6 +58,15 @@
const QString error = localError ? QString::fromUtf8(localError->message) : QStringLiteral("<no error>");
qCWarning(LIBDISCOVER_BACKEND_FLATPAK_LOG).nospace()
<< "Failed to refresh appstream metadata for " << flatpak_remote_get_name(m_remote.get()) << ": " << error;
+
+ g_autoptr(GFile) timestamp = flatpak_remote_get_appstream_timestamp(m_remote.get(), flatpak_get_default_arch());
+ g_autofree char *timestampPath = g_file_get_path(timestamp);
+ const QFileInfo timestampInfo(QFile::decodeName(timestampPath));
+ if (timestampInfo.exists()) {
+ qCWarning(LIBDISCOVER_BACKEND_FLATPAK_LOG).nospace() << "Using the appstream metadata of " << flatpak_remote_get_name(m_remote.get())
+ << " last refreshed on " << timestampInfo.lastModified().toString(Qt::ISODate) << " ("
+ << KFormat().formatSpelloutDuration(timestampInfo.lastModified().msecsTo(QDateTime::currentDateTime())) << " ago)";
+ }
}
m_hasChanged = changed;
Q_EMIT jobRefreshAppstreamMetadataFinished(m_installation, m_remote, changed);
--- a/libdiscover/backends/PackageKitBackend/PackageKitBackend.cpp
+++ b/libdiscover/backends/PackageKitBackend/PackageKitBackend.cpp
@@ -386,7 +386,15 @@
},
Qt::QueuedConnection);
});
- m_appdata->loadAsync();
+ // loading compiles the catalogs' caches when they are newer than the caches, and the XML would stay in our heap
+ auto watcher = new QFutureWatcher<bool>(this);
+ connect(watcher, &QFutureWatcher<bool>::finished, this, [this, watcher] {
+ watcher->deleteLater();
+ m_appdata->loadAsync();
+ });
+ watcher->setFuture(QtConcurrent::run(&m_threadPool, [] {
+ return AppStreamUtils::compileCache({});
+ }));
}
AppPackageKitResource *PackageKitBackend::addComponent(const AppStream::Component &component) const