#761867 debsources: conjunctive (AND-ed) ctags search

#761867#5
Date:
2014-09-16 12:28:13 UTC
From:
To:
We should have a new kind of search under /search that allows to search for
files that contain all of a given number of ctags symbols provided by the user.

Note that, differently from the current ctags search that returns indivudual
locations within files, this new search should probably return entire files,
possibly highlighting the lines containing the requested ctags.

#761867#10
Date:
2014-09-16 12:35:04 UTC
From:
To:
As reminder to self (or future wannabe fixer), there is a related TODO
in python/models.py, with a tentative query implementing this search.

#761867#15
Date:
2015-03-12 13:33:03 UTC
From:
To:
Hello,

I have attached the patch for the conjunctive search.
Raw sql commands are the following:

First I find files with the ctags. (there are 2 tags in the following case)

SELECT ctags.file_id AS file_id
FROM ctags JOIN files ON files.id = ctags.file_id
WHERE ctags.tag IN (:tag_1, :tag_2) GROUP BY ctags.file_id
HAVING count(DISTINCT ctags.tag) = 2

then i use the ids i recovered to find the package names etc. (there are
4 ids as the first command had 4 results.)

SELECT files.id AS file_id,
package_names.name AS package,
packages.version AS version, files.path AS path
FROM files, package_names, packages
WHERE files.package_id = packages.id
AND files.id IN (:id_1, :id_2, :id_3, :id_4)
AND packages.name_id = package_names.id

Cheers

#761867#22
Date:
2015-03-13 10:47:53 UTC
From:
To:
[ just documenting here a quick summary of IRC discussions ]

The patch looks good, but there are concerns that this feature will
allow to DoS the DB, given the ctags table currently contains ~380
million tuples.

We've pinged friendly neighborhood Postgres experts (Myon, mnencia) who
are looking into this to give feedback about: whether the feature is a
good idea at all and/or whether there are better ways to implement this
query.

I duly notice that we already have non-conjunctive ctags-based search.
And it's not given us problems thus far. If we can make the conjunctive
query, performance-wise, "as bad as" the non-conjunctive version, then
it's probably good to do.

Cheers.