I’ve had a few reports over the last few months that folks uploaded new extensions to PGXN but they failed to show up in search results. For example, as of right now, if you search for “distinct”, you get two results, OmniPITR and pgTAP. This despite the fact that the recently-released adaptive_estimator extension is tagged “distinct”. Author Tomas Vondra reported this issue, and it took me a couple of days to realize why it wasn’t showing up.

Here’s the reason: Only “stable” releases are indexed. The first and second adaptive_estimator releases both have their release status set to “testing”. The PGXN API only indexes stable releases. The idea behind that is that you want most folks to continue using stable releases while you work on testing new versions. So when users search the site, only the latest stable release will appear in search results. Similarly, installing an extension via the PGXN client, prefers the latest stable release by default. If you want the most recent, you have to specify the --unstable or --testing option.

So, the upshot is, if you want your extension to appear in the full text search results on the PGXN, site, release a stable version.

That said, I first noticed this issue a while ago, and filed an issue with the idea that, if an extension is uploaded that is not stable, but there are no stable versions, then the extension should be indexed, anyway. The idea here is that, when you first upload it, you don’t have any existing users to keep on a stable release anyway, because there is no stable release. So perhaps we should go ahead and index it. A non-stable release would only be omitted from the index if there was an existing stable release.

Thoughts?

So I’m designing the full text indexing for the PGXN search site. I’m modeling it on CPAN Search, which has been great. There are four search options:

• Full documentation search. This is the most common. Includes doc title and body.
• User search. Search on names, nicknames, email addresses, URIs, etc.
• Distribution search. Search on distribution name, abstract, description, tags, and the README.
• Extension search. Search on extension name and abstract.
• Tag search. Search on tag name only.

The documentation search is the one I’m perhaps least sure about. It assumes that each extension in a distribution will have documentation. But so far that has not really been the practice for PostgreSQL extensions. Most folks seem to stick the documentation in the README. And even then it can be almost nothing. So a search for “count nulls” probably would not find “countnulls” extension, because there is no documentation. What should I do about this? I’m thinking one of:

• Encourage folks to write documentation. I’m going to do this anyway, because the docs will really help the visibility of an extension on the site. It looks like this. If you have no docs for an extension, your extension will not appear in the search results (or perhaps it might, but link to the distribution).

• If there is no documentation for an extension in a distribution, index the README as the documentation. I’m not really keen on this idea, because the README should describe the distribution, how to install it, etc. I’m planning to use it in the distribution-specific index. Documentation of the extension should be more about how the extension works, what it’s interface is, etc. Or so it seems to me, at least (I’m admittedly biased to this practice among CPAN modules). But at least with this approach there would be a link to “documentation” for an extension on the search site.

Erm, not really thinking of any other options. I feel pretty strongly that folks should write docs for their extensions, as much as possible, and I’ve set things up so that, from PGXN’s point of view, at least, you can write documentation in whatever format you like (assuming the format is supported by or added to Text::Markup), as long as they’re in a doc/ or docs directory. I want it to be as easy as possible. But I also want there to be decent search results ASAP.

Comments?