Posts on ⚙️ PGXN Blog

Inter-documentation and image links now work on PGXN

Tue, 08 Sep 2026 22:12:29 +0000

Way back in 2015, I opened a PGXN API issue to allow links between documents rendered by PGXN to work. In 2024 I followed up with another issue to enable relative links to images to work. I mean, everyone wants this, right? It’s how your favorite source code sit works.

Now so does PGXN. As of today, links between documents and to images work. Check it out in the newly-released chdb extension docs, which both link to the chdb and chdb_hook docs and to some nice benchmark graphs.

Improved Markdown Parsing

Mon, 15 Dec 2025 15:55:59 +0000

Quick announcement to say that I’ve replaced the ancient markdown parser with a new one, discount, which supports tables, code fences, definition lists, and more. I reindexed pg_clickhouse this morning and it’s sooo nice to see the table properly formatted.

New uploads will use this parser for Markdown (but not MultiMarkdown) files from now on. I think I’ll start working on reindexing all existing extensions, too. Give me a holler if you don’t see an improvement in your extensions in the next few days.

New PGXN Mastodon Account

Sun, 01 Dec 2024 16:03:09 +0000

Sadly, the home of the PGXN Mastodon bot for the last two years, botsin.space is shutting down. I’ve created a new account, @pgxn@mastodon.social and moved all the followers. Please give it a follow if you didn’t follow the old account, and stay up-to-date on the latest PGXN releases!

Feedback Wanted: Meta Spec Sketch

Thu, 21 Mar 2024 00:00:00 +0000

New post up on Just a Theory, RFC: PGXN Metadata Sketch, seeking feedback on a slew of ideas to update the PGXN Meta Spec. Help us to help make the future of PGXN and the broader extension ecosystem better!

Recent PGXN Improvements

Thu, 22 Feb 2024 00:00:00 +0000

One of the perks of my new gig at Tembo is that I have more time to work on PGXN. In the last ten years I’ve had very little time to give, so things have stagnated. The API, for example, hasn’t seen a meaningful update since 2016!

But that’s all changed now, and every bit of the PGXN architecture has experienced a fair bit of TLC in the last few weeks. A quick review.

Presentation: Introduction to the PGXN

Fri, 02 Feb 2024 15:32:53 +0000

The Tembo blog has posted a presentation on the PGXN architecture. There hasn’t been much coverage of it since a few posts here on the PGXN blog back in 2012, so worth a revisit — and a fair bit changed in the interim, as well!

PGXN Tools Docker Image Updated

Wed, 31 Jan 2024 16:52:19 +0000

Just a quick note to highlight some bug fixes and improvements in the pgxn/pgxn-tools Docker image in the last few weeks.

v1.4.0 adds the GIT_BUNDLE_OPTS and ZIP_BUNDLE_OPTS environment variables. The former is passed to git archive and the latter to zip, and let additional options be passed to those commands. For example, the vectorize extension’s release workflow sets GIT_BUNDLE_OPTS: --add-file META.json because git archive archives only checked-in files, and META.json is not checked in but generated from META.json.in.

Hello Mastodon 🐘

Sat, 18 Feb 2023 22:53:46 +0000

Hey all you Postgres people out there! Just wanted to make a quick post regarding the PGXN twitter bot. In light of the recent announcements by Twitter to charge for API use, I updated PGXN Manager to also post to Mastodon! If you’re have joined the exodus to the Fediverse, give it a follow:

@pgxn@mastodon.social

In fact, I made some time to rewrite the notification bits of the manager service. Previously there was just some code jammed into the upload controller that would make an API call to Twitter. I ripped that out, and added a trigger to the distributions table that posts a a LISTEN/NOTIFY message.

Password Storage Update

Mon, 05 Jul 2021 23:12:12 +0000

Just a quick note to say that PGXN Manager has been updated with more secure password storage. Existing passwords are unmodified, but the next time you change your password, it will be upgraded to a more robust password hashing algorithm that’s more resistant to attacks. I recommend everyone update their passwords. If you have a PGXN Manager account, just hit password reset, enter your username or email address, and check your mail for a reset link. Once you update your password, it will be stored in the new, more secure format.

A Few Belated PGXN Updates

Sat, 15 May 2021 03:16:44 +0000

The last couple weeks I’ve returned to PGXN and made a few updates. Nothing huge, but all long overdue.

First up, PGXN Manager is all TSL now. No public HTTP-only site. What started as a convenient division of labor (http for the public site and https for the authenticated site) turned out to be quite irksome — especially since some browsers wouldn’t load the non-TLS site at all anymore. So I did away with it, and now it’s all TLS, authenticated and not. (The API and search sites have been TLS for a while now.)

PGXN Manager Upgraded

Mon, 31 Aug 2015 21:48:54 +0000

I took a little time this summer to finally address some nagging issues in PGXN Manager, the site to which extensions are uploaded and added to the master repository. Most of the changes were internal, improving the interface through which I sometimes have to re-index a release. There are also a couple of minor changes to the sample Makefile in the How To. But the most important change for new releases going forward is that version ordering is now enforced. That means two things:

PGXN Release Badges

Sat, 11 Apr 2015 04:34:08 +0000

Looks like it’s been close to two years since my last post on the PGXN blog. Apologies for that. I’ve thought for a while maybe I should organize an “extension of the week” series or something. Would there be interest in such a thing?

Meanwhile, I’m finally getting back to posting to report on a fun thing you can now do with your PGXN distributions. Thanks to the Version Badge service from the nice folks at Gemfury, you can badge your distributions! Badges look like this:

How To, Example Makefile Updated

Fri, 21 Jun 2013 00:28:13 +0000

I deployed a new version of Manager last week, and it included a major update to the How To. A lot of the changes are narrative: I tried to keep things shorter and more to-the-point. But from a technical point of view, perhaps the most important improvements are in the Makefile example. Thanks to ongoing work by Cédric Villemain, it now tries harder to stay out of your way, while integrating better with PostgreSQL’s own make targets.

PGXN Distribution Prerelease Versions Fixed

Wed, 19 Jun 2013 19:24:10 +0000

A couple years ago, the Semantic Version specification was updated to require a hyphen between the “patch version” and the “prerelease version.” This despite the fact that v1.0.0 of the spec had been published on the site for a while (see all the gory issues here). Naturally, I had already implemented that format for PGXN in a Perl module and Postgres data type. Since we already had some PGXN extensions with the non-hyphenated format, and I knew supporting the hyphen would break them, I held off updating those implementations for a year or so.

Now updating the API server every five minutes ⓣ

Wed, 02 May 2012 16:53:50 +0000

Now updating the API server every five minutes rather than hourly. New uploads will appear on pgxn.org more quickly.

No site feed for uploads, but I just realized that ⓣ

Wed, 02 May 2012 16:24:43 +0000

No site feed for uploads, but I just realized that the Twitter feed provides exactly that. https://twitter.com/statuses/user_timeline/pgxn.atom

pgxn-utils 0.1.4 is out!

Sun, 15 Apr 2012 21:44:12 +0000

by Dickson S. Guedes

Hi everybody!

I’m proud to tell you that a new version of PGXN Utils was released with this new features:

Git support
Templates and custom templates
PGXN Client integration

Git support

You can start a new extension with or without version control. By default pgxn-utils supports git but it will not create a repository unless you use --git option in the skeleton task.

You can try:

$ pgxn-utils skeleton my_cool_versioned_extension --git

When you create a new extension with git support in addition to creating the skeleton, pgxn-utils will initialize a git repository and create the initial commit.

Lose USE_PGXS in your Makefiles

Wed, 11 Apr 2012 16:36:00 +0000

Traditionally, folks have created extensions for PostgreSQL by copying one of the contrib modules and hacking it into something new. One of the things that comes along for the ride is the Makefile (example). As a result, there are a lot of third-party extensions that use the USE_PGXS variable.

A bit of background. The core contrib extensions generally rely on a relative path to include the core Makefiles needed to build the extension. Because they ship with the core distribution, they can generally expect that the core has already been compiled, the necessary Makefiles have been created, and that they should be built against them. All the assumptions are that the extensions should be built against the source tree in which they are distributed. So there is no need to use pg_config to find PGXS; it already knows where to find what it needs.

Only Stable Releases are Indexed

Fri, 06 Apr 2012 17:02:19 +0000

I’ve had a few reports over the last few months that folks uploaded new extensions to PGXN but they failed to show up in search results. For example, as of right now, if you search for “distinct”, you get two results, OmniPITR and pgTAP. This despite the fact that the recently-released adaptive_estimator extension is tagged “distinct”. Author Tomas Vondra reported this issue, and it took me a couple of days to realize why it wasn’t showing up.

New version of api.pgxn.org released, with a new ⓣ

Tue, 14 Feb 2012 06:55:36 +0000

New version of api.pgxn.org released, with a new index generated. Better parsing of reST docs thanks to @dvarrazzo.

Okay, I think the API sync issue is fixed ⓣ

Fri, 13 Jan 2012 05:43:42 +0000

Okay, I think the API sync issue is fixed; omnipitr v0.3.1 now shows up on the site. https://www.pgxn.org/dist/omnipitr/ #phew

here seem to be some issues with the API syncing ⓣ

Thu, 12 Jan 2012 18:20:14 +0000

here seem to be some issues with the API syncing; will get them fixed today. Sorry for the lack up site updates in the meantime.

PGXN Has a New Home

Thu, 12 Jan 2012 04:51:04 +0000

Day before yesterday, I finally got all of PGXN moved to a new server. I had been using a small server owned by my company, Kineticode, and hosted by Command Prompt. That was fine for a while, but CMD was needing its rack space back, and what with my new job, I was shutting down Kineticode, too. It was time to move PGXN elsewhere.

For a while, I got a lot of support and assistance towards moving PGXN to a PostgreSQL community server. Dave, Magnus, and Stefan kindly spun up a VM for me, and gave me permission to install Perl modules from CPAN, provided I supply them with a script to report to Nagios when Perl modules were out of date, which of course I did. This was necessary because I built PGXN with some pretty recent versions of CPAN modules that are not yet available in Debian stable. I was looking forward to getting things running and integrating with the community authentication service.

Many thanks to depesz for the new PGXN hosting ⓣ

Tue, 10 Jan 2012 05:24:04 +0000

Many thanks to depesz for the new PGXN hosting! https://depesz.com/

PGXN Manager will be back up as soon as your DNS ⓣ

Tue, 10 Jan 2012 05:03:32 +0000

PGXN Manager will be back up as soon as your DNS records update. All the PGXN sites are on their new server.

manager.pgxn.org will be doing down for the move ⓣ

Tue, 10 Jan 2012 03:43:44 +0000

manager.pgxn.org will be doing down for the move to the new server in 10 minutes. #WishMeLuck

PGXN Manager will be going down for the move to ⓣ

Mon, 09 Jan 2012 19:46:27 +0000

PGXN Manager will be going down for the move to the new server this evening. Stay tuned for details!

Anyone interested in building RPMs and/or Debian ⓣ

Mon, 12 Dec 2011 18:29:46 +0000

Anyone interested in building RPMs and/or Debian packages for the various parts of the PGXN infrastruture?

New server fell through, unfortunately ⓣ

Mon, 12 Dec 2011 17:27:10 +0000

New server fell through, unfortunately. Will be looking for a new home for PGXN this week.

PGXN Client 1.0 Released!

Mon, 28 Nov 2011 00:34:34 +0000

Finally, here it is. Well tested, documented, and pampered. With the PGXN Client installing extensions from the PGXN Network is a breeze:

$ pgxn install semver
$ pgxn load semver
$ psql
=# select 'foo'::semver;
ERROR: bad semver value 'foo': expected number at foo
LINE 1: select 'foo'::semver;

Error! Meaning: success!

The number of extensions on PGXN is steadily increasing, so we hope the client will make their adoption even easier.

The client is now extensible: either writing in Python to reuse some of the other commands implementation or writing new self-contained scripts in any language to be invoked by the pgxn commands dispatcher. The first client extensions are already available: PGXN Utils and PGXN:Meta:Validator are targeted for easier development of new extensions.

Thank you Jon Erdman for your donation ⓣ

Thu, 22 Sep 2011 05:18:57 +0000

Thank you Jon Erdman for your donation! Your t-shirt is in the mail. #pgxn #fundraising

Postgres Open

Tue, 13 Sep 2011 16:40:24 +0000

I’m off to Chicago today for Postgres Open, a new PostgreSQL conference. I’m pleased that I’ll be presenting “Get Your Preferred Feature Developed!”

There are lots of developers out there who, like me, have ideas for projects they want to work on for PostgreSQL, but don’t have the free time to make it happen. The idea for this talk is to pitch this fact to an audience of organizations with a major investment in PostgreSQL and an interest in seeing it improve. Perhaps one or more of them will look into sponsoring development of something that interests them, or that they need. This way, interested developers might get paid to work on projects that interest them, to the benefit of the project, the community, and of course their sponsors. Naturally, I’ll be drawing on PGXN as an example of how this sort of thing can work.

PGXN Utils 0.1.3 Released!

Thu, 08 Sep 2011 06:39:00 +0000

Hello everyone!

I’m proud to tell you that a new version of pgxn_utils was released!

In this version some errors with OpenSSL was fixed (thanks @theory to report then), and now you can release an extension to PGXN in five steps even using Ruby 1.8!

Another change was the executable’s name that changed from pgxn_utils to pgxn-utils for a close integration with next version of PGXN Client, but some work need to be done, yet.

My presentation at the 2012 PDXPUG PGDay

Wed, 10 Aug 2011 18:58:09 +0000

My presentation at the 2012 PDXPUG PGDay. It covers the basics of how to create useful extensions to PostgreSQL and distribute them on PGXN---without needing to learn C.

PGXN Utils 0.1.2 Released!

Fri, 24 Jun 2011 23:53:00 +0000

Hello everyone!

I’m proud to tell you that a new version of pgxn_utils was released!

Now you can release a distribution to PGXN in five steps!

First, install it:

gem install pgxn_utils

Second, create your extension, optionally overwrite some defaults:

mkdir $HOME/extensions
cd $HOME/extensions
pgxn_utils skeleton my_extension --maintainer "Dickson S. Guedes"

Third, code!

Fourth, bundle it:

pgxn_utils bundle my_extension
Extension generated at: /home/guedes/extensions/my_extension-0.0.1.zip

Fifth, release it:

pgxn_utils release my_extension-0.0.1.zip
Enter your PGXN username: guedes
Enter your PGXN password: ******
Trying to release my_cool_extension-0.0.1.zip ... released successfully!
Visit: https://manager.pgxn.org/distributions/my_cool_extension/0.0.1

Ah, you can export PGXN_USER and PGXN_PASSWORD if you are tired to type your username and password every time.

Aaaand manager.pgxn.org is back and better than ⓣ

Wed, 01 Jun 2011 04:24:36 +0000

Aaaand manager.pgxn.org is back and better than ever!

manager.pgxn.org will be going down for an upgrade ⓣ

Wed, 01 Jun 2011 03:47:48 +0000

manager.pgxn.org will be going down for an upgrade shortly. Will holler when it’s back.

Validate your META.json ⓣ

Wed, 25 May 2011 17:46:24 +0000

Validate your META.json: cpan PGXN::Meta::Validator; validate_pgxn_meta META.json META.json is OK

PGCon slides ⓣ

Tue, 24 May 2011 21:44:31 +0000

PGCon slides: https://www.pgcon.org/2011/schedule/events/291.en.html

PGXN Utils 0.1.1 Released!

Mon, 23 May 2011 04:53:30 +0000

Hello everyone!

This was a productive weekend that allowed me to work on some new features in pgxn_utils and I’m proud to tell you that a new version was released!

Trying to simplify your extension-development life I’ve added two tasks to pgxn_utils: change and bundle. The first one is just a convenient way to change META information about extension, incrementally, the second one is an easy way to archive your extension in a zip file well named.

The PGCon PGXN talk starts in 17 minutes ⓣ

Fri, 20 May 2011 18:42:52 +0000

The PGCon PGXN talk starts in 17 minutes. https://www.pgcon.org/2011/schedule/events/291.en.html

Sorting @pgxn shirts ⓣ

Thu, 19 May 2011 00:53:19 +0000

Sorting @pgxn shirts. #StillSomeExtras

Release an extension on PGXN this week and get an ⓣ

Wed, 18 May 2011 20:36:54 +0000

Release an extension on PGXN this week and get an invite to the launch party Friday night at #pgcon.

PGXN Utils

Sat, 14 May 2011 01:05:00 +0000

Do you ever have problems with copy and paste? I often did, and that is why I create custom templates for often used files that match certain patterns.

With files from the structure of PostgreSQL’s extensions was the same thing.

I was tired of creating the files and edit the META, controlfile, READMEs, etc. every time I start a new extension and felt that I need something that made me more productive, so I decided to create an automatic generator and share it with the world.

New HOWTO

Fri, 13 May 2011 20:34:46 +0000

I updated the howto yesterday. This document explains how to create a PGXN distribution. If you’re interested in releasing PostgreSQL extensions on PGXN, this document is worth a read.

In essence, it’s really simple: Just create a META.json and upload. But to get the full benefit, there are quite a few other recommendations. Already familiar with it? Here’s the checklist:

Create a META.json
Create a control file
Create a Makefile
Implement the code in the sql and src directories
Write tests in the test directory
Write a Text::Markup-recognizable README
Write Text::Markup-recognizable documentation in the doc directory
Consider including other files: Changes, LICENSE, INSTALL, COPYING, AUTHORS
Release it!

Be sure to read the howto for details. Got feedback or suggestions? Leave a comment!

Using JSONP Callbacks

Fri, 13 May 2011 03:58:33 +0000

Using the new JSONP callback support in the PGXN API, I just added a sidebar to my personal blog that shows a list of all my PGXN distributions. The code is simple:

<div class="links">
 <h2>PGXN Code</h2>
 <script type="text/javascript">
 function pgxn_distros(data) {
 document.write('<dl>');
 for (dist in data.releases) {
 document.write(
 '<dt><a href=https://pgxn.org/dist/' + dist +
 '>' + dist + '</a></dt>' +
 '<dd>' + data.releases[dist].abstract + '</dd>'
 );
 }
 document.write('</dl>');
 }
 </script>
 <script type="text/javascript"
 src="https://api.pgxn.org/user/theory.json?callback=pgxn_distros">
 </script>
</div>

And the output looks like this:

Not bad, eh? AS you can see, JSONP is dead easy to use with the JSON files served by the JSON API. Try that, CPAN!

sudo gem install pgxn_utils ⓣ

Thu, 12 May 2011 23:49:40 +0000

sudo gem install pgxn_utils https://github.com/guedes/pgxn-utils #Awesome

Know Perl? Interested in PGXN? @-message me for an ⓣ

Wed, 11 May 2011 17:07:41 +0000

Know Perl? Interested in PGXN? @-message me for an offer you cannot refuse.

Want to create a PGXN distribution? Dickson Guedes ⓣ

Wed, 11 May 2011 16:22:18 +0000

Want to create a PGXN distribution? Dickson Guedes shows you how. https://pgcasts.com/media/pgxn_utils-usage-example.mpeg #SoAwesome

Distribution Name Constraint

Wed, 04 May 2011 20:42:37 +0000

Following a pgxn-users discussion, I’m looking at adding a constraint on the names distributions can have. I’m thinking this:

CREATE DOMAIN namespace AS CITEXT
CHECK ( VALUE ~ '^[-a-z0-9_]{2,}$' );

So, just ASCII numbers and letters, dash, and underscore, with a minimum length of two characters. Maybe , and + should be allowed, too? What do you think? What constraints do you expect on distribution download file names?

Hrm ⓣ

Tue, 03 May 2011 03:27:44 +0000

Hrm. I wonder if the JSONP callback query param should be called “jsonp” or “callback”?

Updating the API server, might be some instability ⓣ

Tue, 03 May 2011 00:09:51 +0000

Updating the API server, might be some instability over the next few minutes.

New release for the PGXN client

Mon, 02 May 2011 00:59:37 +0000

During the last days I’ve done some lightweight hacking on the PGXN client, and I’ve just released the last package on PyPI, with the still very shy version number of 0.1a4.

First change: the package name. The package is now called pgxnclient, without the point. As Marti has commented, the point in the name could create problems to some distributions which may have needed to munge it into a shape fitting their naming constraints. So, in order to install the last release you will need the command:

PGXN is on CPAN ⓣ

Sat, 30 Apr 2011 00:48:25 +0000

PGXN is on CPAN! https://search.cpan.org/search?query=PGXN

Release most of the toolchain to CPAN ⓣ

Fri, 29 Apr 2011 21:16:39 +0000

Release most of the toolchain to CPAN.

About the Infrastructure: PGXN API

Fri, 29 Apr 2011 20:44:02 +0000

The second piece of the PGXN infrastructure, after PGXN Maanager, is the PGXN API Server. I’ve just finished the API documentation, which covers both the lightweight static file API provided by mirrors and the superset provided by the API server. So now seems like a good time to talk about the design of the API server and how it works.

At its core, the PGXN API server is just another mirror. It has an hourly cron job that rsyncs to the master mirror, updating the mirror. But then it iterates over the rsync log and transforms some things. Here’s what it does:

Writing a client for PGXN

Fri, 29 Apr 2011 00:15:00 +0000

PostgreSQL 9.1 is quickly rolling to the beta phase, and the release will bring a new useful feature: extensions. Well, actually extensions have always existed, but they didn’t have an identity before: what you used to get was a set of loose objects and mysterious functions in your namespace, typically getting in the way in your dump and restore. Now here it is: CREATE EXTENSION, in all its uppercase glory!

So, an extension will exist in your database, and this finally gives the possibility to organize them, this is what PGXN is for: a repository for metadata, documentation and code, neatly packaged and asking to be picked off the shelf. I’ve recently written an extension myself so I got some original material to play with PGXN, which has resulted in quite a pleasant experience: the infrastructure is well done thanks to David’s design, and the API is ridiculously easy to interact with: I’ve literally explored it using curl.

The PGXN API documentation is complete ⓣ

Thu, 28 Apr 2011 19:19:33 +0000

The PGXN API documentation is complete. https://github.com/pgxn/pgxn-api/wiki Have a look!

Just deployed new versions ⓣ

Wed, 27 Apr 2011 19:04:25 +0000

Just deployed new versions. Download files are now .zip instead of .pgz. https://pgxn.org/

PGXN API Docs Published

Wed, 27 Apr 2011 00:27:55 +0000

The PGXN API Documentation is up! I’ve just finished writing the docs for the lightweight REST API provided by all PGXN mirrors. It also documents how the same APIs differ when provided by the API server. Their are four more documents to write still, APIs provided by the API server but not the mirrors (including full-text search); I should be able to get those done tomorrow.

So if you’re interested in use the API for various things, please have a look! I’d appreciate any feedback or corrections.

Case Insensitivity

Tue, 26 Apr 2011 04:09:08 +0000

I pushed out new versions of PGXN::Manager, PGXN::API, and PGXN::Site today, all with the aim of resolving some issues with case-sensitivity. I was alerted to this when Daniele Varrzzo released italian_fts with the Italian tag and the API sync blew up. Oops.

So here’s the deal. The following objects have case-insensitive names on PGXN:

Distributions
Extensions
Versions
Users (nickname)
Tags

These names are case-preserving, so they should show up with the proper capitalization in JSON files and whatnot. But to avoid issues with URLs, I decided that all file names should be lowercase only. Fortunately, nearly everything was lowercase already: only a few nickname JSON files were not lowercased, plus the files for pgTAP. So once I got the code updated, I renamed those files appropriately. In the case of the pgTAP download, I actually re-indexed it, so that the prefix in the zip file would properly be “pgtap-0.25.0” instead of “pgTAP-0.25.0”, since the file name is now pgtap-0.25.0.pgz.

New site deployed with case-sensitivity issues ⓣ

Mon, 25 Apr 2011 18:28:10 +0000

New site deployed with case-sensitivity issues fixed. https://pgxn.org/

Would appreciate feedback on the format of the API ⓣ

Sun, 24 Apr 2011 05:28:55 +0000

Would appreciate feedback on the format of the API docs. So far only “index” and “download” are documented. https://github.com/pgxn/pgxn-api/wiki

About the Infrastructure: PGXN Manager

Sat, 23 Apr 2011 02:54:34 +0000

The PGXN infrastructure is currently made up of four parts:

The network of mirrors, derived from the master mirror, and synchronized on various schedules via rsync (host a mirror).
PGXN Manager (code) is the core of the network. It provides the interface for users to upload releases, processes those releases, indexes them, and puts them on the master mirror. Details below.
PGXN API (code) is a mirror server with benefits. Once an hour, it rsyncs from the master mirror, and does extra processing of new and modified files, notably full-text indexing. I’ll write up some details next week.
PGXN Site (code) powers the main site. It’s a thin wrapper around the API server, using WWW::PGXN to fetch JSON files and convert them to HTML. I’ll write more about this bit next week, too.

Working on the donor shirt design

Thu, 21 Apr 2011 17:10:58 +0000

Working on the donor shirt design.

What Media Types for Browsing?

Thu, 21 Apr 2011 16:56:43 +0000

Thanks to a comment from “Anon,” I’m auditing the media types used to serve files from the “Browse” links on the site. The browse interface is managed by Plack::App::Directory, which gets its media type mapping from this file. What PGXN::API does is simply change the mappings of some of those types to text/plain. As of this commit, files are served as plain text if:

The strings “html”, “x-c”, “xml”, “calendar”, or “vcard” appear in the default media type; or
The file name extension is one of .bat, .css, .eml, .js, .json, .mime, or .swf.

Are there other media types that should be disabled for safe browsing of user-submitted content?

Extension Makefiles for PostgreSQL 9.1

Wed, 20 Apr 2011 19:29:00 +0000

In order to keep distribution packaging as simple as possible, I worked up this Makefile some time ago:

DATA = $(wildcard sql/*.sql)
DOCS = $(wildcard doc/*.txt)
TESTS = $(wildcard test/sql/*.sql)
REGRESS = $(patsubst test/sql/%.sql,%,$(TESTS))
REGRESS_OPTS = --inputdir=test --load-language=plpgsql
MODULES = $(patsubst %.c,%,$(wildcard src/*.c))

PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)

The nice thing about this code is that it has nothing specific to a distribution in it. It figures out what SQL files there are, what doc files there are, and what C files need compiling by just looking in the sql, doc, and src directories, respectively. It also specifies that tests are in the test directory. About the only thing I’ve customized here is adding --load-language=plpgsql to REGRESS_OPTS, as the tests for the distribution I’ve copied this from require PL/pgSQL to run. Simple, and anyone can use it with very little need to tweak it, as long as they don’t mind storing their files in the specified directories.

Building PostgreSQL HEAD in order to update my ⓣ

Wed, 20 Apr 2011 16:09:47 +0000

Building PostgreSQL HEAD in order to update my extensions for the 9.1 extensions support. Will update the HOWTO with details soon.

Deployment done, sites are back up ⓣ

Wed, 20 Apr 2011 05:45:45 +0000

Deployment done, sites are back up. Blog post tomorrow.

Deployment done, sites are back up ⓣ

Wed, 20 Apr 2011 05:45:45 +0000

Deployment done, sites are back up. Blog post tomorrow.

Pushing out new changes to the sites; might be ⓣ

Wed, 20 Apr 2011 05:36:51 +0000

Pushing out new changes to the sites; might be unstable for a few minutes.

Next Thursday: PDXPUG: Distributing Extensions on ⓣ

Fri, 15 Apr 2011 20:14:46 +0000

Next Thursday: PDXPUG: Distributing Extensions on PGXN https://calagator.org/events/1250460492

It's Finally Up: The New PGXN Site!

Thu, 14 Apr 2011 06:32:06 +0000

I’m pleased to announce that the new PGXN SITE went live last night. Some of the things it does:

Full text search of all documentation, distribution metadata and READMEs, extensions metadata, and users
list of the 56 most recent uploads
Find users by first letter of nickname (mostly so search spiders can drill down to all site content)
A tag cloud with the 56 most commonly-used tags (they’re all the same size and color right now because each one is used only once at the moment)
Pages for users (example)
Pages for distributions, with inlined README (example)
Browse unzipped distributions (example)
Pages for documentation (example)
Permalinks for extensions example)

The entire site is backed by the API Server, which is mainly composed of static JSON and HTML files read by the site back end from the local file system. So the site is quite fast. Please browse around, let me know what you think, and please report any bugs you find.

And thew new PGXN is live ⓣ

Wed, 13 Apr 2011 06:35:34 +0000

And thew new PGXN is live! https://pgxn.org/ #woo

Releasing the new PGXN site ⓣ

Wed, 13 Apr 2011 06:24:47 +0000

Releasing the new PGXN site. Might be unstble for the next few minutes…

Releasing the new PGXN site ⓣ

Wed, 13 Apr 2011 06:24:47 +0000

Releasing the new PGXN site. Might be unstable for the next few minutes…

Hrm ⓣ

Wed, 13 Apr 2011 01:23:42 +0000

Hrm. Wondering if I should change the search URL to end in .json. https://api.pgxn.org/index.json

New version of the beta site is up ⓣ

Tue, 12 Apr 2011 04:59:09 +0000

New version of the beta site is up. Plan to switch over tomorrow. https://beta.pgxn.org/

Will be deploying a released candidate for the new ⓣ

Fri, 08 Apr 2011 09:55:49 +0000

Will be deploying a released candidate for the new site tomorrow. Getting close!

Added tag cloud and recent releases page to ⓣ

Thu, 07 Apr 2011 22:07:01 +0000

Added tag cloud and recent releases page to https://beta.pgxn.org/. Check it out!

Thank you Younicycle for your donation ⓣ

Sat, 02 Apr 2011 20:18:34 +0000

Thank you Younicycle for your donation! https://www.younicycle.com/

New home for all the PGXN source code ⓣ

Fri, 01 Apr 2011 22:11:04 +0000

New home for all the PGXN source code: https://github.com/pgxn/

Thoughts on Content Localization

Fri, 01 Apr 2011 05:33:16 +0000

The new site is coming along very nicely. I’m really excited about it. The addition of the API server has turned out to be an inspiration. I never realized how clean the separation between content and presentation could be until I did this – even though I’ve preached that very separation for years in one of my day jobs.

But speaking of that separation, I am running into a bit of an annoyance with content specific to the site. All the content from the network is working beautifully – I just fetch it from the API server. But the site has its own content page independent of the network, including an FAQ, a page on setting up a mirror, a list of our backers, etc. And the ugly bit has to do with localization.

Is it stupid to have a REST API return a different ⓣ

Thu, 31 Mar 2011 20:30:44 +0000

Is it stupid to have a REST API return a different JSON structure based on the value of a query parameter?

If a search requires a query, what status code ⓣ

Fri, 25 Mar 2011 07:52:23 +0000

If a search requires a query, what status code should be returned if there is no query? 404 not found doesn’t seem right.

Thoughts on Indexing and Documentation

Tue, 22 Mar 2011 05:13:00 +0000

So I’m designing the full text indexing for the PGXN search site. I’m modeling it on CPAN Search, which has been great. There are four search options:

Full documentation search. This is the most common. Includes doc title and body.
User search. Search on names, nicknames, email addresses, URIs, etc.
Distribution search. Search on distribution name, abstract, description, tags, and the README.
Extension search. Search on extension name and abstract.
Tag search. Search on tag name only.

The documentation search is the one I’m perhaps least sure about. It assumes that each extension in a distribution will have documentation. But so far that has not really been the practice for PostgreSQL extensions. Most folks seem to stick the documentation in the README. And even then it can be almost nothing. So a search for “count nulls” probably would not find “countnulls” extension, because there is no documentation. What should I do about this? I’m thinking one of:

Wow, @alexeyklyukin is the new PGXN champ ⓣ

Wed, 16 Mar 2011 21:47:51 +0000

Wow, @alexeyklyukin is the new PGXN champ! He now has 6 distributions! https://master.pgxn.org/by/user/alexk.json

manager.pgxn.org is back up ⓣ

Tue, 15 Mar 2011 22:20:11 +0000

https://manager.pgxn.org/ is back up and the issue has been fixed. Thank you for your patience!

Taking down ⓣ

Tue, 15 Mar 2011 21:42:03 +0000

Taking down https://manager.pgxn.org/ to do some troubleshooting. Will get it back up ASAP.

Cranking on the new site ⓣ

Sat, 12 Mar 2011 09:12:07 +0000

Cranking on the new site. Should be able to show something off late next week or early the following.

Reorganized things a big and updated the main ⓣ

Sat, 12 Mar 2011 00:04:16 +0000

Reorganized things a big and updated the main README. https://github.com/theory/pgxn

Question: Ajax and Search Engine Indexing

Fri, 04 Mar 2011 20:20:08 +0000

I’ve started working on the main (search) site in earnest now. The basic layout is done, and I’m working on the distribution view (mockup). My thinking so far has been that I would simply serve a page that requested, say, /dist/pgTAP/, and that page would use Ajax requests to fetch the data from the API server and display stuff. I think this will work pretty well except for one thing: 404s.

First deployment of the API server is up ⓣ

Thu, 03 Mar 2011 05:12:06 +0000

First deployment of the API server is up: https://api.pgxn.org/. Not much to look at without docs , but will build search site with it next.

Updated ⓣ

Sat, 26 Feb 2011 04:13:40 +0000

Updated https://manager.pgxn.org/ to add the abstract key to provides and updated existing distributions.

Modified Meta spec; added `abstract` to the ⓣ

Sat, 26 Feb 2011 01:49:14 +0000

Modified Meta spec; added abstract to the requires object. Markdown: https://pgxn.org/meta/spec.txt; HTML: https://pgxn.org/meta/spec.html

I'm going to have to create META.json-management ⓣ

Sat, 26 Feb 2011 00:06:19 +0000

I’m going to have to create META.json-management tools soon; I just fixed a broken META.json of my own – the second time I’ve had to do so.

Working on the REST API, realized I forgot to ⓣ

Fri, 25 Feb 2011 05:50:24 +0000

Working on the REST API, realized I forgot to include the release date in distribution metadata. Now added and all distributions updated.

Added an "Upload" link to the PGXN site nav, ⓣ

Wed, 23 Feb 2011 22:00:59 +0000

Added an “Upload” link to the PGXN site nav, linking to PGXN manager, the web app for releasing extensions to PGXN.

Just released Text::Markup to CPAN ⓣ

Wed, 23 Feb 2011 01:09:00 +0000

Just released Text::Markup to CPAN. Will be essential for parsing doc files for PGXN. https://p3rl.org/Text::Markup

PGXN Manager is back up ⓣ

Fri, 11 Feb 2011 00:28:18 +0000

PGXN Manager is back up. It now uses the native semver data type: https://master.pgxn.org/dist/semver/

Taking PGXN Manager down for a few minutes to ⓣ

Fri, 11 Feb 2011 00:16:00 +0000

Taking PGXN Manager down for a few minutes to upgrade it to use the new semver data type.

PGXN API RFC

Fri, 04 Feb 2011 04:05:33 +0000

Things slowed up a bit over the last couple of months, I admit. There are any number of reasons for that, not the least were the intrusion of the holidays and a little side project I’ve been hacking on after-hours (and sometimes during-hours). But I’m ramping things up again now and need your feedback on my current plans. Here’s what I’m working on: the search site.

Search Sites and APIs

Well, sort of. First of all, I’ve decided that the “search site” should not be a separate thing. The main site will be the search site. This is following the example of JSAN, as well as feedback from Graham Barr, who created and maintains CPAN Search. Apparently people are often confused that search.cpan.org is separate from www.cpan.org. No point in adding in confusion from the beginning. And besides, now that the PGXN fund-raising is over, I don’t know what else would go on the home page.

Started a new GitHub project to create a PGXN API ⓣ

Mon, 31 Jan 2011 23:25:03 +0000

Started a new GitHub project to create a PGXN API server: https://github.com/theory/pgxn-api

Started a new GitHub project to create a PGXN API ⓣ

Mon, 31 Jan 2011 23:25:03 +0000

Started a new GitHub project to create a PGXN API server: https://github.com/theory/pgxn-api

Want to use the PGXN logo in a presentation paper ⓣ

Wed, 05 Jan 2011 18:53:43 +0000

Want to use the PGXN logo in a presentation paper? Get the artwork under the CC BY-SA license here: https://pgxn.org/art/ /thanks @strongrrl

Mirror administration has been deployed to ⓣ

Tue, 04 Jan 2011 22:58:32 +0000

Mirror administration has been deployed to PGXN::Manager. Check out the newly-generated list: https://master.pgxn.org/meta/mirrors.json

Goooooooaaaal!

Wed, 01 Dec 2010 22:39:00 +0000

I’m pleased to announce that, thanks to a pledge from Enova Financial at the newly-minted “Patron” level, we have achieved our fundraising goal of $25,000. Thanks to Jim “Decibel!” Nasby for putting together the contribution that put us over the top!

I’m thrilled to find that this approach to getting a project going actually works. I’ve had the idea for PGXN for a long time, but knew that I was never going to be able to make it happen unless I could get help. So putting things together, writing a spec and project plan, and putting together the fundraising site with a prominent “fundraising thermometer” and contribution levels…well, it really paid off.

Thank you Enova Financial for your contribution ⓣ

Wed, 01 Dec 2010 20:37:45 +0000

Thank you Enova Financial for your contribution! Blog post soon.

What's on your PGXN wish list? Make it known ⓣ

Wed, 01 Dec 2010 20:15:37 +0000

What’s on your PGXN wish list? Make it known! https://github.com/theory/pgxn/wiki/PGXN-Wish-List/

PGWest Slides, Manager Access, Wish List, Back to Work

Wed, 24 Nov 2010 22:56:54 +0000

PGWest Slides, Back to Work

Well, back to work a bit, anyway. Tomorrow is Thanksgiving in the U.S., so I’m not likely to do much more until next week. But I’ll be heads down on it, then, working to get the search site done.

Before I talk about that, though, I realize that I forgot to post the link to my slides from PGWest. I actually posted them before the talk, I just never got round to mentioning it here. So, here it is:

Fixing Foreign Key Deadlocks in PostgreSQL ⓣ

Wed, 24 Nov 2010 22:32:32 +0000

Fixing Foreign Key Deadlocks in PostgreSQL https://www.justatheory.com/computers/databases/postgresql/fk-locks-project.html #help #community

What's on your PGXN wish list? Add it here ⓣ

Tue, 23 Nov 2010 01:33:38 +0000

What’s on your PGXN wish list? Add it here: https://github.com/theory/pgxn/wiki/PGXN-Wish-List

Slides from pgWest ⓣ

Thu, 18 Nov 2010 17:28:06 +0000

Slides from #pgWest: https://www.slideshare.net/justatheory/building-and-distributing-postgresql-extensions-without-learning-c

PGWest, Search Site Sneak Peak

Wed, 03 Nov 2010 21:09:00 +0000

I’ve been working on my PostgreSQL Conference West presentation, which heavily features PGXN, of course. I think it’s looking good. If you’re at PGWest or are in the San Francisco area and free, come see the talk! Should be a good introduction to creating PostgreSQL extensions and distributing them on PGXN. The latest bit I added is a section on the modifications needed to support the forthcoming CREATE EXTENSION support slated for 9.1. Fortunately it’s dead simple, and will make dealing with extensions in the database a lot simpler, administratively. Really looking forward to that. Of course I’ll post slides once the talk is over.

Checked off a bunch of items on the status page ⓣ

Sun, 24 Oct 2010 21:48:29 +0000

Checked off a bunch of items on the status page. Need to keep it more up-to-date. https://pgxn.org/status.html

Fixed the internal server error in the How To page ⓣ

Sun, 24 Oct 2010 21:39:48 +0000

Fixed the internal server error in the How To page. Check it out. https://manager.pgxn.org/howto

The new PGXN Manager "Internal Server Error" page

Fri, 22 Oct 2010 19:12:14 +0000

The new PGXN Manager “Internal Server Error” page.

Reserved Extensions

Thu, 21 Oct 2010 20:43:00 +0000

I’m thinking about how to add support for reserved extensions. These are extensions that one needs to depend on, but aren’t distributed via PGXN. Primarily, this means stuff distributed with the PostgreSQL core, including:

PostgreSQL itself
Bundled procedural languages (plpgsql, plperl, etc.)
All contributed modules

Basically, anything that an extension might want to declare as a dependency, but that isn’t on PGXN itself.

There are a number of ways to do this. Which do you think would be the best approach?

Creating Distributions

Tue, 19 Oct 2010 22:28:00 +0000

Following the upload of pair to PGXN, I wanted to take a few minutes to write about how to structure a PGXN distribution.

OMG Distribution WTF?

First of all, what is a “distribution” in the PGXN sense? Basically, it’s a collection of one or more PostgreSQL extensions. That’s it.

So why allow more than one extension? Maybe no PGXN distribution will ever have more than one extension. After all, the goal should be many focused, minimalist tools that folks can combine in their apps. But sometimes it doesn’t work out that way.

Fundraising and Development Update

Tue, 19 Oct 2010 18:43:57 +0000

Yesterday was a busy day. In addition to making the first PGXN release, I updated the fundraising spreadsheet and then the thermometer displayed on the main site. The good news is that things are coming along nicely. Thanks to recent contributions from Command Prompt, Marchex, Hitoshi Harada, and 25th-floor, we are now just $2500 short of our goal of $25,000. Thank you all!

Can you help us get to our goal in time for PgWest 2010, which is November 2-4 in San Francisco? I’ll be giving a talk there, “Building and Distributing PostgreSQL Extensions Without Learning C”, in which PGXN will of course be featured. Would be great to announce that the fundraising was successful.

First Upload to PGXN

Tue, 19 Oct 2010 17:46:44 +0000

Last night I deployed PGXN::Manager v0.2.1 and uploaded the first distribution, pair. If you follow that link you’ll see three files:

pair-0.1.0.json is metadata file generated by PGXN manager to describe the distribution. Most of its data is taken from the META.json included in the uploaded zip file, but a few keys, like “sha1”, are generated, and others, like “release_status” are added if they’re not in the included META.json.
pair-0.0.1.readme is a copy of the README file distributed with pair.
pair-0.1.0.pgz is the distribution zip file. That’s the file you want to download, unzip, and build and install.

Following the spec I previously wrote up, there are a number of other files that get created when a new distribution is uploaded to PGXN. For the pair extension, we got:

First extension library uploaded to PGXN and ⓣ

Tue, 19 Oct 2010 04:03:23 +0000

First extension library uploaded to PGXN and making its way to the mirrors. Blog post tomorrow.

Thank you August Zajonc for your contribution ⓣ

Mon, 18 Oct 2010 18:38:40 +0000

Thank you August Zajonc for your contribution!

How Can I Detect a Proxied SSL Request?

Mon, 11 Oct 2010 15:00:29 +0000

I’ve been thinking about how best to create two sections of the PGXN Manager site, one that requires authentication and is on SSL and one that’s public. So far, I’ve just had all the authenticated stuff go to /auth (because I’m using basic auth), but what I want to do now is require authentication if the connection is via SSL. However, using a reverse proxy with mod_proxy to map to the PGXN Manger Plack app running on an internal port, I’ve found no environment variables passed through that would allow me, in code, to determine whether a request is via a proxied SSL or non-SSL connection. Most irritating.

Mail List, SSL

Sat, 09 Oct 2010 06:01:08 +0000

I’m this close to having PGXN Manager ready for a limited beta. I’ve got it running on Kineticode’s server, and have been tweaking things here and there, fixing some bugs and filling in a few missing bits. In the next couple of days I’ll get some more kinks worked out and then start inviting folks in. If you’re interested – especially if you have an extension you’d like to release, leave a comment here to let me know.

RT @theory Hrm… Free GoDaddy OSS SSL cert ( ⓣ

Sat, 09 Oct 2010 03:07:55 +0000

RT @theory Hrm… Free GoDaddy OSS SSL cert (https://bit.ly/9Ek7ea) or Free StartSSL cert (https://www.startssl.com/?app=32)? #ssl #pgxn

PGXN Manager now has 4035 assertions ⓣ

Thu, 07 Oct 2010 05:19:12 +0000

PGXN Manager now has 4035 assertions. Tests: https://github.com/theory/pgxn-manager/tree/master/t/

Thanks to Michael Nacos and Hitoshi Harada for ⓣ

Wed, 06 Oct 2010 02:57:42 +0000

Thanks to Michael Nacos and Hitoshi Harada for their donations. We’re getting closer to our goal!

Thank you depesz for your contribution ⓣ

Fri, 01 Oct 2010 18:33:24 +0000

Thank you depesz for your contribution! https://pgxn.org/contributors.html

Thank you Chris Spotts for your contribution ⓣ

Fri, 01 Oct 2010 18:18:03 +0000

Thank you Chris Spotts for your contribution! https://pgxn.org/contributors.html

Trying to meet our fundraising goal by the end of ⓣ

Thu, 30 Sep 2010 18:06:49 +0000

Trying to meet our fundraising goal by the end of the week, and some folks are stepping up. Have you?

Just successfully uploaded a distribution archive ⓣ

Wed, 29 Sep 2010 23:27:24 +0000

Just successfully uploaded a distribution archive to the test server! #AlmostThere

Conflict and Redirection on POST

Wed, 29 Sep 2010 21:10:00 +0000

Had an interesting discussion on #plack. The upload form, which takes a POST request for an upload, sends a redirect on a successful form submission. This is known as the Post/Redirect/Get (RDG) pattern, though I didn’t know that before DuckDuckGoing it it today. But I realized that the code was not redirecting on a failed form submission, but reloading the form. I was thinking that one should always redirect on POST, and so was looking into a Rails-like flash pattern to cache an error message and the form contents on the redirect. But in this discussion, I learned a few things:

Account Requests and Moderation

Tue, 28 Sep 2010 18:47:37 +0000

Last week I created the PGXN Manager interface for requesting a PGXN user account. It looks like this:

I really like the placeholder support in HTML 5, here nicely rendered by Safari. I’ve also used the jQuery Validation plugin to validate the form fields. So if you try to submit an incomplete form, it will complain before submitting, like so:

The back end does similar validation if you have JavaScript disabled, so it should degrade nicely. I think I might add a Twitter field so @pgxn can credit users for their uploads, but otherwise this is done.

Quick Status Update

Tue, 28 Sep 2010 17:29:48 +0000

Sometimes I wish I weren’t such a perfectionist.

But not often.

A quick update:

So far, I’ve spent 43 hours on the PGXN Manager database. I had estimated 24 hours. Ha ha ha ha ha!

I’ve spent 45 hours on the implementation of the app itself. I had estimated 40 hours.

How will I make up the difference? I’m not sure, really. I’ve already under-recorded my hours quite a lot, basically donating my time to create SemVer and to create the basic HTML layout for the site (I suck at design, but found a good template to base it on). That will likely continue. This is an OSS project, and while I’m getting paid to work on it thanks to our generous contributors, I’m also contributing quite a lot of time to it.

Hacking furiously, hope to have something to show ⓣ

Wed, 22 Sep 2010 16:37:14 +0000

Hacking furiously, hope to have something to show off by tomorrow.

Architecture, New Extension JSON Format

Fri, 17 Sep 2010 17:33:00 +0000

I’m making good progress on PGXN Manager. Hopefully I can start alpha testing it next week. As I mentioned previously, I had estimated 40 hours of work to create it, but was hoping to get it done in around 30 (because I spent 10 extra hours on the database design). So far I’m at 23 hours, so it’s looking pretty good.

Architecturally, I’ve gone for a very minimal Plack-based app. No Catalyst, Jifty, or even Dancer. Just a very simple Plack app that uses Router::Simple::Sinatraish to route URIs to the appropriate controller actions (which are just class methods). The controller just dispatches to Template::Declare-based templates for the HTML rendering. I guess I’ve kind of created my own framework here, but really, there ain’t much to it. This app is simple enough that I couldn’t see the use of adding all the overhead of a framework.

Implementation of the upload server is humming ⓣ

Fri, 17 Sep 2010 04:43:06 +0000

Implementation of the upload server is humming along. https://github.com/theory/pgxn-manager/

On Legacy File Systems

Wed, 08 Sep 2010 23:41:00 +0000

When we last looked at the organization of the mirror, I was pretty happy with the design except for one thing: The use of the letter hashing variables in the URI templates. It’s just ugly and, damnit, is it really necessary anymore? I mean, sure, I had to work around this issue in Bricolage back in 2005, but on modern file systems like zfs and ext3, does it really matter anymore?

I was chatting about this with Schwern just now, and he thought it just didn’t matter anymore. So I asked my fellow PGX associate Jeff Frost about this, and he said, “Used to be anytime you went above 1000 entries in one directory, things would start to slow down. Certainly reiserfs, xfs, jfs, and zfs don’t suffer that issue.” But what about ext3?

Status Update: DB API, Extension Versions RFC

Tue, 07 Sep 2010 18:46:38 +0000

I spent most of the time I had to work on PGXN the last two weeks creating the database for PGXN Manager. I had estimated 24 hours of work to design the database. So far I’ve logged 34 hours. But I think that will come out in the wash, really, because I did a lot of work to generate JSON from database functions. This is code I had originally expected to do in the app layer. I’m starting work on that this week, with an estimated 40 hours. Hope I can do it in 30. :-)

Turns out you don't need ALIAS after all ⓣ

Tue, 07 Sep 2010 16:52:23 +0000

Turns out you don’t need ALIAS after all. Updated post: https://blog.pgxn.org/post/1053165383/alias-in-vogue#update-20100907 #plpgsql

Thank you @darkixion for your latest donation ⓣ

Thu, 02 Sep 2010 18:20:12 +0000

Thank you @darkixion for your latest donation! https://pgxn.org/contributors.html #sponsors

ALIAS Back in Vogue

Thu, 02 Sep 2010 13:00:00 +0000

I’ve been hard at work on PGXN Manager, the app for users to upload distributions to PGXN. I am of course following my own dictum: “the database is the model.” As a result, I’ve been creating an API for creating, updating, and deleting entities, as well as generating JSON (more on that later).

I’m also using PostgreSQL 9. The motivation to make the jump to 9.0 was to to try to use the JSON data type patch, but I abandoned it when I couldn’t get it to compile. (I might come back to it later, but right now I’m trying to practice YAGNI and JFDI so that I don’t end up owning a yak farm). But there are other reasons to stick with 9.0, like improved hstore support, the DO statement, and named parameters.

Generating JSON in SQL is fun ⓣ

Wed, 01 Sep 2010 06:56:11 +0000

Generating JSON in SQL is fun! https://bit.ly/dolkKb #json #pgxn

Changed "owner" to maintainer in the meta spec ⓣ

Tue, 24 Aug 2010 21:26:09 +0000

Changed “owner” to maintainer in the meta spec https://github.com/theory/pgxn/wiki/PGXN-Meta-Spec

PGXN Manager now has users and admins ⓣ

Mon, 23 Aug 2010 23:28:33 +0000

PGXN Manager now has users and admins. https://github.com/theory/pgxn-manager/commit/0be7dd7d819e8bb622bfbe7415f1647aeede8dfa #progress

A RESTful Directory

Sat, 21 Aug 2010 18:52:00 +0000

Following my post outlining a possible network directory structure, Aristotle Pagaltzis saw fit to bug me via email about a different approach. I couldn’t understand WTF he was talking about until today. Then it lit my brain on fire. As a result, I now think that there is a much better way to organize the metadata files for the PGXN – one that happens not to include any symbolic links (which is something that Andreas König has been flagging, via email, as a possible bottleneck).

Thank you Jeff Davis for your contriubution ⓣ

Fri, 20 Aug 2010 21:30:58 +0000

Thank you Jeff Davis for your contriubution! https://pgxn.org/contributors.html

RFC: The PGXN Metadata Specification

Tue, 17 Aug 2010 13:00:00 +0000

I’ve posted a draft of the “PGXN distribution metadata specification,” or PGXN Meta Spec." This document specifies the structure and format of the META.json file that PGXN will require in every distribution. In fact, this is the only required file in a distribution. Its job is to describe the distribution, its extensions, and its dependencies, among other things. This file is key to the whole thing.

To create it, I’ve ported the CPAN Meta Spec, version 2, but with all deprecated fields removed, and some of the more complex stuff taken out. I also made a couple of the “required” fields “optional.” At its simplest, the file might look something like this:

First draft of the PGXN Meta Spec ⓣ

Mon, 16 Aug 2010 23:54:27 +0000

First draft of the PGXN Meta Spec. https://github.com/theory/pgxn/wiki/PGXN-Meta-Spec

About the Logo

Mon, 16 Aug 2010 13:00:00 +0000

I decided to get on creating the PGXN logo sooner rather that later, both to create a stronger identity for the project and to have a uniform look across the various services (the site, blog, twitter stream, identi.ca, etc.). So I began hunting around for useful iconography to inspire a design direction.

Let me tell, you, that’s a lot easier than it sounds. After browsing through images returned for queries such as “pixin,” “pig skin,” “zen,” “zen stones,” “pixies,” and god knows what else, I finally focused on “database icon.” This was much more useful, as I hit on images such as the one to the right. Pretty standard database icon, right? Three disks stacked up. I’m not sure why this is the classic graphical abstraction for a database, but I’ve seen it a zillion times. So often that it’s a cliché.

Thoughts on the Network Directory Structure

Sat, 14 Aug 2010 23:59:12 +0000

I’ve been thinking about the arrangement of stuff to be distributed to the mirrors. In doing so, I’ve kept three goals in mind:

Allow things to scale.
Try to put things in intuitive locations.
Allow for metadata to be stored to act as a Web API.

The first two are a bit mutually-contradicting. Thinking scalability mainly means minimizing the chances that too many files can be put into a single directory. CPAN started out with all author directories in a single directory. Given the sheer number of CPAN contributors, this quickly got to be a bottleneck. To correct for that, they started “hashing” the first two letters of author names to create subdirectories. My CPAN directory, for example, is D/DW/DWHEELER. That doesn’t quite make for an intuitive location, but it’s not bad, and the tradeoff seems sufficient to keep things sane.

Started documenting admin stuff in the wiki ⓣ

Sat, 14 Aug 2010 22:19:39 +0000

Started documenting admin stuff in the wiki. https://github.com/theory/pgxn/wiki

Start Your Mirrors!

Fri, 13 Aug 2010 23:45:00 +0000

Work has finally started. We now have mirroring. This was the first task for the project, and it’s now checked off on the status page. Hurrah!

This turned out to be a pretty simple task, of course. For now I’m using the Kineticode (my other employer) server to host both the PGXN site and the master mirror. This is temporary, just a way to get things up and going as quickly as possible. The master mirror runs rsyncd with read-only access to the pgxn path. So all you have to do to mirror it is set up a cron job like:

New Blog feed sans Tweets ⓣ

Fri, 13 Aug 2010 22:12:00 +0000

New Blog feed sans Tweets: https://feedrinse.com/services/rinse/?rinsedurl=1be2d432150dc8f60f15ce3e4d521b64

We have mirroring ⓣ

Fri, 13 Aug 2010 19:31:25 +0000

We have mirroring! https://pgxn.org/mirroring.html

Thank you Greg Smith for your contribution ⓣ

Fri, 13 Aug 2010 16:58:47 +0000

Thank you Greg Smith for your contribution! https://pgxn.org/contributors.html

Thank you www.urbandb.com for your contribution ⓣ

Thu, 12 Aug 2010 21:05:55 +0000

Thank you www.urbandb.com for your contribution! https://pgxn.org/contributors.html

Follow us on identi.ca ⓣ

Thu, 12 Aug 2010 17:50:44 +0000

Thank you @xdg for your contribution ⓣ

Thu, 12 Aug 2010 17:30:00 +0000

Thank you @xdg for your contribution! https://pgxn.org/contributors.html

Fundraising Update

Wed, 04 Aug 2010 19:50:24 +0000

This project started when I wrote up the specification and got general approval from pgsql-hackers. Then, at PGCon, it made the list of development priorities for PostgreSQL 9.1 (when I was then calling it “PGAN”). It was when MyYearbook.com’s Gavin Roy pledged a founding contribution of $5,000 (also at PGCon) that I was able to put in the time to create the fundraising site and start soliciting donations. I’m really grateful to Gavin for stepping up like that, putting his money on the line on the assumption that I would be as good as my word and get the work done.

Blog and Twitter

Wed, 04 Aug 2010 03:01:00 +0000

I’m pleased to announce two things:

This blog. Here I’ll make regular posts as I develop PGXN, and to keep it up to date with regular operational updates and release announcements (for extensions on PGXN, not just PGXN itself) once the site goes live. Subscribe to the feed to stay up-to-date in your favorite reader.
The PGXN Twitter stream. Follow along as our fundraising draws to an exciting conclusion and development begins. Once the site is launched, new extension releases will be tweeted, as well.