The Install Guide Installs No Database

There is a job posting open at Percona for a technical writer on PostgreSQL. Under the heading of what would really make you stand out, it lists a track record of contributing to open source, a solid understanding of internals, familiarity with at-rest encryption, and the ability to build automated documentation testing and validation tools. I have written database documentation before, at developer depth, in review cycles with the engineers who built the thing. But it was MongoDB, years ago, and I would be lying if I claimed PostgreSQL. The honest options were to say so and hope, or to go and learn it in the one way I trust, which is to pick something with a sharp edge and run into it deliberately. So I did not read their documentation. I executed it.

Nobody Runs the Documentation

A docs pipeline is a strange machine when you look straight at it. It lints prose. It checks that links resolve. It builds a static site and screams if a page fails to render. What it never does, in any shop I have worked in, is run the commands on the page and see whether they still work. This is a peculiar gap, because documentation is the only part of a product that is executed by a human being with no error handling. Code has tests. Infrastructure has a plan and an apply. Documentation has a reader at eleven at night, pasting a block into a terminal and believing it. Code samples rot silently and specifically. A function gets renamed and the prose gets updated because the prose is what a reviewer reads; the code block keeps its syntax highlighting, keeps looking authoritative, and keeps being wrong. Nothing in the build has an opinion about this, because rendering correctly and being correct are unrelated properties. The idea, then, is not complicated. Take the commands out of the Markdown. Run them against a real server. Report where a reader following along would stop.

The First Twenty Minutes

I started by hand, because starting by hand is how you learn what the tool needs to be. The target was pg_tde, Percona's transparent data encryption extension for PostgreSQL. It is a good target for a reason that has nothing to do with the job: it is the kind of feature where the documentation carries real weight. Nobody encrypts a database casually. You do it because somebody is going to audit you, and you follow the instructions exactly. I built the most faithful stand-in for a fresh Ubuntu server I could manage without provisioning one: Ubuntu 24.04 in a container, a non-root user, a working sudo, nothing else installed. The sudo matters. The docs say sudo, and if you strip it you are testing a document you have edited rather than the one they published. Then I walked the install page, one block at a time. The first five commands are fine. Install some dependencies, fetch the percona-release package, install it, enable the PostgreSQL 18 repository, update the cache. All of it works, on arm64, natively, which was a pleasant surprise on its own. The sixth command is this:

sudo apt-get install -y percona-pg-tde(pg-version)

That is not a wrong package name. Parentheses are shell metacharacters, so what a reader gets is this:

bash: syntax error near unexpected token `('

The intent is obvious from the worked example immediately below it, which says percona-pg-tde18 and works perfectly. So a careful reader shrugs and moves on. But the page's own structure invites you to copy the general form first, and the block is styled exactly like the five correct blocks above it. It has earned your trust and then spent it. I checked whether this was a rendering artefact of the docs source. It is not. It is on the published page. It is also on the yum page, and twice on the uninstall page, which makes it a template defect rather than a typo.

The Part Where There Is No Database

That was the appetiser. Here is the finding. The install page ends with the extension installed. At the bottom there is a Next Steps button, and it goes to the configuration page, which opens by telling you to either edit postgresql.conf or connect with psql as a superuser and run an ALTER SYSTEM. I ran it. What I got was this:

sudo: unknown user postgres

So I stopped and looked at what the install had actually produced:

$ dpkg -l | grep -E 'percona|postgres'
libpq5:arm64        3:18.6-1.noble
percona-pg-tde18    1:2.2.2-1.noble
percona-release     1.0-34.generic

$ command -v postgres psql pg_ctlcluster initdb
(nothing)

There is no server. There is no psql. There is no cluster, no postgres system user, and no systemd unit for the next page to restart. What the documented install path gives you is pg_tde.so, a handful of pg_tde_* command line tools, and the Postgres client library, sitting in a directory waiting for a database that was never mentioned. The missing step is sudo apt-get install -y percona-ppg-server-18. It appears nowhere in the install path. The page's Preconditions section quietly assumes Percona Server for PostgreSQL is already there, but it never says so and never links to where you would get it. If you already run Percona's distribution, none of this touches you and the page is fine. If you arrived because you searched for how to encrypt a PostgreSQL database, you follow two pages, complete both, and hit a wall with no indication of which of the last twenty commands went wrong. Thirteen further commands across three pages cannot run, all for this one reason. I want to be careful about how much weight this carries. It is not a code bug, nobody's data is at risk, and an experienced Percona operator would never notice. It is a documentation defect, which is a smaller thing. But it is the specific kind of smaller thing that costs a new user an evening and sends them to a competitor's page, and it is invisible to every check in a normal docs pipeline, because both pages render beautifully.

Then I Wrote The Tool, And The Tool Lied To Me

Finding this by hand took an afternoon. Finding it again next month, after the docs change, would take another afternoon, which means nobody will do it. So the afternoon is only worth something if it becomes a thing that runs. The tool is called assay. It pulls the code blocks out of Markdown, works out which ones a reader would actually run, and executes them in a container against a live server. Two commands: a static half that needs no container and is cheap enough for every pull request, and a slow half that boots a machine and walks a documentation path in reading order.

Terminal recording: the published install command failing with a bash syntax error, then assay lint reporting eight failures across the pg_tde doc set, then a grep showing the canary string readable in the plain heap file and absent from the encrypted one.

The reading order is the whole design. My first instinct was a fresh container per page, because that is what a well-behaved test suite does and it makes every result independent. It is also exactly wrong. Documentation is a chain. Page three assumes what page one told you to install, and the defect I had just spent an afternoon finding is a page assuming state that no earlier page establishes. Run each page in a clean room and that class of bug becomes structurally invisible, because every page looks fine on its own. So a run is one machine, pages execute in the order the docs link them, and state accumulates exactly as it does for a reader. I pointed it at the doc set and it told me there were sixteen failures. There are not sixteen bugs. There is one missing server, and then thirteen commands that fail with unknown user postgres because of it. A tool that reports that as sixteen defects is not being thorough, it is inflating its own findings, and the first maintainer who reads that report correctly concludes the tool is noise and mutes it forever. The number is worse than useless because it is confidently wrong in the direction that flatters the tool. So the runner now attributes cascades. When a failure carries a signature that means a prerequisite is missing, and an earlier failure carries the same signature, the later one is marked blocked and pointed at its root cause. The report reads:

PASS  4   FAIL  3   BLOCKED  13   ADAPTED  4

13 further commands could not run; all blocked by setup.md:22
They are downstream of one root cause, not separate defects.

3 documentation defects -> exit 1

Three, not sixteen. The thirteen are still listed, because they are evidence, but they are not counted as findings. The static half made the same mistake in a different costume. My identifier rule looks for a page that inspects a table it never created, which catches two genuine defects: a page that creates albums and then checks table_name, and a page that creates customer_table and then tells you to inspect test1. Both leave a reader staring at a relation that does not exist. Uncalibrated, that rule reported eleven failures. Four were real. The other seven were system catalogs that nobody creates because PostgreSQL ships them, and fragment pages that use mytable on purpose because the reader is expected to bring their own. Three different things look identical to a naive matcher, and only one of them is a bug. This is the second time I have shipped a rule that was confidently wrong before it was calibrated, and the first time cost me more. In chokepoint I set an example policy threshold at 0.7 that sat above the entire measured separation band, which meant it would have shipped looking authoritative and never fired once, on anything, ever. I now assume that any threshold or heuristic I write is wrong until it has been run against a real corpus and argued with. The calibration is in the tests here, pinning both the defects the rule must catch and the shapes it must not.

The Category That Makes It Trustworthy

There is a third outcome besides pass and fail, and it is the one that decides whether anybody believes the report. systemctl restart postgresql cannot work in a container. There is no init system to talk to. That is my harness being a container, not Percona's documentation being wrong, and counting it as a defect would be dishonest in a way that quietly poisons everything else in the report. So substitutions are declared in the manifest, with a written reason, and they show up in the output labelled as what they are:

substitutions:
  - match: "systemctl restart postgresql"
    replace: "sudo pg_ctlcluster 18 main restart"
    reason: "no init system in a container; pg_ctlcluster is Debian's documented equivalent"

The same applies to priming the apt package lists before starting. A freshly provisioned Ubuntu server has them; a container image ships with them deleted. Without priming, the docs would appear to be missing an apt-get update that they are entirely right to omit. Every one of these is a place where I have put my thumb on the scale, so every one of them is printed in the report where somebody can disagree with me. A tool that hides its environmental assumptions is asking to be trusted. A tool that prints them is asking to be checked, which is the only request that survives contact with a maintainer who did not write it.

The Rest Of The List

Compressed, because the pattern is more interesting than the inventory. A shell command tagged as SQL. The configuration page offers psql -d template1 -c 'CREATE EXTENSION pg_tde;' inside a block marked sql. The three blocks above it are correctly marked and genuinely SQL, so by the time you reach this one the tag has earned your trust. Pasted into psql, it raises a syntax error at or near psql. Run in a shell, it is fine. A config file line tagged as bash, in the other direction, on the same page. A precondition that is not true. The validation page opens by saying that once the extension is enabled you can start encrypting data. You cannot. Its own worked example fails with principal key not configured, because you also need a key provider, which is documented on a different page. The information exists. The page just states a precondition it does not meet and then hands you an example that cannot run. And my favourite, because it is the one a reader has no chance of diagnosing. The keyring page walks you through setting a database-scoped key, then offers a tip: check your key information, including its creation time, with pg_tde_default_key_info(). Follow the page exactly and that function returns four empty fields. It reports the default key, set through a global provider, which is not what the page just told you to make. The right function is pg_tde_key_info(), and it returns everything, creation timestamp included. A reader who hits that sees blank output and has no way to tell whether their key failed to set or they ran the wrong function. It is the second one. Their encryption is working perfectly.

It Does Work

I want to end on this, because a post that is only a list of defects gives a false impression of the thing. Once the missing server install is supplied, the whole path works exactly as documented. The key provider is created, the key is set, the table is created with the tde_heap access method, and pg_tde_is_encrypted returns true. And then you can go and look, which is the part I enjoyed most. I wrote the same recognisable string into an ordinary table and an encrypted one, forced a checkpoint, asked PostgreSQL where each table lives on disk, and grepped the raw heap files:

plain_albums  : 1
albums (tde)  : 0

One row, two tables, the same string. Readable in the raw file of one, absent from the other. There is something very satisfying about a security feature you can confirm with grep. The extension does what it says. It is the eight steps in front of it that need the work.

What I Actually Think This Is

The findings are worth reporting and I will report them upstream. But the finding under the findings is that none of these are hard to catch. Every one was found by a few hundred lines of Python doing the least imaginative thing available, which is running the commands. The reason nobody catches them is not difficulty. It is that documentation sits in a category we have collectively agreed is prose, and prose gets reviewed by reading. We built an entire discipline called docs-as-code, moved documentation into git, put it through pull requests and CI, and then declined to do the one thing that makes code code, which is executing it to see if it works. A docs build that checks a page renders is checking the least interesting property that page has.

Method, since it matters. I did not write the Python here by hand; I ran this the way I run all my technical work now, which is directing Claude and deciding what to test next. The judgement calls in this post are mine and so are the mistakes, including the sixteen-failures one, which I made first and caught second. The words are mine throughout, which is the one part I have never delegated. The tool is assay, Apache-2.0. The findings, with the environment, commands and observed output for each item, are in findings/pg_tde-2.2.2.md. Everything was run against Percona Server for PostgreSQL 18.6.1 with pg_tde 2.2.2 on Ubuntu 24.04, arm64, on 5 September 2026.