Decent article, but some remarks:<p>1) It seems these two statements conflict with each other:<p>> The oldest such transaction sets the cutoff—referred to as the "MVCC horizon." Until that transaction completes, every dead tuple <i>newer than its snapshot</i> is retained.<p>and<p>> For example, imagine three analytics queries, each running for 40 seconds, staggered 20 seconds apart. No individual query would trigger a timeout for running too long. But because one is always active, the horizon never advances, and the effect on vacuum is the same as one transaction that never ends.<p>If the three analytics *transactions* (it's transactions that matter, not queries, although there is some subtlety around deferred transactions not acquiring a snapshot until the first query) are started
at different times, they will have staggered snapshots and so once the first completes, this should allow the vacuum to advance.<p>2) Although the problem about this query:<p><pre><code> SELECT * FROM jobs
WHERE status = 'pending'
ORDER BY run_at
LIMIT 1
FOR UPDATE SKIP LOCKED;
</code></pre>
having to consider dead tuples is a genuine concern and performance problem,
this can also be mitigated by adding a monotonically increasing column
and adding a `WHERE column < ?` clause, provided you have also added an
index to make that pagination efficient.
This way you don't need to consider dead tuples and they 'only' waste space
whilst waiting to be vacuumed, rather than also bogging down read perf.<p>There is a little subtlety around how you guarantee that the column is monotonically
increasing, given concurrent writers, but the answer to that depends on what tricks
you can fit into your application.<p>3) I almost want to say that the one-line summary is 'Don't combine (very) long-running transactions with (very) high transaction rates in Postgres'<p>(Is this a fair representation?)
For (2): the problem is that the index would still need to keep the dead tuples, until Postgres is positively certain that no transaction holds them, and runs vacuum over them. It may speed up things a bit, but would still overflow the disk storage eventually. It may still prevent other tables from being vacuumed, too!
For disk usage, yes this doesn't address anything.<p>But for read performance (which is IMO what the section in the article was motivated by), it doesn't actually matter to have a bunch of entries corresponding to dead tuples in your index, provided Postgres doesn't need to actually consider the dead tuples as part of your query.<p>So if you have a monotonically increasing `job_id` and that's indexed, then
so long as you process your jobs in increasing `job_id` order, you can use the index and guarantee you don't have to keep reconsidering the dead tuples corresponding to jobs that already completed (if that makes sense).<p>[This is because the index is a b-tree, which supports efficient (O(log n) page reads for n entries) seeking on (any prefix of) the columns in the index.]