Found while clearing the box before a benchmark: a backend from the unique_conc suite, started Sat Jul 25 16:38:33, still running three days later at 98.9% CPU.
PID PPID STARTED ELAPSED TIME %CPU COMMAND
1017577 1 Sat Jul 25 16:38:33 2026 3-02:37:08 3-01:50:19 98.9 postgres: postgres uconc [local] INSERT
It is a userspace spin, not a wait
State: R (running)
utime = 26577363 ticks (~3 days of pure userspace CPU)
stime = 4586 ticks (essentially nothing in the kernel)
voluntary_ctxt_switches: 33
nonvoluntary_ctxt_switches: 19881103
33 voluntary context switches in three days. It never yields, never blocks, never sleeps. This is a CPU-bound loop, not a lock wait and not a slow query.
It outlived its postmaster
PPID is 1 and the cluster is a dead test directory (/tmp/pgcolumnar-uconc.ZCXwt0/data) whose postmaster is long gone. A backend normally notices postmaster death while waiting on a latch; a loop that never waits and never reaches a CHECK_FOR_INTERRUPTS never finds out. So whatever the loop is, it contains no interrupt check — which is why this was still burning a core three days later instead of dying with its cluster.
That also makes it a gate-integrity problem: it silently contaminated every measurement taken on this box since Jul 25. At least one "loaded box" retraction this week was measured alongside it.
Where it is looping
Sampled six times with gdb over ~20 seconds (non-assert -O2 build, so deeper frames are unreliable; the top frames and the pgcolumnar symbols are the signal). The stack is consistently an INSERT with index maintenance:
#0 LWLockReleaseInternal / LWLockAttemptLock / hash_search_with_hash_value (varies)
...
#3 _bt_doinsert
#4 btinsert
#5 ExecInsertIndexTuples
#6 ExecInsert
#7 ExecModifyTable
#8 standard_ExecutorRun
and, in other samples, pgcolumnar frames reached from inside that same loop:
ColumnarReadOptions () from pgcolumnar.so (via systable_endscan / index_close)
ColumnarReadZoneMapList () from pgcolumnar.so (via systable_beginscan / index_open)
ColumnarComputeFullyDeletedGroups () from pgcolumnar.so
delete_vector_flush_buffer () from pgcolumnar.so
ColumnarReadOptions and ColumnarReadZoneMapList are catalog scans, and they are being re-entered from within the row-insert loop. Per-row catalog scans in a write path is the same shape as the quadratic fetch-by-row problem, and would explain a statement that makes progress so slowly it looks infinite — but I have not proven which of the two this is:
- a genuinely infinite loop, or
- a finite loop whose per-row cost is so bad it had not finished in three days.
Distinguishing them is the first job. The evidence does not settle it and I am not going to guess.
Reproduction status
Not yet reproduced. The suite is unique_conc (concurrent inserts against a unique constraint); the cluster used FIFOs s1.in/s2.in for two psql sessions, so a lost session may be part of the setup. Its server log shows only an unrelated startup error:
ERROR: extension "citext" is not available
The process has been killed and the container rebooted, so this instance is gone. Full evidence — process state, cluster layout, and all six backtrace samples — was captured before it was destroyed.
Why this is worth its own issue
Whatever the loop turns out to be, two properties are defects on their own:
- No interrupt check. A loop in a write path that cannot be cancelled, and does not notice postmaster death, is a hang that a DBA cannot get out of without
kill -9. CHECK_FOR_INTERRUPTS in the loop is the floor.
- Per-row catalog reads in the insert path.
ColumnarReadOptions / ColumnarReadZoneMapList should be read once per statement, not per row, regardless of what caused this particular spin.
Related: #143 (fetch-by-row quadratic), #155 (bulk-load throughput).
Found while clearing the box before a benchmark: a backend from the
unique_concsuite, started Sat Jul 25 16:38:33, still running three days later at 98.9% CPU.It is a userspace spin, not a wait
33 voluntary context switches in three days. It never yields, never blocks, never sleeps. This is a CPU-bound loop, not a lock wait and not a slow query.
It outlived its postmaster
PPIDis 1 and the cluster is a dead test directory (/tmp/pgcolumnar-uconc.ZCXwt0/data) whose postmaster is long gone. A backend normally notices postmaster death while waiting on a latch; a loop that never waits and never reaches aCHECK_FOR_INTERRUPTSnever finds out. So whatever the loop is, it contains no interrupt check — which is why this was still burning a core three days later instead of dying with its cluster.That also makes it a gate-integrity problem: it silently contaminated every measurement taken on this box since Jul 25. At least one "loaded box" retraction this week was measured alongside it.
Where it is looping
Sampled six times with
gdbover ~20 seconds (non-assert-O2build, so deeper frames are unreliable; the top frames and the pgcolumnar symbols are the signal). The stack is consistently an INSERT with index maintenance:and, in other samples, pgcolumnar frames reached from inside that same loop:
ColumnarReadOptionsandColumnarReadZoneMapListare catalog scans, and they are being re-entered from within the row-insert loop. Per-row catalog scans in a write path is the same shape as the quadratic fetch-by-row problem, and would explain a statement that makes progress so slowly it looks infinite — but I have not proven which of the two this is:Distinguishing them is the first job. The evidence does not settle it and I am not going to guess.
Reproduction status
Not yet reproduced. The suite is
unique_conc(concurrent inserts against a unique constraint); the cluster used FIFOss1.in/s2.infor two psql sessions, so a lost session may be part of the setup. Its server log shows only an unrelated startup error:The process has been killed and the container rebooted, so this instance is gone. Full evidence — process state, cluster layout, and all six backtrace samples — was captured before it was destroyed.
Why this is worth its own issue
Whatever the loop turns out to be, two properties are defects on their own:
kill -9.CHECK_FOR_INTERRUPTSin the loop is the floor.ColumnarReadOptions/ColumnarReadZoneMapListshould be read once per statement, not per row, regardless of what caused this particular spin.Related: #143 (fetch-by-row quadratic), #155 (bulk-load throughput).