Skip to content

Use Split Update when partition key is updated and expand is in progress - #2374

Open
andr-sokolov wants to merge 3 commits into
apache:mainfrom
andr-sokolov:check_splitupdate
Open

andr-sokolov wants to merge 3 commits into
apache:mainfrom
andr-sokolov:check_splitupdate

Conversation

@andr-sokolov

@andr-sokolov andr-sokolov commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

When a partitioned table is distributed by key, and ALTER TABLE ... EXPAND
PARTITION PREPARE has been executed, and some partitions have already been
reorganized, but some have not, the distribution of the partitions is different.
In this case we should use Split Update when partition key is updated because
the table rows may be moved across segments.

There is no need to use Split Update when the table is distributed randomly, so
the check for nattrs is added at the beginning of check_splitupdate.

The bug led to missing rows when joining tables. Queries to reproduce the bug
are added to the tests.

The bug report is got from PR warehouse-pg/warehouse-pg#391

@andr-sokolov
andr-sokolov marked this pull request as ready for review November 20, 2026 08:15
@tuhaihe

tuhaihe commented Nov 20, 2026

Copy link
Copy Markdown
Member

As I understand it, we have just got a bug report instead of copying their code. Is that right? If so, all looks good.

@andr-sokolov

Copy link
Copy Markdown
Collaborator Author

As I understand it, we have just got a bug report instead of copying their code. Is that right? If so, all looks good.

yes, their code is not copied

Comment thread src/test/regress/sql/bfv_planner.sql Resolved
Comment thread src/backend/optimizer/prep/preptlist.c Resolved
@Alena0704

Copy link
Copy Markdown
Collaborator

Hi, thanks for the fix! I tested commit 482abcb with the Postgres planner on a three-segment cluster with assertions enabled and found the following issues:

  1. A whole-row partition key causes an assertion failure, even without expansion.
postgres=# SET optimizer=off;
SET
postgres=# CREATE TABLE wr (a int, b int) DISTRIBUTED BY (a) PARTITION BY LIST ((wr));
CREATE TABLE
postgres=# CREATE TABLE wr1 PARTITION OF wr FOR VALUES IN ('(1,2)');
NOTICE:  table has parent, setting distribution columns to match parent table
CREATE TABLE
postgres=# CREATE TABLE wr2 PARTITION OF wr FOR VALUES IN ('(1,3)');
NOTICE:  table has parent, setting distribution columns to match parent table
CREATE TABLE
postgres=# EXPLAIN (COSTS OFF) UPDATE wr SET b=3;
FATAL:  Unexpected internal error (assert.c:50)
DETAIL:  ("attno > 0", File: "preptlist.c", Line: 644)
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
The connection to the server was lost. Attempting reset: Failed.
  1. A BEFORE UPDATE trigger on the second leaf partition is silently skipped.
postgres=# SET optimizer=off;
SET
postgres=# CREATE EXTENSION IF NOT EXISTS gp_debug_numsegments;
CREATE EXTENSION
postgres=# SELECT gp_debug_set_create_table_default_numsegments(2);
 gp_debug_set_create_table_default_numsegments 
-----------------------------------------------
 2
(1 row)

postgres=# CREATE TABLE tr (a int, b int, c int) DISTRIBUTED BY (a) PARTITION BY RANGE(b);
CREATE TABLE
postgres=# CREATE TABLE tr1 PARTITION OF tr FOR VALUES FROM (0) TO (10);
NOTICE:  table has parent, setting distribution columns to match parent table
CREATE TABLE
postgres=# CREATE TABLE tr2 PARTITION OF tr FOR VALUES FROM (10) TO (20);
NOTICE:  table has parent, setting distribution columns to match parent table
CREATE TABLE
postgres=# INSERT INTO tr SELECT i,15,0 FROM generate_series(1,20) i;
INSERT 0 20
postgres=# CREATE FUNCTION tr_before() RETURNS trigger AS $$ BEGIN NEW.c := 999; RETURN NEW; END $$ LANGUAGE plpgsql;
CREATE FUNCTION
postgres=# CREATE TRIGGER tr_before BEFORE UPDATE ON tr2 FOR EACH ROW EXECUTE FUNCTION tr_before();
CREATE TRIGGER
postgres=# ALTER TABLE tr EXPAND PARTITION PREPARE;
ALTER TABLE
postgres=# ALTER TABLE tr1 SET WITH (REORGANIZE=TRUE) DISTRIBUTED BY(a);
ALTER TABLE
postgres=# EXPLAIN (COSTS OFF) UPDATE tr SET b=b+1;
                          QUERY PLAN                           
---------------------------------------------------------------
 Update on tr
   Update on tr1 tr_1
   Update on tr2 tr_2
   ->  Explicit Redistribute Motion 3:3  (slice1; segments: 3)
         ->  Split Update
               ->  Append
                     ->  Seq Scan on tr1 tr_1
                     ->  Seq Scan on tr2 tr_2
 Optimizer: Postgres query optimizer
(9 rows)

postgres=# UPDATE tr SET b=b+1;
UPDATE 20
postgres=# SELECT b,c,count(*) FROM tr GROUP BY b,c ORDER BY b,c;
 b  | c | count 
----+---+-------
 16 | 0 |    20 --should be 16 | 999 | 20
(1 row)
  1. MERGE reports 40 affected rows when it updates only 20.
postgres=# SET optimizer=off;
SET
postgres=# CREATE EXTENSION IF NOT EXISTS gp_debug_numsegments;
NOTICE:  extension "gp_debug_numsegments" already exists, skipping
CREATE EXTENSION
postgres=# SELECT gp_debug_set_create_table_default_numsegments(2);
 gp_debug_set_create_table_default_numsegments 
-----------------------------------------------
 2
(1 row)

postgres=# CREATE TABLE mg (a int, b int) DISTRIBUTED BY(a) PARTITION BY RANGE(b);
CREATE TABLE
postgres=# CREATE TABLE mg1 PARTITION OF mg FOR VALUES FROM(0) TO(10);
NOTICE:  table has parent, setting distribution columns to match parent table
CREATE TABLE
postgres=# CREATE TABLE mg2 PARTITION OF mg FOR VALUES FROM(10) TO(20);
NOTICE:  table has parent, setting distribution columns to match parent table
CREATE TABLE
postgres=# INSERT INTO mg SELECT i,5 FROM generate_series(1,20)i;
INSERT 0 20
postgres=# ALTER TABLE mg EXPAND PARTITION PREPARE;
ALTER TABLE
postgres=# ALTER TABLE mg2 SET WITH(REORGANIZE=TRUE) DISTRIBUTED BY(a);
ALTER TABLE
postgres=# SELECT gp_debug_set_create_table_default_numsegments(3);
 gp_debug_set_create_table_default_numsegments 
-----------------------------------------------
 3
(1 row)

postgres=# CREATE TABLE mg_ref AS SELECT generate_series(1,20)a DISTRIBUTED BY(a);
SELECT 20
postgres=# EXPLAIN(COSTS OFF) MERGE INTO mg USING mg_ref s ON mg.a=s.a WHEN MATCHED THEN UPDATE SET b=15;
                             QUERY PLAN                              
---------------------------------------------------------------------
 Merge on mg
   Merge on mg1 mg_1
   Merge on mg2 mg_2
   ->  Explicit Redistribute Motion 3:3  (slice1; segments: 3)
         ->  Split Merge
               ->  Hash Join
                     Hash Cond: (s.a = mg.a)
                     ->  Broadcast Motion 3:3  (slice2; segments: 3)
                           ->  Seq Scan on mg_ref s
                     ->  Hash
                           ->  Append
                                 ->  Seq Scan on mg1 mg_1
                                 ->  Seq Scan on mg2 mg_2
 Optimizer: Postgres query optimizer
(14 rows)

postgres=# MERGE INTO mg USING mg_ref s ON mg.a=s.a WHEN MATCHED THEN UPDATE SET b=15;
MERGE 40 --should be 20
postgres=# SELECT count(*) FROM mg2 JOIN mg_ref USING(a);
 count 
-------
    20 
(1 row)

The MERGE updates 20 rows, but reports MERGE 40 instead of MERGE 20. The subsequent SELECT correctly returns 20. It looks like both the DELETE and INSERT halves of each split UPDATE are counted.

  1. Updating a non-partition-key column acquires an additional AccessShareLock on a partition excluded by pruning, even without expansion.
postgres=# SET optimizer=off;
SET
postgres=# 
postgres=# CREATE TABLE lk(a int,b int,c int)
postgres-#   DISTRIBUTED BY(a) PARTITION BY RANGE(b);
CREATE TABLE
postgres=# CREATE TABLE lk1 PARTITION OF lk FOR VALUES FROM(0) TO(10);
NOTICE:  table has parent, setting distribution columns to match parent table
CREATE TABLE
postgres=# CREATE TABLE lk2 PARTITION OF lk FOR VALUES FROM(10) TO(20);
NOTICE:  table has parent, setting distribution columns to match parent table
CREATE TABLE
postgres=# INSERT INTO lk VALUES(1,5,0);
INSERT 0 1
postgres=# 
postgres=# BEGIN;
BEGIN
postgres=*# EXPLAIN(COSTS OFF) UPDATE lk SET c=1 WHERE b=5;
             QUERY PLAN              
-------------------------------------
 Update on lk
   Update on lk1 lk_1
   ->  Seq Scan on lk1 lk_1
         Filter: (b = 5)
 Optimizer: Postgres query optimizer
(5 rows)

postgres=*# 
postgres=*# SELECT c.relname,l.mode
postgres-*# FROM pg_locks l JOIN pg_class c ON c.oid=l.relation
postgres-*# WHERE l.pid=pg_backend_pid()
postgres-*#   AND c.relname IN('lk','lk1','lk2')
postgres-*# ORDER BY 1,2;
 relname |      mode       
---------+-----------------
 lk      | ExclusiveLock
 lk1     | AccessShareLock
 lk1     | ExclusiveLock
 lk2     | AccessShareLock --should not be locked
(4 rows)

postgres=*# ROLLBACK;
ROLLBACK

This additional lock is held until the transaction ends and can conflict with an ACCESS EXCLUSIVE lock on the pruned partition.

@Alena0704
Alena0704 self-requested a review November 20, 2026 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants