Search before asking
Paimon version
paimon 1.4.1
Compute Engine
flink 1.17
Minimal reproduce step
I am using the Dedicated Compaction feature to perform independent compaction. Since my table has multiple partitions, I initiated a separate compaction task for each partition. However, the tasks encountered errors during execution and kept restarting
Dedicated Compaction Flink Action Jar :
compact --warehouse hdfs://rzhdfs/paimon/rz --database rz_dwd --table dwd_cj_post_info --table_conf sink.parallelism=32 --partition platform=news --table_conf write-buffer-spillable=true --table_conf consumer-id=post_info_news_compaction --table_conf read.batch-size=256 --table_conf sink.use-managed-memory-allocator=true
What doesn't meet your expectations?
here is the error message:
2026-09-07 16:14:04
java.lang.RuntimeException: java.lang.IllegalArgumentException: Self-suppression not permitted
at org.apache.paimon.table.sink.TableCommitImpl.maintain(TableCommitImpl.java:362)
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:244)
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:101)
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:215)
at org.apache.paimon.flink.sink.CommitterOperator.notifyCheckpointComplete(CommitterOperator.java:192)
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.notifyCheckpointComplete(StreamOperatorWrapper.java:104)
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.notifyCheckpointComplete(RegularOperatorChain.java:145)
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.notifyCheckpoint(SubtaskCheckpointCoordinatorImpl.java:467)
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.notifyCheckpointComplete(SubtaskCheckpointCoordinatorImpl.java:400)
at org.apache.flink.streaming.runtime.tasks.StreamTask.notifyCheckpointComplete(StreamTask.java:1430)
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$notifyCheckpointCompleteAsync$16(StreamTask.java:1371)
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$notifyCheckpointOperation$19(StreamTask.java:1410)
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsWhenDefaultActionUnavailable(MailboxProcessor.java:367)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:352)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.IllegalArgumentException: Self-suppression not permitted
at java.lang.Throwable.addSuppressed(Throwable.java:1082)
at org.apache.paimon.fs.FileIO.overwriteFileUtf8(FileIO.java:362)
at org.apache.paimon.fs.hadoop.HadoopFileIO.overwriteFileUtf8(HadoopFileIO.java:177)
at org.apache.paimon.fs.FileIO.overwriteHintFile(FileIO.java:371)
at org.apache.paimon.utils.HintFileUtils.commitHint(HintFileUtils.java:117)
at org.apache.paimon.utils.HintFileUtils.commitEarliestHint(HintFileUtils.java:98)
at org.apache.paimon.utils.SnapshotManager.commitEarliestHint(SnapshotManager.java:741)
at org.apache.paimon.table.ExpireSnapshotsImpl.writeEarliestHint(ExpireSnapshotsImpl.java:315)
at org.apache.paimon.table.ExpireSnapshotsImpl.innerExpireUntil(ExpireSnapshotsImpl.java:270)
at org.apache.paimon.table.ExpireSnapshotsImpl.expireUntil(ExpireSnapshotsImpl.java:157)
at org.apache.paimon.table.ExpireSnapshotsImpl.expire(ExpireSnapshotsImpl.java:151)
at org.apache.paimon.table.AbstractFileStoreTable.lambda$newExpireRunnable$2(AbstractFileStoreTable.java:501)
at org.apache.paimon.table.sink.TableCommitImpl.maintain(TableCommitImpl.java:404)
at org.apache.paimon.table.sink.TableCommitImpl.lambda$maintain$4(TableCommitImpl.java:371)
at org.apache.paimon.shade.guava30.com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:321)
at org.apache.paimon.table.sink.TableCommitImpl.maintain(TableCommitImpl.java:368)
... 24 more
Caused by: java.io.FileNotFoundException: File does not exist: /paimon/rz/rz_dwd.db/dwd_cj_post_info/snapshot/EARLIEST (inode 1972126668) [Lease. Holder: DFSClient_NONMAPREDUCE_856901003_87, pending creates: 2]
Anything else?
No response
Are you willing to submit a PR?
Search before asking
Paimon version
paimon 1.4.1
Compute Engine
flink 1.17
Minimal reproduce step
I am using the Dedicated Compaction feature to perform independent compaction. Since my table has multiple partitions, I initiated a separate compaction task for each partition. However, the tasks encountered errors during execution and kept restarting
Dedicated Compaction Flink Action Jar :
compact --warehouse hdfs://rzhdfs/paimon/rz --database rz_dwd --table dwd_cj_post_info --table_conf sink.parallelism=32 --partition platform=news --table_conf write-buffer-spillable=true --table_conf consumer-id=post_info_news_compaction --table_conf read.batch-size=256 --table_conf sink.use-managed-memory-allocator=true
What doesn't meet your expectations?
here is the error message:
2026-09-07 16:14:04
java.lang.RuntimeException: java.lang.IllegalArgumentException: Self-suppression not permitted
at org.apache.paimon.table.sink.TableCommitImpl.maintain(TableCommitImpl.java:362)
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:244)
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:101)
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:215)
at org.apache.paimon.flink.sink.CommitterOperator.notifyCheckpointComplete(CommitterOperator.java:192)
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.notifyCheckpointComplete(StreamOperatorWrapper.java:104)
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.notifyCheckpointComplete(RegularOperatorChain.java:145)
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.notifyCheckpoint(SubtaskCheckpointCoordinatorImpl.java:467)
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.notifyCheckpointComplete(SubtaskCheckpointCoordinatorImpl.java:400)
at org.apache.flink.streaming.runtime.tasks.StreamTask.notifyCheckpointComplete(StreamTask.java:1430)
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$notifyCheckpointCompleteAsync$16(StreamTask.java:1371)
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$notifyCheckpointOperation$19(StreamTask.java:1410)
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsWhenDefaultActionUnavailable(MailboxProcessor.java:367)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:352)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.IllegalArgumentException: Self-suppression not permitted
at java.lang.Throwable.addSuppressed(Throwable.java:1082)
at org.apache.paimon.fs.FileIO.overwriteFileUtf8(FileIO.java:362)
at org.apache.paimon.fs.hadoop.HadoopFileIO.overwriteFileUtf8(HadoopFileIO.java:177)
at org.apache.paimon.fs.FileIO.overwriteHintFile(FileIO.java:371)
at org.apache.paimon.utils.HintFileUtils.commitHint(HintFileUtils.java:117)
at org.apache.paimon.utils.HintFileUtils.commitEarliestHint(HintFileUtils.java:98)
at org.apache.paimon.utils.SnapshotManager.commitEarliestHint(SnapshotManager.java:741)
at org.apache.paimon.table.ExpireSnapshotsImpl.writeEarliestHint(ExpireSnapshotsImpl.java:315)
at org.apache.paimon.table.ExpireSnapshotsImpl.innerExpireUntil(ExpireSnapshotsImpl.java:270)
at org.apache.paimon.table.ExpireSnapshotsImpl.expireUntil(ExpireSnapshotsImpl.java:157)
at org.apache.paimon.table.ExpireSnapshotsImpl.expire(ExpireSnapshotsImpl.java:151)
at org.apache.paimon.table.AbstractFileStoreTable.lambda$newExpireRunnable$2(AbstractFileStoreTable.java:501)
at org.apache.paimon.table.sink.TableCommitImpl.maintain(TableCommitImpl.java:404)
at org.apache.paimon.table.sink.TableCommitImpl.lambda$maintain$4(TableCommitImpl.java:371)
at org.apache.paimon.shade.guava30.com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:321)
at org.apache.paimon.table.sink.TableCommitImpl.maintain(TableCommitImpl.java:368)
... 24 more
Caused by: java.io.FileNotFoundException: File does not exist: /paimon/rz/rz_dwd.db/dwd_cj_post_info/snapshot/EARLIEST (inode 1972126668) [Lease. Holder: DFSClient_NONMAPREDUCE_856901003_87, pending creates: 2]
Anything else?
No response
Are you willing to submit a PR?