Skip to content

[Bug] Dedicated Compaction log error Self-suppression not permitted #9674

Description

@maxinfan

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

paimon 1.4.1

Compute Engine

flink 1.17

Minimal reproduce step

I am using the Dedicated Compaction feature to perform independent compaction. Since my table has multiple partitions, I initiated a separate compaction task for each partition. However, the tasks encountered errors during execution and kept restarting

Dedicated Compaction Flink Action Jar :
compact --warehouse hdfs://rzhdfs/paimon/rz --database rz_dwd --table dwd_cj_post_info --table_conf sink.parallelism=32 --partition platform=news --table_conf write-buffer-spillable=true --table_conf consumer-id=post_info_news_compaction --table_conf read.batch-size=256 --table_conf sink.use-managed-memory-allocator=true

What doesn't meet your expectations?

here is the error message:
2026-09-07 16:14:04
java.lang.RuntimeException: java.lang.IllegalArgumentException: Self-suppression not permitted
at org.apache.paimon.table.sink.TableCommitImpl.maintain(TableCommitImpl.java:362)
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:244)
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:101)
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:215)
at org.apache.paimon.flink.sink.CommitterOperator.notifyCheckpointComplete(CommitterOperator.java:192)
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.notifyCheckpointComplete(StreamOperatorWrapper.java:104)
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.notifyCheckpointComplete(RegularOperatorChain.java:145)
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.notifyCheckpoint(SubtaskCheckpointCoordinatorImpl.java:467)
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.notifyCheckpointComplete(SubtaskCheckpointCoordinatorImpl.java:400)
at org.apache.flink.streaming.runtime.tasks.StreamTask.notifyCheckpointComplete(StreamTask.java:1430)
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$notifyCheckpointCompleteAsync$16(StreamTask.java:1371)
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$notifyCheckpointOperation$19(StreamTask.java:1410)
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsWhenDefaultActionUnavailable(MailboxProcessor.java:367)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:352)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.IllegalArgumentException: Self-suppression not permitted
at java.lang.Throwable.addSuppressed(Throwable.java:1082)
at org.apache.paimon.fs.FileIO.overwriteFileUtf8(FileIO.java:362)
at org.apache.paimon.fs.hadoop.HadoopFileIO.overwriteFileUtf8(HadoopFileIO.java:177)
at org.apache.paimon.fs.FileIO.overwriteHintFile(FileIO.java:371)
at org.apache.paimon.utils.HintFileUtils.commitHint(HintFileUtils.java:117)
at org.apache.paimon.utils.HintFileUtils.commitEarliestHint(HintFileUtils.java:98)
at org.apache.paimon.utils.SnapshotManager.commitEarliestHint(SnapshotManager.java:741)
at org.apache.paimon.table.ExpireSnapshotsImpl.writeEarliestHint(ExpireSnapshotsImpl.java:315)
at org.apache.paimon.table.ExpireSnapshotsImpl.innerExpireUntil(ExpireSnapshotsImpl.java:270)
at org.apache.paimon.table.ExpireSnapshotsImpl.expireUntil(ExpireSnapshotsImpl.java:157)
at org.apache.paimon.table.ExpireSnapshotsImpl.expire(ExpireSnapshotsImpl.java:151)
at org.apache.paimon.table.AbstractFileStoreTable.lambda$newExpireRunnable$2(AbstractFileStoreTable.java:501)
at org.apache.paimon.table.sink.TableCommitImpl.maintain(TableCommitImpl.java:404)
at org.apache.paimon.table.sink.TableCommitImpl.lambda$maintain$4(TableCommitImpl.java:371)
at org.apache.paimon.shade.guava30.com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:321)
at org.apache.paimon.table.sink.TableCommitImpl.maintain(TableCommitImpl.java:368)
... 24 more
Caused by: java.io.FileNotFoundException: File does not exist: /paimon/rz/rz_dwd.db/dwd_cj_post_info/snapshot/EARLIEST (inode 1972126668) [Lease. Holder: DFSClient_NONMAPREDUCE_856901003_87, pending creates: 2]

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions