[Bug] Paimon Native Scanner returns FORBIDDEN (403) on specific tables when using private S3-compatible storage, JNI scanner works fine

[Jerome786]

Search before asking

• I had searched in the issues and found no similar issues.

Version

doris-4.0.5

What's Wrong?

When querying Paimon tables via an External Catalog backed by a private S3-compatible storage, some tables consistently return AccessDenied / FORBIDDEN when read by the Paimon Native Scanner. Switching to the JNI scanner resolves the issue immediately.

The problem is table-specific within the same catalog and schema: tables pay_manager, relation, term succeed with native scanner, while ods_pxf_db_cm_contract_pay_info_fd always fails.

What You Expected?

fix it in next version

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[zclllyybb]

Breakwater-GitHub-Analysis-Slot: slot_f855cb78fc4c

Initial analysis:

I checked the Doris 4.0.5 Paimon scan path. This does not look like a general catalog credential failure, because the same catalog works through the JNI scanner and some tables also work with the native path.

The relevant difference is the reader path:

• With force_jni_scanner=false, Doris can convert a Paimon DataSplit to raw ORC/Parquet files and read them through the native file scanner. In that path, FE normalizes each raw file path through LocationPath and passes backend S3 properties such as endpoint, region, credentials/token, and use_path_style to BE. BE then reads the object through Doris' native S3 client.
• If the split has a Paimon deletion file/deletion vector, the native Paimon reader opens that deletion file separately through the same BE file reader before applying row filters.
• With the JNI scanner, Doris passes Hadoop-style properties to PaimonJniScanner, and Paimon Java reads the serialized Paimon split itself. That path does not exercise the same BE native S3 object-access path.

So the issue is actionable as a likely Paimon native-reader compatibility problem for this S3-compatible storage path, or a real object-permission difference on the specific raw/deletion files used by ods_pxf_db_cm_contract_pay_info_fd. The current issue is not enough to distinguish those two cases.

Useful next checks:

1. Please attach the BE log around the failure, including the full Doris error line with code=FORBIDDEN, request id, and the object path/key that failed. It is especially important to know whether the failed object is a data file or a Paimon deletion-vector/deletion file.
2. Please run EXPLAIN VERBOSE for the failing table with set force_jni_scanner=false and include the paimonNativeReadSplits, PaimonSplitStats, dataFileNum, deleteFileNum, and deleteSplitNum lines. The same output for one working table in the same catalog would make the difference clear.
3. Please provide the catalog properties with secrets masked, but keep endpoint, region, URI/warehouse, storage scheme (s3://, s3a://, etc.), use_path_style / s3.path-style-access, session-token usage, and any paimon.* or fs.s3a.* settings visible.
4. Please provide a minimal failing query, the exact build hash for doris-4.0.5, and whether this table has deletes/updates/deletion vectors while the working tables are append-only.

Temporary workaround: keep using set force_jni_scanner=true for the affected table until the native path can be confirmed and fixed.