Skip to content

Conversation

@kevinjqliu
Copy link
Contributor

@kevinjqliu kevinjqliu commented Dec 28, 2025

Rationale for this change

I noticed we dump the entire error traceback in log warn messages. This causes a wall of text when using the CLI.

This PR conditionally dump traceback only for debug log level. This provides cleaner user-facing output by default while still making detailed exception info available for troubleshooting via debug logging.

  • Users now see clean one-line warnings instead of full tracebacks.
  • Developers can enable debug logging to see detailed exception info.

Using the shorthand, exc_info in logging.warning's kwargs:
https://docs.python.org/3/library/logging.html#:~:text=If%20exc_info%20does%20not%20evaluate%20as%20false%2C%20it%20causes%20exception%20information%20to%20be%20added%20to%20the%20logging%20message.%20If%20an%20exception%20tuple%20(in%20the%20format%20returned%20by%20sys.exc_info())%20or%20an%20exception%20instance%20is%20provided%2C%20it%20is%20used%3B%20otherwise%2C%20sys.exc_info()%20is%20called%20to%20get%20the%20exception%20information.

Are these changes tested?

Are there any user-facing changes?

Before

Could not initialize FileIO: pyiceberg.io.pyarrow.PyArrowFileIO
Traceback (most recent call last):
  File "/Users/kevinliu/repos/iceberg-python/pyiceberg/io/__init__.py", line 321, in _import_file_io
    module = importlib.import_module(module_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kevinliu/.pyenv/versions/3.12.11/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 999, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/Users/kevinliu/repos/iceberg-python/pyiceberg/io/pyarrow.py", line 54, in <module>
    import pyarrow as pa
ModuleNotFoundError: No module named 'pyarrow'

After

Could not initialize FileIO: pyiceberg.io.pyarrow.PyArrowFileIO

Move exception info from warning to debug level in import error handlers.
This provides cleaner user-facing output by default while still making
detailed exception info available for troubleshooting via debug logging.

Changes:
- FileIO import failures (pyiceberg.io.__init__)
- LocationProvider import failures (pyiceberg.table.locations)
- Catalog import failures (pyiceberg.catalog.__init__)
- File deletion failures (pyiceberg.catalog.__init__)

Users now see clean one-line warnings instead of full tracebacks.
Developers can enable debug logging to see detailed exception info.
Copy link
Contributor

@Fokko Fokko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable, thanks @kevinjqliu for working on this, and thanks @nssalian & @geruh for the review 🚀

@Fokko Fokko merged commit 7b84d10 into apache:main Jan 6, 2026
8 checks passed
Fokko pushed a commit that referenced this pull request Jan 6, 2026
<!--
Thanks for opening a pull request!
-->

<!-- In the case this PR will resolve an issue, please replace
${GITHUB_ISSUE_ID} below with the actual Github issue id. -->
<!-- Closes #${GITHUB_ISSUE_ID} -->

# Rationale for this change
After #2867, I realized that there's no way to set log level for the
CLI.
This PR introduces 2 ways to set log levels, `--log-level` and
`PYICEBERG_LOG_LEVEL`. Default log level is `WARNING`

## Are these changes tested?
Yes

## Are there any user-facing changes?
Yes

<!-- In the case of user-facing changes, please add the changelog label.
-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants