You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ParquetWriterExec's derived Debug prints object store options, which can hold secret keys #7558
Found while working on #7128. ParquetWriterExec in native/core/src/execution/operators/parquet_writer.rs derives Debug, so {:?} of the operator prints output_path and object_store_options as they are. The options come from NativeConfig.extractObjectStoreOptions and can hold credentials such as fs.s3a.secret.key or an Azure SAS token, and the path can carry a password or a SAS query.
Comet does not format the plan with {:?} today, so nothing leaks yet. A debug log, a dbg!, a test assertion or a DataFusion error that formats the plan with Debug would print the secrets.
Steps to reproduce
Format a ParquetWriterExec built with fs.s3a.secret.key in its options with {:?}. The key's value appears in the output.
Expected behavior
Debug for ParquetWriterExec prints the path without credentials (the helpers #7127 adds) and only the keys of object_store_options, not their values.
Additional context
The plan display (DisplayAs) and the writer's messages are handled in the #7128 pull requests; this is the remaining way the operator can print its options.
Activity