Add configuration of log level and formatting for Auto Instrumentation

[jeremydvoss]

Description

Proposal for adding configuration of log level and formatting for Auto Instrumentation. Main subject to decide on is which env vars to use:
OTEL_LOG_LEVEL: Exists in the SDK but unused. Speced out to be the level for the SDK's internal logger, not the LoggingHandler.
OTEL_PYTHON_LOG_LEVEL: Currently only used by Python Logging Instrumentation
OTEL_PYTHON_LOG_FORMAT: Currently only used by Python Logging Instrumentation
OTEL_PYTHON_LOG_HANDLER_LEVEL: Possible new env var that is clearer and does not map to the same --log_level flag as OTEL_LOG_LEVEL

Fixes #4034

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• Test A

Does This PR Require a Contrib Repo Change?

• Yes. - Link to PR:
• No.

Checklist:

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[roger-collins-self]

According to the specification, OTEL_LOG_LEVEL is to be used for the SDK loggers, not the instrumented OTLP loggers.

The logging config should only apply to the opentelemetry logger namespace and not the root logger.

[jeremydvoss]

According to the specification, OTEL_LOG_LEVEL is to be used for the SDK loggers, not the instrumented OTLP loggers.

The logging config should only apply to the opentelemetry logger namespace and not the root logger.

I see your point. I'll bring up in the next sig. Since this is a broader matter, I have also made an issue to make sure we tackle this before logging stability.

[jeremydvoss]

Added tests.

[jeremydvoss]

@roger-collins-self I switched to using OTEL_PYTHON_LOG_LEVEL instead of OTEL_LOG_LEVEL. This must by why the logging instrumentation does the same.

[jeremydvoss]

Looks like there's some issue with Python 3.8. Will investigate.

[jeremydvoss]

I don't understand this error. How might it be related to this change?

https://github.com/open-telemetry/opentelemetry-python/actions/runs/15004918710/job/42163726837#step:5:408

/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/argparse.py:1548: ArgumentError
____________________________ TestArgs.test_exporter ____________________________

self = <test_run.TestArgs testMethod=test_exporter>
_ = <MagicMock name='execl' id='139772420527680'>

    @patch("opentelemetry.instrumentation.auto_instrumentation.execl")
    def test_exporter(self, _):  # pylint: disable=no-self-use
        with patch("sys.argv", ["instrument", "2"]):
>           auto_instrumentation.run()

opentelemetry-instrumentation/tests/auto_instrumentation/test_run.py:108: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
opentelemetry-instrumentation/src/opentelemetry/instrumentation/auto_instrumentation/__init__.py:65: in run
    parser.add_argument(
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/argparse.py:1386: in add_argument
    return self._add_action(action)
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/argparse.py:1749: in _add_action
    self._optionals._add_action(action)
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/argparse.py:1590: in _add_action
    action = super(_ArgumentGroup, self)._add_action(action)
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/argparse.py:1400: in _add_action
    self._check_conflict(action)
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/argparse.py:1539: in _check_conflict
    conflict_handler(action, confl_optionals)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <argparse._ArgumentGroup object at 0x7f1f4d7c3fd0>
action = _StoreAction(option_strings=['--log_level'], dest='log_level', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
conflicting_actions = [('--log_level', _StoreAction(option_strings=['--log_level'], dest='log_level', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None))]

    def _handle_conflict_error(self, action, conflicting_actions):
        message = ngettext('conflicting option string: %s',
                           'conflicting option strings: %s',
                           len(conflicting_actions))
        conflict_string = ', '.join([option_string
                                     for option_string, action
                                     in conflicting_actions])
>       raise ArgumentError(action, message % conflict_string)
E       argparse.ArgumentError: argument --log_level: conflicting option string: --log_level

[mzealey]

https://markandruth.co.uk/2025/05/30/getting-opentelemetry-logging-working-in-python might be useful for some gotchas and workarounds while this PR is not yet merged.

[JWinermaSplunk]

Hi @jeremydvoss

I am looking to get some of this functionality that you previously worked on merged. I was hoping that using some of your previous changes in #4921 is ok?

Thank you.