Skip to content

BatchProcessor and PeriodicExportingMetricReader cause TypeError: 'NoneType' object is not callable on os.fork() after being garbage collected #5452

Description

@pranaysb

Describe your environment

OS: macOS
Python version: Python 3.12, 3.13 (tested); root cause is a plain os.register_at_fork + weakref interaction, not version-specific
SDK version: 1.44.0
API version: 1.44.0

What happened?

If a BatchProcessor or PeriodicExportingMetricReader is instantiated and subsequently garbage collected, any future calls to os.fork() will print a TypeError: 'NoneType' object is not callable stack trace to sys.stderr.
This happens because the after_in_child callbacks registered with os.register_at_fork in their __init__ methods do not check if the weak reference has died before calling it.

Steps to Reproduce

import os
import sys
from opentelemetry.sdk._shared_internal import BatchProcessor
class DummyExporter:
    def export(self, batch): pass
    def shutdown(self): pass
class DummyMetrics:
    def register_queue_size(self, callback): pass
    def drop_items(self, count): pass
    def finish_items(self, count, error): pass
def reproduce():
    processor = BatchProcessor(
        exporter=DummyExporter(),
        schedule_delay_millis=5000,
        max_export_batch_size=512,
        export_timeout_millis=30000,
        max_queue_size=2048,
        exporting="Span",
        metrics=DummyMetrics()
    )
    # Shutdown allows the worker thread to exit and the processor to be GC'd
    processor.shutdown()
    del processor
    
    pid = os.fork()
    if pid == 0:
        os._exit(0)
    else:
        os.waitpid(pid, 0)
reproduce()

Expected Result

The script executes cleanly without any errors printed to stderr, because the processor object has already been garbage collected.

Actual Result

Exception ignored in: <function BatchProcessor.__init__.<locals>.<lambda> at 0x104f5ab60>
Traceback (most recent call last):
  File "/opentelemetry-sdk/src/opentelemetry/sdk/_shared_internal/__init__.py", line 119, in <lambda>
    os.register_at_fork(after_in_child=lambda: weak_reinit()())  
TypeError: 'NoneType' object is not callable

Additional context

The fix is to safely check if the weak reference is alive before calling it, similar to how it is properly handled in ConcurrentMultiSpanProcessor in opentelemetry/sdk/trace/__init__.py. This prevents log spam in multi-process applications (like celery workers or gunicorn) that dynamically create and destroy pipelines.

Would you like to implement a fix?

Yes

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions