Skip to content

[Bug]: Intermittent empty-body HTTP 500s -- "NoMethodError: undefined method 'tracestate' for an instance of OpenTelemetry::Context" #513

Description

@reevejd

Problem Description

A Rails app using Puma and the instana gem intermittently returns empty-body 500s. The failing requests never reach application controllers and are not recorded in Instana, and the 500 response carries none of the usual headers (x-request-id, x-runtime, x-instana-*). I think this is a sibling issue of #507.

I've opened a PR for what I think is the fix here: #514.

In the meantime I've applied this monkey patch to mitigate the issue:

if defined?(Instana::Trace::Samplers)
  module Instana
    module Trace
      module Samplers
        def self.should_sample?(trace_id:, parent_context:, links:, name:, kind:, attributes:)
          tracestate =
            begin
              span_context = OpenTelemetry::Trace.current_span(parent_context).context
              if span_context.respond_to?(:tracestate)
                span_context.tracestate
              else
                OpenTelemetry::Trace::Tracestate::DEFAULT
              end
            rescue StandardError
              OpenTelemetry::Trace::Tracestate::DEFAULT
            end

          Result.new(decision: :__record_only__, tracestate: tracestate)
        end
      end
    end
  end
end

Minimal, Complete, Verifiable, Example

require "instana"

# A non-recording span built from an OpenTelemetry::Context (as start_span
# produces while the agent is not ready) exposes that Context as its #context.
poisoned_parent = OpenTelemetry::Trace.context_with_span(
  Instana::Trace.non_recording_span(OpenTelemetry::Context.empty),
  parent_context: OpenTelemetry::Context.empty
)

Instana::Trace::Samplers.should_sample?(
  trace_id: "t", parent_context: poisoned_parent,
  links: nil, name: "rack", kind: :server, attributes: nil
)
# => NoMethodError: undefined method 'tracestate' for an instance of OpenTelemetry::Context
#      .../instana/samplers/samplers.rb:71:in 'should_sample?'

Gemfile.lock

GEM
  remote: https://rubygems.org/
  specs:
    base64 (0.3.0)
    bigdecimal (4.1.2)
    cgi (0.5.2)
    concurrent-ruby (1.3.7)
    csv (3.3.5)
    ffi (1.17.4)
    instana (2.7.1)
      base64 (>= 0.1)
      cgi
      concurrent-ruby (>= 1.1)
      csv (>= 0.1)
      logger
      oj (>= 3.0.11)
      opentelemetry-api (~> 1.4)
      opentelemetry-common
      sys-proctable (>= 1.2.2)
    logger (1.7.0)
    oj (3.17.3)
      bigdecimal (>= 3.0)
      ostruct (>= 0.2)
    opentelemetry-api (1.9.0)
      logger
    opentelemetry-common (0.24.0)
      opentelemetry-api (~> 1.0)
    ostruct (0.6.3)
    rack (3.2.6)
    sys-proctable (1.3.0)
      ffi (~> 1.1)

PLATFORMS
  ruby
  x86_64-linux-gnu

DEPENDENCIES
  instana (= 2.7.1)
  opentelemetry-api (= 1.9.0)
  opentelemetry-common (= 0.24.0)
  rack (~> 3.0)

BUNDLED WITH
   2.6.9

Ruby Version

3.4.7

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions