Python not catching root span_id's originating from a JavaScript front-end, so distributed tracing doesn't quite work about honeycomb-opentelemetry-python

[robbkidd]

As a side note, @ericnograles, thank you for providing such thorough information in your issues! They are like exemplars for how to give enough information for someone else to help! 💞

[robbkidd]

Hi there, @ericnograles!

My first suspect is propagation header format.

I notice that JS frontend is configured in code to propagate trace information with B3 headers.

// setup Propagator
const propagator = new otelCore.CompositePropagator({
  propagators: [
    new B3Propagator(),
    new B3Propagator({ injectEncoding: B3InjectEncoding.MULTI_HEADER })
  ]
})
provider.register({
  contextManager: new ZoneContextManager(),
  propagator
});

If your Python backend is not also configured for B3 propagation—which is not the default; propagation format defaults to W3C an OTel SDK—then that would explain this behavior. Both services emit spans, but the backend starts a new trace for its spans because it ignores the B3 header on the incoming traffic from the frontend.

The options I see:

• Configure the Python backend to propagate tracing in-and-out in B3 format. Depending on how you are configuring OpenTelemetry in the backend, this could be as simple as setting the OTEL_PROPAGATORS environment variable to "b3" .
• Configure the JS frontend to propagate with W3C format, unless you have some specific reasons for using B3 like communications with other components in which B3 is the recommended or only supported format. W3C is the default in an OTel SDK, so this option might be a matter of removing the propagator construction and its use in provider.register().

[ericnograles]

Too kind @robbkidd, glad to be helpful!

I did remove the propagator from the construction on the JS side and unfortunately still no dice (see screenshot below).

Web App

Flask Backend

Logs

I did manage to copy the log of the HTTP call going to Honeycomb. Interestingly, as you see, the parent_id doesn't seem to get set. Can you spot anything here that might hint at what I may need to do?

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.honeycomb.io:443
{
    "name": "/packets",
    "context": {
        "trace_id": "0xd7504147033ab9c111149b73a0167f3f",
        "span_id": "0x3bdd746aa7281e2e",
        "trace_state": "[]"
    },
    "kind": "SpanKind.SERVER",
    "parent_id": null,
    "start_time": "2023-05-31T14:41:37.882378Z",
    "end_time": "2023-05-31T14:41:37.997296Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "SampleRate": 1,
        "http.method": "GET",
        "http.server_name": "127.0.0.1",
        "http.scheme": "https",
        "net.host.port": 5000,
        "http.host": "127.0.0.1:5000",
        "http.target": "/packets",
        "net.peer.ip": "127.0.0.1",
        "http.user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36",
        "net.peer.port": 53114,
        "http.flavor": "1.1",
        "http.route": "/packets",
        "http.status_code": 200
    },
    "events": [],
    "links": [],
    "resource": {
        "attributes": {
            "telemetry.sdk.language": "python",
            "telemetry.sdk.name": "opentelemetry",
            "telemetry.sdk.version": "1.17.0",
            "service.name": "diffit-api-local",
            "honeycomb.distro.version": "0.2.1b0",
            "honeycomb.distro.runtime_version": "3.10.4"
        },
        "schema_url": ""
    }
}

[robbkidd]

@ericnograles parent_id not set is still an indicator that propagation is not aligned between your frontend and backend.

Some things I can think to look at:

1. Can you see the headers on HTTP requests received by your backend? If the frontend is correctly configured to propagate W3C trace context, you should see traceparent headers on requests from your frontend.
2. Can you share your updated JS OTel config?
3. Can you share your Python backend's OTel config as well? And any OTEL_* environment variables you may have set there.
   • ⚠️ REDACT any secrets you find in the values for these variables before sharing.

[github-actions]

Marking this issue as stale because it has been open 14 days with no activity. Please add a comment if this is still an ongoing issue; otherwise this issue will be automatically closed in 7 days.

[github-actions]

Closing this issue due to inactivity. Please see our Honeycomb OSS Lifecyle and Practices.