Problem is that if I'm trying to push traces from python code into otel collector, I get errors.
2021-10-09T03:01:33.665Z info service/service.go:411 Starting OpenTelemetry Collector... {"Version": "v0.22.0", "GitHash": "0345f0d2", "NumCPU": 2}
2021-10-09T03:01:33.669Z info service/service.go:593 Using memory ballast {"MiBs": 204}
2021-10-09T03:01:33.669Z info service/service.go:255 Setting up own telemetry...
2021-10-09T03:01:33.670Z info service/telemetry.go:102 Serving Prometheus metrics {"address": "0.0.0.0:8888", "level": 0, "service.instance.id": "be91d61f-ee6e-445a-9275-d9c4ac5343c6"}
2021-10-09T03:01:33.670Z info service/service.go:292 Loading configuration...
2021-10-09T03:01:33.672Z info service/service.go:303 Applying configuration...
2021-10-09T03:01:33.749Z info service/service.go:324 Starting extensions...
2021-10-09T03:01:33.749Z info builder/extensions_builder.go:53 Extension is starting... {"component_kind": "extension", "component_type": "health_check", "component_name": "health_check"}
2021-10-09T03:01:33.749Z info healthcheckextension/healthcheckextension.go:40 Starting health_check extension {"component_kind": "extension", "component_type": "health_check", "component_name": "health_check", "config": {"TypeVal":"health_check","NameVal":"health_check","Port":13133}}
2021-10-09T03:01:33.750Z info builder/extensions_builder.go:59 Extension started. {"component_kind": "extension", "component_type": "health_check", "component_name": "health_check"}
2021-10-09T03:01:33.750Z info builder/exporters_builder.go:302 Exporter is enabled. {"component_kind": "exporter", "exporter": "otlp"}
2021-10-09T03:01:33.750Z info builder/exporters_builder.go:302 Exporter is enabled. {"component_kind": "exporter", "exporter": "logging"}
2021-10-09T03:01:33.750Z info service/service.go:339 Starting exporters...
2021-10-09T03:01:33.750Z info builder/exporters_builder.go:92 Exporter is starting... {"component_kind": "exporter", "component_type": "otlp", "component_name": "otlp"}
2021-10-09T03:01:33.751Z info builder/exporters_builder.go:97 Exporter started. {"component_kind": "exporter", "component_type": "otlp", "component_name": "otlp"}
2021-10-09T03:01:33.752Z info builder/exporters_builder.go:92 Exporter is starting... {"component_kind": "exporter", "component_type": "logging", "component_name": "logging"}
2021-10-09T03:01:33.752Z info builder/exporters_builder.go:97 Exporter started. {"component_kind": "exporter", "component_type": "logging", "component_name": "logging"}
2021-10-09T03:01:33.752Z info memorylimiter/memorylimiter.go:108 Memory limiter configured {"component_kind": "processor", "component_type": "memory_limiter", "component_name": "memory_limiter", "limit_mib": 428867584, "spike_limit_mib": 134217728, "check_interval": 5}
2021-10-09T03:01:33.752Z info builder/pipelines_builder.go:203 Pipeline is enabled. {"pipeline_name": "logs", "pipeline_datatype": "logs"}
2021-10-09T03:01:33.752Z info memorylimiter/memorylimiter.go:108 Memory limiter configured {"component_kind": "processor", "component_type": "memory_limiter", "component_name": "memory_limiter", "limit_mib": 428867584, "spike_limit_mib": 134217728, "check_interval": 5}
2021-10-09T03:01:33.752Z info builder/pipelines_builder.go:203 Pipeline is enabled. {"pipeline_name": "traces", "pipeline_datatype": "traces"}
2021-10-09T03:01:33.752Z info service/service.go:352 Starting processors...
2021-10-09T03:01:33.752Z info builder/pipelines_builder.go:51 Pipeline is starting... {"pipeline_name": "logs", "pipeline_datatype": "logs"}
2021-10-09T03:01:33.752Z info builder/pipelines_builder.go:61 Pipeline is started. {"pipeline_name": "logs", "pipeline_datatype": "logs"}
2021-10-09T03:01:33.752Z info builder/pipelines_builder.go:51 Pipeline is starting... {"pipeline_name": "traces", "pipeline_datatype": "traces"}
2021-10-09T03:01:33.752Z info builder/pipelines_builder.go:61 Pipeline is started. {"pipeline_name": "traces", "pipeline_datatype": "traces"}
2021-10-09T03:01:33.752Z info builder/receivers_builder.go:230 Receiver is enabled. {"component_kind": "receiver", "component_type": "otlp", "component_name": "otlp", "datatype": "traces"}
2021-10-09T03:01:33.752Z info builder/receivers_builder.go:230 Receiver is enabled. {"component_kind": "receiver", "component_type": "otlp", "component_name": "otlp", "datatype": "logs"}
2021-10-09T03:01:33.752Z info builder/receivers_builder.go:105 Ignoring receiver as it is not used by any pipeline {"component_kind": "receiver", "component_type": "prometheus", "component_name": "prometheus", "receiver": "prometheus"}
2021-10-09T03:01:33.752Z info builder/receivers_builder.go:230 Receiver is enabled. {"component_kind": "receiver", "component_type": "zipkin", "component_name": "zipkin", "datatype": "traces"}
2021-10-09T03:01:33.752Z info builder/receivers_builder.go:230 Receiver is enabled. {"component_kind": "receiver", "component_type": "jaeger", "component_name": "jaeger", "datatype": "traces"}
2021-10-09T03:01:33.752Z info service/service.go:364 Starting receivers...
2021-10-09T03:01:33.752Z info builder/receivers_builder.go:70 Receiver is starting... {"component_kind": "receiver", "component_type": "otlp", "component_name": "otlp"}
2021-10-09T03:01:33.752Z info otlpreceiver/otlp.go:93 Starting GRPC server on endpoint 0.0.0.0:4317 {"component_kind": "receiver", "component_type": "otlp", "component_name": "otlp"}
2021-10-09T03:01:33.752Z info otlpreceiver/otlp.go:130 Setting up a second GRPC listener on legacy endpoint 0.0.0.0:55680 {"component_kind": "receiver", "component_type": "otlp", "component_name": "otlp"}
2021-10-09T03:01:33.752Z info otlpreceiver/otlp.go:93 Starting GRPC server on endpoint 0.0.0.0:55680 {"component_kind": "receiver", "component_type": "otlp", "component_name": "otlp"}
2021-10-09T03:01:33.752Z info otlpreceiver/otlp.go:108 Starting HTTP server on endpoint 0.0.0.0:55681 {"component_kind": "receiver", "component_type": "otlp", "component_name": "otlp"}
2021-10-09T03:01:33.752Z info builder/receivers_builder.go:75 Receiver started. {"component_kind": "receiver", "component_type": "otlp", "component_name": "otlp"}
2021-10-09T03:01:33.752Z info builder/receivers_builder.go:70 Receiver is starting... {"component_kind": "receiver", "component_type": "zipkin", "component_name": "zipkin"}
2021-10-09T03:01:33.752Z info builder/receivers_builder.go:75 Receiver started. {"component_kind": "receiver", "component_type": "zipkin", "component_name": "zipkin"}
2021-10-09T03:01:33.752Z info builder/receivers_builder.go:70 Receiver is starting... {"component_kind": "receiver", "component_type": "jaeger", "component_name": "jaeger"}
2021-10-09T03:01:33.752Z info static/strategy_store.go:201 No sampling strategies provided or URL is unavailable, using defaults {"component_kind": "receiver", "component_type": "jaeger", "component_name": "jaeger"}
2021-10-09T03:01:33.752Z info builder/receivers_builder.go:75 Receiver started. {"component_kind": "receiver", "component_type": "jaeger", "component_name": "jaeger"}
2021-10-09T03:01:33.752Z info healthcheck/handler.go:128 Health Check state change {"component_kind": "extension", "component_type": "health_check", "component_name": "health_check", "status": "ready"}
2021-10-09T03:01:33.752Z info service/service.go:267 Everything is ready. Begin running and processing data.
2021-10-09T03:05:52.417Z info exporterhelper/queued_retry.go:276 Exporting failed. Will retry the request after interval. {"component_kind": "exporter", "component_type": "otlp", "component_name": "otlp", "error": "failed to push trace data via OTLP exporter: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"", "interval": "4.615761008s"}
config:
exporters:
otlp:
endpoint: tempo.monitoring:4317
service:
pipelines:
metrics: null
traces:
exporters:
- otlp
receivers:
jaeger:
protocols:
grpc:
endpoint: 0.0.0.0:14250
thrift_http:
endpoint: 0.0.0.0:14268
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:55681
nodeSelector:
apps: "true"
standaloneCollector:
enabled: false
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import (
OTLPSpanExporter,
)
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import (
BatchSpanProcessor,
ConsoleSpanExporter,
)
span_exporter = OTLPSpanExporter(
# endpoint="tempo.monitoring:4317",
endpoint="10.120.4.103:4317",
insecure=True
)
provider = TracerProvider()
processor = BatchSpanProcessor(ConsoleSpanExporter())
span_processor = BatchSpanProcessor(span_exporter)
provider.add_span_processor(processor)
provider.add_span_processor(span_processor)
trace.set_tracer_provider(provider)
tracer = trace.get_tracer(__name__)
with tracer.start_as_current_span("foo"):
with tracer.start_as_current_span("bar"):
with tracer.start_as_current_span("baz"):
print("Hello world from OpenTelemetry Python!")
span_exporter = OTLPSpanExporter(
endpoint="tempo.monitoring:4317",
insecure=True
)
curl tempo.monitoring:3100/api/traces/695df1582064293a3be4fbac4285c779
{"batches":[{"resource":{"attributes":[{"key":"telemetry.sdk.language","value":{"stringValue":"python"}},{"key":"telemetry.sdk.name","value":{"stringValue":"opentelemetry"}},{"key":"telemetry.sdk.version","value":{"stringValue":"1.0.0"}},{"key":"service.name","value":{"stringValue":"unknown_service"}}]},"instrumentationLibrarySpans":[{"instrumentationLibrary":{"name":"__main__"},"spans":[{"traceId":"aV3xWCBkKTo75PusQoXHeQ==","spanId":"heVf3cr2TEk=","parentSpanId":"2JXZSaNcNVk=","name":"baz","kind":"SPAN_KIND_INTERNAL","startTimeUnixNano":"1633749090560979667","endTimeUnixNano":"1633749090561020135","status":{}},{"traceId":"aV3xWCBkKTo75PusQoXHeQ==","spanId":"2JXZSaNcNVk=","parentSpanId":"VyXEDPYNvOc=","name":"bar","kind":"SPAN_KIND_INTERNAL","startTimeUnixNano":"1633749090560949791","endTimeUnixNano":"1633749090561049979","status":{}},{"traceId":"aV3xWCBkKTo75PusQoXHeQ==","spanId":"VyXEDPYNvOc=","name":"foo","kind":"SPAN_KIND_INTERNAL","startTimeUnixNano":"1633749090560726098","endTimeUnixNano
kgcm opentelemetry-collector-agent -o yaml
apiVersion: v1
data:
relay: |
exporters:
logging: {}
otlp:
endpoint: tempo.monitoring:4317
extensions:
health_check: {}
processors:
batch: {}
memory_limiter:
ballast_size_mib: 204
check_interval: 5s
limit_mib: 409
spike_limit_mib: 128
receivers:
jaeger:
protocols:
grpc:
endpoint: 0.0.0.0:14250
thrift_http:
endpoint: 0.0.0.0:14268
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:55681
prometheus:
config:
scrape_configs:
- job_name: opentelemetry-collector
scrape_interval: 10s
static_configs:
- targets:
- ${MY_POD_IP}:8888
zipkin:
endpoint: 0.0.0.0:9411
service:
extensions:
- health_check
pipelines:
logs:
exporters:
- logging
processors:
- memory_limiter
- batch
receivers:
- otlp
traces:
exporters:
- otlp
processors:
- memory_limiter
- batch
receivers:
- otlp
- jaeger
- zipkin
kind: ConfigMap