Coder Social home page Coder Social logo

opentelemetry-java-contrib's Introduction

OpenTelemetry Java Contrib

Build

This project is intended to provide helpful libraries and standalone OpenTelemetry-based utilities that don't fit the express scope of the OpenTelemetry Java or Java Instrumentation projects. If you need an easier way to bring observability to remote JVM-based applications and workflows that isn't easily satisfied by an SDK feature or via instrumentation, this project is hopefully for you.

Provided Libraries

Getting Started

# Apply formatting
$ ./gradlew spotlessApply

# Build the complete project
$ ./gradlew build

# Run integration tests
$ ./gradlew integrationTest

# Clean artifacts
$ ./gradlew clean

Contributing

The Java Contrib project was initially formed to provide methods of easy remote JMX metric gathering and reporting, which is actively in development. If you have an idea for a similar use case in the metrics, traces, or logging domain we would be very interested in supporting it. Please open an issue to share your idea or suggestion. PRs are always welcome and greatly appreciated, but for larger functional changes a pre-coding introduction can be helpful to ensure this is the correct place and that active or conflicting efforts don't exist.

Triagers (@open-telemetry/java-contrib-triagers):

Approvers (@open-telemetry/java-contrib-approvers):

Maintainers (@open-telemetry/java-contrib-maintainers):

Emeritus maintainers:

Learn more about roles in the community repository.

Thanks to all the people who already contributed!

opentelemetry-java-contrib's People

Contributors

akats7 avatar anosek-an avatar anuraaga avatar atshaw43 avatar breedx-splk avatar cyrille-leclerc avatar dehaansa avatar dependabot[bot] avatar github-actions[bot] avatar inikem avatar jack-berg avatar jeanbisutti avatar jkwatson avatar jonathanwamsley avatar kittylyst avatar kuba-wu avatar laurit avatar likethesalad avatar mxiamxia avatar oertl avatar opentelemetrybot avatar psx95 avatar renovate[bot] avatar rmfitzpatrick avatar roberttoyonaga avatar stefankurek avatar sylvainjuge avatar thpierce avatar trask avatar zeitlinger avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

opentelemetry-java-contrib's Issues

[maven extension] Exception if OTLP Exporter not configured properly

Description
NullPointerException if OTLP Exporter endpoint is not set.

[ERROR] Internal error: java.lang.NullPointerException -> [Help 1]
org.apache.maven.InternalErrorException: Internal error: java.lang.NullPointerException
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:120)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: java.lang.NullPointerException
    at io.opentelemetry.maven.OpenTelemetrySdkService.getPropagators (OpenTelemetrySdkService.java:188)
    at io.opentelemetry.maven.OtelExecutionListener.sessionStarted (OtelExecutionListener.java:80)
    at io.opentelemetry.maven.ChainedExecutionListener.sessionStarted (ChainedExecutionListener.java:38)
    at org.apache.maven.lifecycle.internal.DefaultExecutionEventCatapult.fire (DefaultExecutionEventCatapult.java:61)
    at org.apache.maven.lifecycle.internal.DefaultExecutionEventCatapult.fire (DefaultExecutionEventCatapult.java:42)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:74)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/InternalErrorException

Steps to reproduce
Run a Maven build with mvn -Dmaven.ext.class.path={path}/opentelemetry-maven-extension-1.7.0-SNAPSHOT.jar verify

Expectation
Build runs but doesn't capture/export traces

What applicable config did you use?
Only what was on the above command line.

Relevant Environment Information
Version: 1.7.0-SNAPSHOT
OS: macOS 11.4
Compiler (if manually compiled): AdoptOpenJDK-11.0.11+9

Additional context

[Maven extension] cant set service name

I want to set the service.name to something other than Maven.

.\mvnw.cmd -e clean install --threads 3 -Dotel.traces.exporter=otlp -Dotel.exporter.otlp.endpoint=http://10.1.0.203:4317 -Dotel.instrumentation.maven.mojo.enabled=false -Dmaven.ext.class.path=c:\opt\opentelemetry-maven-extension-1.10.0-20220112.132157-1.jar -Dotel.resource.attributes=service.name=cats,service.namespace=mammals -Dotel.service.name=django

In this line I've set the service name in 2 ways, but neither of them seems to have any effect
https://github.com/open-telemetry/opentelemetry-java/tree/main/sdk-extensions/autoconfigure

What am I doing wrong?

[maven-extension] Disable Mojo spans

Is your feature request related to a problem? Please describe.
In playing with the extension today on a build of Quarkus, it generates 17400 spans as each Mojo/plugin in a module build has its own span.

Though this information is beneficial in diagnosing a problem it does add a lot more information than is probably necessary for regular project builds. It might also be interesting to provide a list of mojo names that should have spans and ignore the rest, so known plugins that are troublesome can always have spans created?

Describe the solution you'd like
Ability to turn off span creation at the mojo level. Which option should be the default is possibly another discussion.

Describe alternatives you've considered
It would be possible to provide a sampling strategy to exclude them, but I think it's better to prevent their creation in the first place if that's what is desired.

Additional context
Happy to work on a fix for this if there's interest and agreement

Latest JMX gatherer broken with collector JMX receiver

JMX Gatherer 1.4.x release is unable to communicate with the collector (last collector release doesn't support latest OTLP metric attributes).

I think this could be fixed by bumping to 1.5.x where both labels + attributes are sent (or we can wait for the next collector version which will support full OTLP 0.9).

Versioning

We need to establish versioning and release support practices. At this time it seems* tracking w/ opentelemetry-java versions similar to opentelemetry-java-instrumentation is likely not appropriate given our reliance on their released, stable versions, as well as our probable release cadence. Basing our releases on their latest versions in some capacity could provide advantages in transparency and expected sdk functionality though, so this approach may still be desired.

When we begin releasing contributions this or whatever convention is desired should be adopted, though initial 0.0.x may be necessary.

Sampler that passes spans based on name prefix

Users have asked for a way to ensure that their manually created spans are never sampled out while still allowing the rest of instrumentation to sample normally. One approach I'm suggesting here is to create a new sampler that can be preconfigured with a set of span name prefixes to always include. That is, if a span name starts with one of the prefixes, it always passes. If it does not, then this sampler will delegate to another sampler.

Might be a little tricky to choose a name for this. Something like NamePrefixedForceInclusionSampler.

Maven plugin for static instrumentation using Java agent

Dear @open-telemetry/java-contrib-maintainers I'd like to contribute a Maven plugin, capable of ahead-of-time Java application instrumentation using Java OTel agent (aka "static instrumentation").

The goal of the project is:

  • implement proof of concept how existing agent can be integrated into the Maven build cycle to instrument applications during build time
  • provide a solution for all use cases where dynamic instrumentation is not feasible (primary example - AWS lambda suffering cold start performance issues with dynamic instrumentation)
  • measure startup performance dynamic vs static proving the point

Future goal within the community would be to improve the plugin from PoC into real product shape.

The code has been developed by a group of AGH (university in Krakow, Poland) students for their BSc thesis project (3rd year of studies), with active help from Splunk side.

The code can be found here: https://github.com/agh-static-x/Maven-Plugin

[Maven Extension] Mojo Goal Execution span name is too high cardinality

Description

Mojo Goal Execution span name is too high cardinality due to the fact it includes the artifactId of the project on which it's executed.

Actual: $mojo.groupId:$mojo.artifactId (execution.phase) @ $project.artifactId
It would be better cardinality to just do $mojo.groupId:$mojo.artifactId (execution.phase)

Make v. Gradle tasks

Re: #10 (comment) the adoption of Make in this project may be a point of contention or usability hindrance. Usage of Make isn't standardized across Otel, but is used.

Available:
https://github.com/open-telemetry/opentelemetry-java
https://github.com/open-telemetry/opentelemetry-proto
https://github.com/open-telemetry/opentelemetry-go
https://github.com/open-telemetry/opentelemetry-go-contrib
https://github.com/open-telemetry/opentelemetry-php
https://github.com/open-telemetry/opentelemetry-collector
https://github.com/open-telemetry/opentelemetry-collector-contrib
https://github.com/open-telemetry/opentelemetry.io

Unavailable:
https://github.com/open-telemetry/opentelemetry-java-instrumentation
https://github.com/open-telemetry/opentelemetry-js
https://github.com/open-telemetry/opentelemetry-js-contrib
https://github.com/open-telemetry/opentelemetry-python (except docs)
https://github.com/open-telemetry/opentelemetry-ruby
https://github.com/open-telemetry/opentelemetry-cpp
https://github.com/open-telemetry/opentelemetry-rust
https://github.com/open-telemetry/opentelemetry-dotnet
https://github.com/open-telemetry/opentelemetry-dotnet-contrib
https://github.com/open-telemetry/opentelemetry-erlang
https://github.com/open-telemetry/opentelemetry-erlang-api

Of the projects using it, four are in golang and two others are polyglot, where Make could be a natural choice. Java projects are split in usage.

Should we continue to use Make or should we only adopt direct gradlew usage? In the hope to avoid bikeshedding, we should be mainly concerned where the stability, maintainability, and general accessibility of the project are in potential jeopardy.

My 2ยข: I like the relatively intuitive nature of Make as a standardize interface and don't see its usage as an issue. It also plays nicely w/ my shell's completion/suggestion plugin where gradle does not. If gradle is adopted, then at the least we should define and document high level tasks similar to the current make targets to prevent loaded invocations.

JMX Metrics: allow specifying multiple target systems

Is your feature request related to a problem? Please describe.
It's not currently possible to use multiple target system scripts for the same target server without running multiple java processes, each with a different property value.

Describe the solution you'd like
otel.jmx.target.system should take comma-separated names and run all applicable scripts on the interval:

-Dotel.jmx.target.system=jvm,kafka,kafka-consumer

Describe alternatives you've considered
An alternative would be to import and run other target scripts in the scripts themselves, but configurability would suffer and this doesn't feel intuitive to me.

Add labels to entry level tasks for new contributors

Hi,

based on open-telemetry/community#469 I have added open-telemetry/opentelemetry-java-contrib to Up For Grabs:

https://up-for-grabs.net/#/filters?names=653

There are currently no issues with label help wanted. Please add this label to your entry level tasks, so people can find a way to contribute easily.

If "help wanted" is not the right label, let me know and I can change it (e.g. to "good first issue" or "up-for-grabs"), or you can provide a pull request by editing https://github.com/up-for-grabs/up-for-grabs.net/blob/gh-pages/_data/projects/opentelemetry-java-contrib.yml

Thanks!

X-Ray Remote Sampler does not poll for new rules

Description
Currently, the X-Ray sampler only polls the GetSamplingRules once on startup:

This is problematic because if a customer creates a new sampling rule, it will never be picked up by the application until it is redeployed.

Steps to reproduce

  1. Create a sample app using the X-Ray Remote Sampler
  2. Run the application
  3. Create a new sampling rule on the AWS Console
  4. Observe that the new sampling rule is never picked up by the app

Expectation
Instead, the remote sampler should poll the GetSamplingRules API periodically, every 5 minutes.

cc @anuraaga

JMX: Better modularize integration testing patterns

Is your feature request related to a problem? Please describe.
Currently adding integration tests to the JMX Metric Gatherer consists largely of repeating patterns and expanding the configureContainers() method of https://github.com/open-telemetry/opentelemetry-java-contrib/blob/master/contrib/jmx-metrics/src/test/groovy/io/opentelemetry/contrib/jmxmetrics/IntegrationTest.groovy.

Describe the solution you'd like
There should be a clear format for expected metrics and a more intuitive and maintainable system of registering expected target test containers and determining their startup behavior. Ideally this will allow easier test authoring and maintenance as more target systems are added.

Additional context
The assertTraces and related helpers in opentelemetry-java-instrumentation could be one inspiration for this effort.

X-Ray formatted trace ID injection

Is your feature request related to a problem? Please describe.
I would like to be able to instrument common logging libraries like Log4J and Logback to inject trace IDs that are formatted in the AWS X-Ray format, as opposed to the default W3C format.

Describe the solution you'd like
This solution is already available in the downstream AWS-distributed Java agent (source). I would like a similar library instrumentation available so I can use it without needing the agent.

Describe alternatives you've considered
Injecting trace IDs in W3C format works for manual trace -> log correlation, but for automated correlation done by AWS services like ServiceLens, the X-Ray format is required.

cc @anuraaga

Unable to push to PR branch

Description
When trying to update #190 with PR feedback I found I was unable to push my changes. This occurs with simple additions and after rebasing the branch and attempting to force push:

remote: Resolving deltas: 100% (4/4), completed with 4 local objects.
remote: error: GH006: Protected branch update failed for refs/heads/jmxremovemanualflush.
remote: error: Required status check "EasyCLA" is expected.
To github.com:open-telemetry/opentelemetry-java-contrib.git
 ! [remote rejected] jmxremovemanualflush -> jmxremovemanualflush (protected branch hook declined)
error: failed to push some refs to 'github.com:open-telemetry/opentelemetry-java-contrib.git'

I am not too familiar with the EasyCLA process but it appears its status check must have been considered successful for local branches with the current protection rule. It is possible I'm doing something incorrectly.

[Maven Extension] Maven parallel / multithreaded builds fail

My reactor fails with

java.lang.IllegalStateException: A span has already been started for com.github.ekryd.sortpom:sortpom-maven-plugin:3.0.0:sort {execution: project}
    at io.opentelemetry.maven.SpanRegistry.putSpan (SpanRegistry.java:90)
    at io.opentelemetry.maven.OtelExecutionListener.mojoStarted (OtelExecutionListener.java:205)                                                                                         
    at io.opentelemetry.maven.ChainedExecutionListener.mojoStarted (ChainedExecutionListener.java:88)
    at org.apache.maven.lifecycle.internal.DefaultExecutionEventCatapult.fire (DefaultExecutionEventCatapult.java:84)                                                                    
    at org.apache.maven.lifecycle.internal.DefaultExecutionEventCatapult.fire (DefaultExecutionEventCatapult.java:42)                                                                   
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:204)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)                                                                                                  
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)                                                                                                  
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)     
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:196)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:186)

I effective ran ./mvnw validate -e --threads 1C with an external OTEL_EXPORTER_OTLP_ENDPOINT

I disabled the sortpom, and then maven enforcer, and then it failed on clean plugin. Only way to survive the build is --threads 1.

Supporting B3 baggage propagation

Is your feature request related to a problem? Please describe.
Adding support for Jaeger B3 baggage header propagation. Open Telemetry does not support baggage header propagation in a format expected by Jaeger Open tracing. Open Telemetry Java Issue-3661 and Open Telemetry Specification Issue-1971 have been raised for this. As per comments of maintainers this can be added in opentelemetry-java-contrib.

Describe the solution you'd like
Add new Baggage propagator B3BaggagePropagator to make it backward compatible. This will be based on baggage header format mentioned in Jaeger B3TextMapCodec.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Add Support for Multiple ObjectNames

Is your feature request related to a problem? Please describe.
There is an issue when multiple mbeans are under different object names but are similar enough that they could be given the same metric name and differentiated through labels. This isn't possible currently because when a second call to instrument with the same metric name happens, it overrides the previous call to instrument.

EX:

def beanActiveTasks = otel.mbeans("org.apache.cassandra.metrics:type=ThreadPools,path=*,scope=*,name=PendingTasks")
otel.instrument(beanActiveTasks, "cassandra.current_tasks", "Number of tasks in queue with the given task status.", "1", 
    ["stage_name":{ mbean -> mbean.name().getKeyProperty("scope")},
    "task_status":{"pending"}],
    "Value", otel.&doubleValueObserver)

def beanPendingTasks = otel.mbeans("org.apache.cassandra.metrics:type=ThreadPools,path=*,scope=*,name=ActiveTasks")
otel.instrument(beanPendingTasks, "cassandra.current_tasks", "Number of tasks in queue with the given task status.", "1", 
    ["stage_name":{ mbean -> mbean.name().getKeyProperty("scope")},
    "task_status":{"active"}],
    "Value", otel.&doubleValueObserver)

Here is where there are two different calls to instrument with the same metric name. They end up overriding each other when passed through.

Describe the solution you'd like
A way to make this possible is by adding an additional signature to otel.mbeans( ) that allows multiple object names to be passed in.

EX:

def beanActiveTasks = otel.mbeans(["org.apache.cassandra.metrics:type=ThreadPools,path=*,scope=*,name=PendingTasks",
"org.apache.cassandra.metrics:type=ThreadPools,path=*,scope=*,name=ActiveTasks"])
otel.instrument(beanActiveTasks, "cassandra.current_tasks", "Number of tasks in queue with the given task status.", "1", 
    ["stage_name":{ mbean -> mbean.name().getKeyProperty("scope")},
    "task_status":{ mbean -> mbean.name().getKeyProperty("name")}],
    "Value", otel.&doubleValueObserver)

Add Support for Multiple Attributes

Is your feature request related to a problem? Please describe.
Add Ability to pass in multiple Attributes into one otel.instrument() call.

Describe the solution you'd like
What I propose is adding additional signatures to otel.instrument that support attributes in the format of Map<String, Map<String,String>> where the strings represent <Attribute, <LabelKey, LabelValue>>

Example:
currently what is required to collect this metric

otel.instrument(beantomcatconnectors, "tomcat.threads.idle", "description", "1",
    ["proto_handler" : { mbean -> mbean.name().getKeyProperty("name") }],
    "currentThreadCount", otel.&doubleValueObserver)
otel.instrument(beantomcatconnectors, "tomcat.threads.busy", "description", "1", 
    ["proto_handler" : { mbean -> mbean.name().getKeyProperty("name") }],
    "currentThreadsBusy", otel.&doubleValueObserver)

What it would like with the proposed solution

otel.instrument(beantomcatconnectors, "tomcat.threads", "description", "1",
    ["proto_handler" : { mbean -> mbean.name().getKeyProperty("name") }],
    ["currentThreadCount": ["Thread Type": "current"],
    "currentThreadsBusy": ["Thread Type": "busy"]], 
    otel.&doubleValueObserver)

[Maven Extension] Logging options

More control over the logs would be nice. I want tracing on by default, so if the endpoint is inaccessible for whatever reason Id like to silence this sort of thing

Jan 04, 2022 12:05:29 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
SEVERE: Failed to export spans. The request could not be executed. Full error message: Failed to connect to /10.1.1.1:4317

Can use https://maven.apache.org/ref/3.8.4/apidocs/org/slf4j/impl/SimpleLogger.html rather than stdout, like https://github.com/gitflow-incremental-builder/gitflow-incremental-builder/blob/master/src/main/java/com/vackosar/gitflowincrementalbuild/boundary/MavenLifecycleParticipant.java

And/or a separate -Dotel.log=none

http.url based Sampler

There is a request in java instrumentation repo to filter out spans for specific endpoints, such as healthchecks.

There is also a corresponding specification issue.

Before attempting to submit a specification proposal, I would like to try to create a POC sampler in this repository. It would take a list of url patterns and a delegate sampler. Sampler.shouldSample will then check if passed attributes contain http.url semantic attribute. If they do and that url matches one of the patterns, io.opentelemetry.sdk.trace.samplers.SamplingDecision#DROP will be returned. Otherwise, decision will be delegated to the delegate sampler.

At this moment I don't propose nor plan any auto-configuration support for this sampler.

Request for a few additional component owners

We would like to have (at least) 2 owners for each component in this repository. The component owners will be automatically assigned to review PRs that touch files in those components.

aws-xray, jfr-streaming, maven-extension, and runtime-attach have (at least) 2 owners already.

jmx-metrics needs one additional owner - @Mrod1598 would you be interested? @rmfitzpatrick any other recommendations for people we could ask?

contrib-samplers also needs one additional owner - any volunteers?

disruptor-processor currently has no owners - @bogdandrutu are you still interested in this component?

AWS X-Ray Centralized Sampler

Is your feature request related to a problem? Please describe.
Today we already have a Jaeger remote sampler in the Java SDK, but we don't have one that targets AWS X-Ray, it would be nice to have one sampler for X-Ray.

Describe the solution you'd like
A sampler that takes the service name, a fallback sampler, and a endpoint parameter that can point to the X-Ray service or to the X-Ray receiver with proxy server enabled, that fetches the sampling config and does centralized sampling.

Triaging issues with `comp:xxx` labels similarly to what otel-collector-contrib does

Is your feature request related to a problem? Please describe.

It's hard today to see which issue is associated to which component (e.g. aws-xray, jfr-streaming, jmx-metrics, maven-extension, runtime-attach...).

Describe the solution you'd like

Could we simplify this triaging process considering adopting the solution chosen by the OpenTelemetry Collector Contrib repo with the comp:xxx labels (see here)?

Our labels could lok like:

  • comp:aws-xray
  • comp:jfr-streaming
  • comp:jmx-metrics
  • comp:maven-extension
  • comp:runtime-attach
  • comp:samplers

Describe alternatives you've considered

Additional context

Add support for cli params/custom label in groovy script

Currently, when you are trying to monitor the JVM metrics(using JMX receiver) for multiple processes/applications on a single host, it is impossible to differentiate which process the metrics originated from.
Having support for adding custom labels to the groovy script through command line params or a config would provide the required flexibility to re-use the same groovy script to solve the problem stated above and also give end users the option to add more labels.
Example

 jmx:
 jmx/service1:
  jar_path: /home/opentelemetry-java-contrib-jmx-metrics-0.0.1-SNAPSHOT.jar
  endpoint: localhost:9050
  target_system: jvm
  custom_lables: 
     <label-key>: <label-value>
  collection_interval: 10s

OR

 jmx:
 jmx/service1:
  jar_path: /home/opentelemetry-java-contrib-jmx-metrics-0.0.1-SNAPSHOT.jar
  endpoint: localhost:9050
  groovy_script: test.groovy --labels [<key>:<value>, ...]
  collection_interval: 10s

Missing Attributes on existing MBean causes partial collection failure

Description
If an MBean exists but the requested attribute is missing, an AttributeNotFoundException is thrown and only the metrics collected before that exception are returned.

Steps to reproduce
This was observed in a Windows system that lacked the OpenFileDescriptorCount metric of the java.lang:type=OperatingSystem MBean.

Expectation
The metric requested would be missing, but metrics listed after it in the groovy script would still be collected.

What applicable config did you use?
def os = otel.mbean("java.lang:type=OperatingSystem") otel.instrument(os, "jvm.os.files.open", "open file descriptors", "1", "OpenFileDescriptorCount", otel.&longValueObserver)

[Maven Extension] Maven daemon build fails on second run

Reproduce as in #160 but use mvnd instead of mvn
https://github.com/apache/maven-mvnd/releases/tag/0.7.1

mvnd validate -Dotel.traces.exporter=otlp -Dotel.exporter.otlp.endpoint=http://10.1.1.1:4317 -Dmaven.ext.class.path=/tmp/opentelemetry-maven-extension-1.10.0-SNAPSHOT.jar -T3

The first run is successful, but the second fails with:

[INFO] Processing build on daemon e432b3dc
[INFO] Scanning for projects...
[ERROR] Internal error: java.lang.IllegalStateException: Root span already defined RecordEventsReadableSpan{traceId=ab428d41352f1a99385654f781e68ffc, spanId=fe206e623f88a74d, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, name=Build: org.apache.maven:maven:4.0.0-alpha-1-SNAPSHOT, kind=SERVER, attributes=AttributesMap{data={maven.project.version=4.0.0-alpha-1-SNAPSHOT, maven.project.groupId=org.apache.maven, maven.project.artifactId=maven}, capacity=128, totalAddedValues=3}, status=ImmutableStatusData{statusCode=UNSET, description=}, totalRecordedEvents=0, totalRecordedLinks=0, startEpochNanos=1641289758683000000, endEpochNanos=1641289762712689563} -> [Help 1]
org.apache.maven.InternalErrorException: Internal error: java.lang.IllegalStateException: Root span already defined RecordEventsReadableSpan{traceId=ab428d41352f1a99385654f781e68ffc, spanId=fe206e623f88a74d, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, name=Build: org.apache.maven:maven:4.0.0-alpha-1-SNAPSHOT, kind=SERVER, attributes=AttributesMap{data={maven.project.version=4.0.0-alpha-1-SNAPSHOT, maven.project.groupId=org.apache.maven, maven.project.artifactId=maven}, capacity=128, totalAddedValues=3}, status=ImmutableStatusData{statusCode=UNSET, description=}, totalRecordedEvents=0, totalRecordedLinks=0, startEpochNanos=1641289758683000000, endEpochNanos=1641289762712689563}
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:120)
        at org.apache.maven.cli.DaemonMavenCli.execute(DaemonMavenCli.java:825)
        at org.apache.maven.cli.DaemonMavenCli.doMain(DaemonMavenCli.java:244)
        at org.apache.maven.cli.DaemonMavenCli.main(DaemonMavenCli.java:222)
        at org.mvndaemon.mvnd.daemon.Server.handle(Server.java:573)
        at org.mvndaemon.mvnd.daemon.Server.client(Server.java:262)
        at org.mvndaemon.mvnd.daemon.Server.accept(Server.java:230)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Root span already defined RecordEventsReadableSpan{traceId=ab428d41352f1a99385654f781e68ffc, spanId=fe206e623f88a74d, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, name=Build: org.apache.maven:maven:4.0.0-alpha-1-SNAPSHOT, kind=SERVER, attributes=AttributesMap{data={maven.project.version=4.0.0-alpha-1-SNAPSHOT, maven.project.groupId=org.apache.maven, maven.project.artifactId=maven}, capacity=128, totalAddedValues=3}, status=ImmutableStatusData{statusCode=UNSET, description=}, totalRecordedEvents=0, totalRecordedLinks=0, startEpochNanos=1641289758683000000, endEpochNanos=1641289762712689563}
        at io.opentelemetry.maven.SpanRegistry.setRootSpan(SpanRegistry.java:39)
        at io.opentelemetry.maven.OtelExecutionListener.sessionStarted(OtelExecutionListener.java:139)
        at io.opentelemetry.maven.ChainedExecutionListener.sessionStarted(ChainedExecutionListener.java:36)
        at org.apache.maven.lifecycle.internal.DefaultExecutionEventCatapult.fire(DefaultExecutionEventCatapult.java:61)
        at org.apache.maven.lifecycle.internal.DefaultExecutionEventCatapult.fire(DefaultExecutionEventCatapult.java:42)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:74)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
        ... 7 common frames omitted

Kill the daemon with mvnd --stop and try again and it works.

Improve attribute mapping for X-Ray Remote Sampler

Description
Currently, the attribute matching performed by the X-Ray remote sampler is confusing or incorrect for two attributes:

Expectation

  1. The X-Ray Sampling Rule attribute "service name" is currently mapped to the span name. Given OTel's convention of naming spans after operations and not services, this doesn't seem accurate. Instead (or in addition to the span name), this attribute should be matched against otel.resource.service.name semantic attribute.
  2. The X-Ray Sampling Rule attribute "HTTP URL" is currently mapped to the http.target semantic attribute. In addition, it should also map to the http.url semantic attribute.

Additional context
The documentation should be updated to reflect these mappings, see aws-otel/aws-otel.github.io#201

Also, @anuraaga can you confirm that the arbitrary key-value pairs of attributes that X-Ray users can define on the AWS console can be used to match against any semantic attribute?

JakartaEE Integration

Is your feature request related to a problem? Please describe.
Allow JakartaEE applications to use OpenTelementry.
APIs and their implementations could also use this.

Describe the solution you'd like
Add a CDI bean producer to properly wire the OpenTelementry(OT) context and allow the usage of OT anotations in any JakartaEE application.

Describe alternatives you've considered
This can be directly included in the opentelemetry-java lib, but it seems to me that here is a better fit.

Additional context
The Eclipse MicroProfile community has created a sandbox project to evaluate OpenTelementry and the future of Microprofile OpenTracing. This is part of that brainstorming.
I volunteer to move this forward if you guys like the idea.

[Maven Extension] Release lean and fat JARs

With respect to #86
the size of the opentelemetry-maven-extension v1.10.0-alpha now stands at 16mb. Its certainly convenient to have a single JAR to point to with -Dmaven.ext.class.path=, but the inefficiency is passed elsewhere in the system.
I want to declare the lean version in /.mvn/extensions.xml
You could make another JAR with everything in it (opentelemetry-maven-extension-all-1.10.0-alpha.jar) for the command line use case.

[maven-extension] Failure using the Otel Maven extension declared as an `<extension>` in `pom.xml` because the `pom.xml` of the Otel Maven extension lists dependencies even though they are bundled

Description

Failure using the Maven extension declared as a in pom.xml because the pom.xml of the Maven extension lists dependencies even though they are bundled in the opentelemetry-maven-extension jar (which is a uber-jar).

The pom.xml of io.opentelemetry.contrib:opentelemetry-maven-extension:1.6.0-alpha is wrong as it lists as runtime dependencies the OpenTelemetry and GRPC jars that have been integrated in the opentelemetry-maven-extension jar thanks to the Gradle Shadow Plugin.

https://repo1.maven.org/maven2/io/opentelemetry/contrib/opentelemetry-maven-extension/1.6.0-alpha/opentelemetry-maven-extension-1.6.0-alpha.pom

opentelemetry-maven-extension-1.6.0-alpha.pom

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
   <!-- This module was also published with a richer model, Gradle metadata,  -->
   <!-- which should be used instead. Do not delete the following line which  -->
   <!-- is to indicate to Gradle or any Gradle module metadata file consumer  -->
   <!-- that they should prefer consuming it instead. -->
   <!-- do_not_remove: published-with-gradle-metadata -->
   <modelVersion>4.0.0</modelVersion>
   <groupId>io.opentelemetry.contrib</groupId>
   <artifactId>opentelemetry-maven-extension</artifactId>
   <version>1.6.0-alpha</version>
   <name>OpenTelemetry Java Contrib</name>
   <description>Maven extension to observe Maven builds with distributed traces using OpenTelemetry SDK</description>
   <url>https://github.com/open-telemetry/opentelemetry-java-contrib</url>
   <licenses>
      <license>
         <name>The Apache License, Version 2.0</name>
         <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
      </license>
   </licenses>
   <developers>
      <developer>
         <id>opentelemetry</id>
         <name>OpenTelemetry</name>
         <url>https://github.com/open-telemetry/opentelemetry-java-contrib/discussions</url>
      </developer>
   </developers>
   <scm>
      <connection>scm:git:[email protected]:open-telemetry/opentelemetry-java-contrib.git</connection>
      <developerConnection>scm:git:[email protected]:open-telemetry/opentelemetry-java-contrib.git</developerConnection>
      <url>[email protected]:open-telemetry/opentelemetry-java-contrib.git</url>
   </scm>
   <dependencies>
      <dependency>
         <groupId>org.codehaus.plexus</groupId>
         <artifactId>plexus-component-annotations</artifactId>
         <version>2.1.0</version>
         <scope>runtime</scope>
      </dependency>
      <dependency>
         <groupId>io.opentelemetry</groupId>
         <artifactId>opentelemetry-api</artifactId>
         <version>1.6.0</version>
         <scope>runtime</scope>
      </dependency>
      <dependency>
         <groupId>io.opentelemetry</groupId>
         <artifactId>opentelemetry-sdk</artifactId>
         <version>1.6.0</version>
         <scope>runtime</scope>
      </dependency>
      <dependency>
         <groupId>io.opentelemetry</groupId>
         <artifactId>opentelemetry-sdk-trace</artifactId>
         <version>1.6.0</version>
         <scope>runtime</scope>
      </dependency>
      <dependency>
         <groupId>io.opentelemetry</groupId>
         <artifactId>opentelemetry-semconv</artifactId>
         <version>1.6.0-alpha</version>
         <scope>runtime</scope>
      </dependency>
      <dependency>
         <groupId>io.opentelemetry</groupId>
         <artifactId>opentelemetry-exporter-otlp</artifactId>
         <version>1.6.0</version>
         <scope>runtime</scope>
      </dependency>
      <dependency>
         <groupId>io.opentelemetry</groupId>
         <artifactId>opentelemetry-exporter-otlp-trace</artifactId>
         <version>1.6.0</version>
         <scope>runtime</scope>
      </dependency>
      <dependency>
         <groupId>io.grpc</groupId>
         <artifactId>grpc-netty-shaded</artifactId>
         <version>1.39.0</version>
         <scope>runtime</scope>
      </dependency>
   </dependencies>
</project>

Steps to reproduce
Provide a (runnable) recipe for reproducing the error.

  • Add the io.opentelemetry.contrib:opentelemetry-maven-extension:1.6.0-alpha extension to a pom.xml (example here)

    <project>
        ...
        <build>
            ...
            <extensions>
                <extension>
                    <groupId>io.opentelemetry.contrib</groupId>
                    <artifactId>opentelemetry-maven-extension</artifactId>
                    <version>1.6.0-alpha</version>
                </extension>
            </extensions>
        </build>
    </project>
  • Run you build

    export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317"
    mvn verify
    
  • We expect the build to succeed but Maven reports a problem of dependencies of the io.opentelemetry.contrib:opentelemetry-maven-extension:jar:1.6.0-alpha extension that should not have any extension

    [INFO] Scanning for projects...
    [ERROR] [ERROR] Some problems were encountered while processing the POMs:
    [ERROR] Unresolveable build extension: Plugin io.opentelemetry.contrib:opentelemetry-maven-extension:1.6.0-alpha or one of its dependencies could not be resolved: Failed to collect dependencies at io.opentelemetry.contrib:opentelemetry-maven-extension:jar:1.6.0-alpha -> io.grpc:grpc-netty-shaded:jar:1.39.0 -> io.grpc:grpc-core:jar:[1.39.0] @
     @
    [ERROR] The build could not read 1 project -> [Help 1]
    [ERROR]
    [ERROR]   The project com.example:demo-parent:0.0.30-SNAPSHOT (/Users/cyrilleleclerc/git/cyrille-leclerc/multi-module-maven-project/pom.xml) has 1 error
    [ERROR]     Unresolveable build extension: Plugin io.opentelemetry.contrib:opentelemetry-maven-extension:1.6.0-alpha or one of its dependencies could not be resolved: Failed to collect dependencies at io.opentelemetry.contrib:opentelemetry-maven-extension:jar:1.6.0-alpha -> io.grpc:grpc-netty-shaded:jar:1.39.0 -> io.grpc:grpc-core:jar:[1.39.0]: No versions available for io.grpc:grpc-core:jar:[1.39.0] within specified range -> [Help 2]
    [ERROR]
    [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
    [ERROR] Re-run Maven using the -X switch to enable full debug logging.
    [ERROR]
    [ERROR] For more information about the errors and possible solutions, please read the following articles:
    [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
    [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/PluginManagerException
    

Expectation

The pom.xml of io.opentelemetry.contrib:opentelemetry-maven-extension:jar:1.6.0-alpha should not list any compile or runtime dependency

What applicable config did you use?
N/A

Relevant Environment Information
Version: io.opentelemetry.contrib:opentelemetry-maven-extension:jar:1.6.0-alpha , Maven 3.8.2

Additional context
Add any other context about the problem here.

JMX Metrics: Static configuration files

From #4 (comment) and similar discussions, a standardized MBean object and attribute specification format should be designed and implemented to allow for JMX Metric Gatherer functionality without the caveats that arise from arbitrary groovy script execution.

At a broad view, these scripts should allow for the configuration of jmx service url, authentication information, and a list of MBeans and their attributes to target otel instruments to be recorded on a configured interval. The implemented support of such files can be via extending the current JMX Metric Gatherer or distributing as a separate utility to disallow groovy script execution in total.

Recommended prior art audit would include jmxtrans and genericJMX

Support export via kafka

Is your feature request related to a problem? Please describe.
The open telemetry collector project supports a kafka based collector, that according to the description can read otlp protos from a configured topic, instead of using grpc as the network layer. Currently i dont see a way to configure the instrumentation to publish messages to kafka, instead of using grpc.

Describe the solution you'd like
an export wrapper that would allow me to publish messages to kafka

Describe alternatives you've considered
dont know yet if this can affect or improve performance

Additional context
Add any other context or screenshots about the feature request here.

[maven-extension] Add the OpenTelemetry Maven Extension to the Maven documentation

Is your feature request related to a problem? Please describe.

The Maven documentation lists the Maven extensions in [Apache/ Maven/ Available Extensions]https://maven.apache.org/extensions/index.html). The OpenTelemetry Maven Extension should be listed here

We should add the OpenTelemetry Maven extension to this documentation page for better visibility.

Describe the solution you'd like

Declare the OpenTelemetry Maven extension, it could look like:

Extension Maintainer Description
opentelemetry The OpenTelemetry community OpenTelemetry instrumentation of Maven builds capturing Maven mojo executions as distributed traces spans.

Describe alternatives you've considered

Not documenting the OpenTelemetry extension on Maven website.

Additional context

N/A

Donating a Maven OpenTelemetry extension to the OpenTelemetry project?

Noter that this message is a cross post of open-telemetry/community#832 as suggested by @trask .


Dear community,

We would like to donate to the OpenTelemetry project a Maven OpenTelemetry extension that we have created to observe the execution of Maven builds.

This Maven instrumentation integrates with other OpenTelemetry capable CI/CD tools such as Jenkins pipelines (Jenkins OpenTelemetry plugin), Ansible deployments (Ansible OpenTelemetry plugin) or the otel-cli.

Here is an example of distributed trace providing visibility of a Jenkins / Maven CI pipeline.

It's very straightforward to test the extension on any existing Maven project as we have documented here.

We have very positive feedback from CI/CD practitioners when we show the capabilities and the extension is already used by Elastic on its production CI platform in conjunction with the Jenkins OpenTelemetry plugin.

We would like to donate the Maven OpenTelemetry extension to the OpenTelemetry project / CNCF foundation because we feel that it would be the best place to develop this integration and because we see the opportunity to create an entire ecosystem of OpenTelemetry capable CI/CD tools.

Note that we preferred to build this integration as a Maven extension rather than as an auto instrumentation that would be part of the OpenTelemetry Java agent because the Maven community is used to define its builds adding plugins and extension to build definitions (pom.xml) rather than wiring a Java auto-instrumentation agent when launching the "mvn" command.

Would the OpenTelemetry Java Contrib project be interested in hosting such Maven OpenTelemetry integration?

Following the donations guidelines, I feel this donation can be qualified as a "small donation" and a simple Pull Request would then be enough.

[Maven Extension] Maven runners

Just as Jenkins has agents, and there should be a span/node for each one, also Maven has threads/runners. Running builds in mvnd gives a better indication of how efficiently the modules are processed by each runner, but it would be nicer to have a span/node for each thread/runner.

jmx-metrics: Integration Tests don't pass locally without bumping timing for waiting for metrics to 120s

Description
In the waitAndAssertMetrics function We give it at most 30s to get metrics but locally that never seems to be enough.

KafkaIntegrationTest$KafkaConsumerIntegrationTest > endToEnd() FAILED
    org.awaitility.core.ConditionTimeoutException: Assertion condition defined as a io.opentelemetry.contrib.jmxmetrics.AbstractIntegrationTest 
    Expecting actual not to be empty within 30 seconds.

It's not isolated to kafka, it's all integrations.

Steps to reproduce
Set Docker Resources to:
image

run ./gradlew integrationTest

Expectation
Integration tests pass

Proposed Solution
Bump the duration to 120s most integration tests won't hit this on actions as it has a lot more room to work with so it'd only affect local development.

Sporadic build failure

2021-09-24T18:14:00.4353917Z OtlpIntegrationTests > end to end with stdin config: true FAILED
2021-09-24T18:14:00.4380292Z     Condition not satisfied:
2021-09-24T18:14:00.4380937Z 
2021-09-24T18:14:00.4381693Z     receivedMetrics.size() == 1
2021-09-24T18:14:00.4382530Z     |               |      |
2021-09-24T18:14:00.4383191Z     []              0      false
2021-09-24T18:14:00.4385365Z         at io.opentelemetry.contrib.jmxmetrics.OtlpIntegrationTests.end to end with stdin config: #useStdin(OtlpIntegrationTests.groovy:37)

full build log

[Help needed] Deploying opentelemetry-sqlcommenter-java to maven central

SQLCommenter is a library donated by Google to OpenTelemetry. We are in a process to of migrating the source code from google-cloud-sqlcommenter to opentelemetry-sqlcommenter.

SQLCommenter contains middleware/plugins that adds the capability of instrumenting SQL queries with code that executes the queries i.e. information like action, controller, dbdriver, etc and opentelemetry-trace(if present). More Details here

One of the supported language is Java, which includes spring framework and hibernate ORM (https://github.com/open-telemetry/opentelemetry-sqlcommenter/tree/main/java/sqlcommenter-java).

For the process of migration, we have moved in the code, documentation and renamed the packages to point to opentelemetry instead of google-cloud. Now our next task is to publish the packages to maven central. We had similar requirements from Python team and the suggestion was to create a ticket for that so that they can share a limited scope token (to only deploy packages for our repo into opentelemetry organization in PyPi): link.

Need help to understand the process followed for opentelemetry-java-* libraries and how they are published. Also if we need any access to publish opentelemetry-sqlcommenter-java to maven central under opentelemetry org.

cc: @jsuereth

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.