Comments (9)
@anvo1115 Thanks for the report, but, as far as I can tell from the thread dump, that doesn't look deadlocked to me. reactor-http-epoll-3
is waiting to lock 0x00000000cf5c3e78
which is held by main
. However, main
is waiting to lock 0x00000000f19e0530
which isn't held by any other thread in the dump. In other words, judging by the thread dump, once the request that's being made by MicroserviceWebClient
has completed, processing can proceed.
from spring-boot.
The issue is that the request is not completed.
But after we exclude PrometheusExemplarsAutoConfiguration , the request passed.
from spring-boot.
from spring-boot.
Unfortunately, the thread dump doesn't explain why the request did not complete. None of the threads appear to be in the process of making an HTTP request so it's not clear why the thread that's waiting for one to complete is stuck. If you would like us to spend some more time investigating, please spend some time providing a complete yet minimal sample that reproduces the problem. You can share it with us by pushing it to a separate repository on GitHub or by zipping it up and attaching it to this issue.
from spring-boot.
If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.
from spring-boot.
Having looked again at spring-projects/spring-framework#32996, I think I now understand what's happening here.
The main
thread is making an HTTP request during bean creation and while Framework's singleton lock is held. While the request's reactive and using WebClient
, block()
is being called so the main
thread cannot proceed until the request has completed. reactor-http-epoll-3
is the thread that's performing the HTTP request. WebClient
has been instrumented and the observation for the request is being stopped. This results in an attempt to update the last exemplar. This gets stuck because it tries to use LazyTracingSpanContextSupplier
which needs to retrieve the Tracer
. Doing so requires Framework's singleton lock which cannot be obtained as it's held by main
.
This should already be fixed in Framework 6.2.0-M3 but we need to work something out for earlier releases.
@anvo1115, you could avoid the problem by not doing things on multiple threads while also blocking. That would either mean that you stop calling block
or that you use an imperative HTTP client.
On our side, it's becoming increasingly apparent that we need a better way of breaking the MeterRegistry <-> Tracer
cycle that exemplars cause. We'll discuss this with the observability team.
from spring-boot.
@anvo1115 it would be interesting to know if defining the following bean works around the problem for you:
@Bean
TracerSpanContextSupplier spanContextSuppler(Tracer tracer) {
return new TracerSpanContextSupplier(tracer);
}
static class TracerSpanContextSupplier implements SpanContextSupplier {
private final Tracer tracer;
TracerSpanContextSupplier(Tracer tracer) {
this.tracer = tracer;
}
@Override
public String getTraceId() {
Span currentSpan = currentSpan();
return (currentSpan != null) ? currentSpan.context().traceId() : null;
}
@Override
public String getSpanId() {
Span currentSpan = currentSpan();
return (currentSpan != null) ? currentSpan.context().spanId() : null;
}
@Override
public boolean isSampled() {
Span currentSpan = currentSpan();
if (currentSpan == null) {
return false;
}
Boolean sampled = currentSpan.context().sampled();
return sampled != null && sampled;
}
private Span currentSpan() {
return this.tracer.currentSpan();
}
}
from spring-boot.
If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.
from spring-boot.
Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.
from spring-boot.
Related Issues (20)
- Run Docker-based tests serially
- Run Docker-based tests serially
- Combine tests for standard and Bitnami images into single classes
- Replace RFC 7807 by RFC 9457 in property documentation HOT 2
- Combine tests for standard and Bitnami images into single classes
- Rework fix for #21992 to avoid polluting Gradle Module Metadata with mavenOptional… variants
- SpringBoot Liquibase Support of rollback automatically HOT 1
- Improve classpath index documentation for reproducible builds HOT 3
- Tracing propagation does not work in ControllerAdvice with async controllers since 3.2.7 HOT 6
- Polish gh-40023
- Rework fix for #21992 to avoid polluting Gradle Module Metadata with mavenOptional… variants
- Rework fix for #21992 to avoid polluting Gradle Module Metadata with mavenOptional… variants
- Update Kotlin DSL examples that configure the environment of bootBuildImage to be additive
- Update Kotlin DSL examples that configure the environment of bootBuildImage to be additive
- Make buildSrc's custom tasks consistently abstract
- Make buildSrc's custom tasks consistently abstract
- Make buildSrc's custom tasks consistently abstract
- TransactionalOperator is not configured in Data Neo4j HOT 6
- OtlpMetricsExportAutoConfiguration uses default OTLP metrics url when value comes from Dynamic Properties HOT 2
- Spring Boot 3.X Upgrade: Custom Health Indicator Not Recognized in Actuator Liveness Group HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spring-boot.