Comments (8)
Tagging @getsaurabh02 @msfroh @bbarani to see if they may be aware of any changes. In parallel, looking through the commit history to see if I can find some commit which could've cause this.
from opensearch.
One of the commits (on the same day when regression started) which touch aggregation path slightly: 8332859 [Can be evaluated if this could have had some impact]
from opensearch.
@mgodwan This looks related to the #13179 where @bowenlan-amzn has added cluster setting to dynamically disable filter rewrite optimization.
Based on the description it reduces the deciding threshold for rewrite filters from 1024 to 24. Meaning if the date histogram aggregation include more than 24 buckets (e.g. hourly aggregation of 1 day), we won't use the optimization After this change, we will probably see regression for date_histogram_hourly_agg of big5 workload. That will be handled after the long term solution merged in next.
from opensearch.
The change causing this is adding a dynamic cluster setting to decrease the threshold of apply our optimization on date histogram. The threshold is the number of filters rewritten from date histogram. Previous 1024 is reported to causing regression on pmc workload.
Since it's a dynamic setting, it won't actually cause regression for users and instead giving them ability to tune for their workload.
The PR for long term fix: #13317
from opensearch.
Thanks @bowenlan-amzn
Since it's a dynamic setting, it won't actually cause regression for users and instead giving them ability to tune for their workload.
Is this setting enabled for the benchmark setup where we are seeing regression?
from opensearch.
The setting is a threshold. This operation of http workload currently exceed the threshold so our previous optimization is disabled, hence the regression.
from opensearch.
[Triage - attendees 1 2 3 4 5 6]
@mgodwan Thanks for creating this issue. This looks like a potential release blocking for v2.14. Please let me know if you need any help getting eyes on this issue.
from opensearch.
The setting is a threshold. This operation of http workload currently exceed the threshold so our previous optimization is disabled, hence the regression.
@bowenlan-amzn Do we need to revisit the threshold defaults in that case as the current ones have shown to cause regression?
from opensearch.
Related Issues (20)
- [Feature Request] Add 'show' command to the keystore CLI HOT 1
- [Tiering] API contract for status API HOT 1
- [Tiering] API contract for shard level updates HOT 1
- [AUTOCUT] Gradle Check Failure on push to 2.x HOT 1
- [AUTOCUT] Gradle Check Failure on push to 2.x HOT 1
- [AUTOCUT] Gradle Check Failure on push to main HOT 1
- [BUG] DATETIME_FORMATTER_CACHING_SETTING experimental feature should not default to true HOT 35
- [AUTOCUT] Gradle Check Failure on push to main HOT 1
- [BUG] [Flaky Test] org.opensearch.repositories.azure.AzureBlobContainerRetriesTests.testReadNonexistentBlobThrowsNoSuchFileException
- Switch macos runner from `macos-13` to `macos-latest` once github finalize their migrations
- [BUG] very slow docker boot start on v2.13.0 HOT 1
- [META] Top N Query Shapes by Latency and Resource Usage HOT 1
- [AUTOCUT] Gradle Check Failure on push to 2.x
- [BUG] Allocation and Rebalance Constraints of WeightFunction are incorrectly reset to default if indexBalanceFactor, shardBalanceFactor or preferPrimaryShardRebalanceBuffer is updated HOT 1
- [BUG] AzureBlobContainerRetriesTests.testReadBlobWithRetries is flaky HOT 1
- [AUTOCUT] Gradle Check Failure on push to 2.x
- [BUG] Flaky test - org.opensearch.indices.IndicesRequestCacheIT.testStaleKeysCleanupWithMultipleIndices
- [AUTOCUT] Gradle Check Failure on push to main HOT 1
- [Feature Request] Create setting to disable CompoundFile usage HOT 1
- [Feature Request] Refactor ICache.stats() to return an interface rather than ImmutableCacheStatsHolder HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opensearch.