Comments (9)
Synced up with @khushbr and provided her the details.
The reason we don't see regression on 2.13 is because the commit Khushboo mentioned has not been merged onto 2.13 branch. It is there on 2.x branch and once we schedule nightlies for 2.14 after 2.13 release tomorrow we should see similar pattern as 3.0 on 2.x line, if in fact that is the offending commit.
from opensearch.
Commit at the tip of main branch for 03/21 run:
Light weight Transport action to verify local term before fetching cluster-state from remote
Commit at the tip of main branch for 03/22 run:
Catch task description error
Delta commits:
- Catch task description error
- Add release-notes for 2.13.0
- Add 2.14.0 to version & BWC
- Update the support version to 2.13 as term-check is merged to 2.x
- Fix Flaky SimpleQueryStringIT Tests
- Update supported version for the wait_for_completion parameter in open&clone&shrink&split APIs
- Integrate with CPU admission controller for cluster-manager Read API's.
- [Remote Migration] Changes for Primary Relocation during migration
As next steps, I will run the OpenSearch Benchmark against the suspect commits to narrow down. Thank You @rishabh6788 for discussion.
from opensearch.
The Indexing throughput degradation is seen only on 3.0.0, refer below screenshot taken from https://opensearch.org/benchmarks/
The version 2.12.0 and 2.13.0 performance numbers are close.
from opensearch.
I looked at the delta commits and one suspect is https://github.com/opensearch-project/OpenSearch/pull/12494/files,
@bbarani Do we have support to run the Nightly Benchmarks against a commit ? Can we run the HTTP corpus against the above commit ?
from opensearch.
@rishabh6788 can you help @khushbr ?
from opensearch.
The Indexing throughput degradation is seen only on 3.0.0, refer below screenshot taken from https://opensearch.org/benchmarks/
The version 2.12.0 and 2.13.0 performance numbers are close.
I still notice the regression (especially when security is disabled) in main branch
from opensearch.
Baseline with Light weight Transport action to verify local term before fetching cluster-state from remote
Min Throughput | Mean Throughput | Median Throughput | Max Throughput | |
---|---|---|---|---|
1 | 207733 | 220321 | 219700 | 232087 |
2 | 206696 | 215810 | 212435 | 228198 |
3 | 219180 | 228342 | 225731 | 240500 |
Uptill ([Remote Migration] Changes for Primary Relocation during migration ) OSB runs (in docs/s):
Min Throughput | Mean Throughput | Median Throughput | Max Throughput | |
---|---|---|---|---|
1 | 191314 | 200558 | 199832 | 210299 |
2 | 183816 | 197962 | 194975 | 216139 |
3 | 185448 | 193235 | 192683 | 200703 |
Uptill (Update supported version for the wait_for_completion parameter in open&clone&shrink&split APIs ) OSB runs(in docs/s):
Min Throughput | Mean Throughput | Median Throughput | Max Throughput | |
---|---|---|---|---|
1 | 187156 | 193726 | 193174 | 200601 |
2 | 188290 | 196304 | 195373 | 205012 |
3 | 191328 | 200612 | 199125 | 210423 |
Full Set Commits taken on 03/22 (Catch task description error) OSB runs(in docs/s):
Min Throughput | Mean Throughput | Median Throughput | Max Throughput | |
---|---|---|---|---|
1 | 192710 | 201019 | 199823 | 210869 |
2 | 191049 | 201927 | 199053 | 216989 |
3 | 193736 | 205920 | 203444 | 221371 |
from opensearch.
The OSB dashboard plots the Max of Mean (which for single run is Mean value) Throughput. For the runs on and prior to 03/21, the Mean throughput is > 200K
.
On running the same https_logs
workload in my setup, I see the Mean Throughput
value decline with the change [Remote Migration] Changes for Primary Relocation during migration , dropping to ~19K
(see table above)
from opensearch.
I see that the throughput has increased again for all of the benchmarks . We have added a change to memoize a value rather than looking into index settings for same : #12994 and backported it to 2.x (2.14) as well . The results of 2.14 and 3.0 are on par with 2.13 now.
from opensearch.
Related Issues (20)
- [Feature Request] Improve performance of sort by _id HOT 3
- [BUG] org.opensearch.indices.IndicesRequestCacheIT.testCacheCleanupWithDefaultSettings {p0={"search.concurrent_segment_search.enabled":"true"}} is flaky
- [Feature Request] Support mapping for object type and new settings for derived fields
- Refactor the writable warm index feature flag
- [Feature Request] Support for Experimental Codecs HOT 3
- [BUG] Incorrect docs.deleted count with Soft Delete Enabled HOT 1
- [Remote Cluster State] Add XContent serde for ClusterState.Custom
- [Remote Cluster State] Upload ephemeral cluster state objects(cluster blocks, nodes, customs) to remote HOT 1
- [Remote Cluster State] Download the cluster state objects asynchronously and in parallel HOT 1
- [Remote Cluster State] Add new transport call to send cluster state term and version HOT 1
- [BUG] XContentBuilder.toString() will close the builder itself HOT 1
- [BUG] IndicesRequestCacheIT timing out - abandoned because suite timeout was reached HOT 2
- [AUTOCUT] Gradle Check Failure on push to main HOT 1
- [BUG] org.opensearch.remotemigration.RemoteMigrationIndexMetadataUpdateIT.testIndexSettingsUpdatedEvenForMisconfiguredReplicas if flaky HOT 1
- [BUG] Cluster Manager task throttler blocks network threads HOT 1
- [Feature Request] Parallelise allocation decider executions to prevent high priority tasks from timing out
- [Feature Request] Optimise Throttling Allocation Decider
- [Feature Request] Evaluate removal of DATETIME_FORMATTER_CACHING_SETTING feature flag HOT 1
- Handle remote store custom metadata during docrep to remote migration
- [Feature Request] Cluster settings with restricted update permissions HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opensearch.