Comments (4)
I forked this repository and made a commit that replaces the old Jackson with the newer FasterXML/Jackson:
brunnsbe@ca36d5e
I however have problems running all the tests as quite many of them throws an OutOfMemoryException:
...
[ERROR] TestZkHelixPropertyStore.testSet » OutOfMemory unable to create new native thr...
[ERROR] TestZkHelixPropertyStore.testSetInvalidPath » OutOfMemory unable to create new...
[ERROR] TestZkHelixPropertyStore.testZkClientMonitor » OutOfMemory unable to create ne...
[ERROR] TestZkHelixPropertyStore.testZkTriggeredCallback » OutOfMemory unable to creat...
[ERROR] TestZkManagerWithAutoFallbackStore.testBasic » OutOfMemory unable to create ne...
[INFO]
[ERROR] Tests run: 866, Failures: 177, Errors: 0, Skipped: 103
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Apache Helix 0.9.1-SNAPSHOT:
[INFO]
[INFO] Apache Helix ....................................... SUCCESS [ 2.061 s]
[INFO] Apache Helix :: Core ............................... FAILURE [ 01:00 h]
[INFO] Apache Helix :: Admin Webapp ....................... SKIPPED
[INFO] Apache Helix :: Restful Interface .................. SKIPPED
[INFO] Apache Helix :: HelixAgent ......................... SKIPPED
[INFO] Apache Helix :: Recipes ............................ SKIPPED
[INFO] Apache Helix :: Recipes :: Rabbitmq Consumer Group . SKIPPED
[INFO] Apache Helix :: Recipes :: Rsync Replicated File Store SKIPPED
[INFO] Apache Helix :: Recipes :: distributed lock manager SKIPPED
[INFO] Apache Helix :: Recipes :: distributed task execution SKIPPED
[INFO] Apache Helix :: Recipes :: service discovery ....... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:00 h
[INFO] Finished at: 2019-08-13T09:23:19+03:00
[INFO] ------------------------------------------------------------------------
...
from helix.
Will take a look.
from helix.
Hi @brunnsbe,
Thanks so much for working on Helix!
We did a performance experiment profile and find for a specific use case -- Helix controller leadership switch, using FasterXML/jackson along with some other performance improv changes, we get round around ~3x faster performance result.
Baseline profile:
The org.apache.helix.zookeeper.datamodel.serializer.ZNRecordSerializer.deserialize took ~60% of the time for org.apache.helix.manager.zk.DistributedLeaderElection.onControllerChange, where onControllerChange overall takes ~60s. This function call is CPU heavy.
After changing to FasterXML/jackson and made the ObjectMapper static, ZNRecordSerializer.deserialize only take ~10% of the total time, where onControllerChange overall takes ~20s. It becomes IO bounded.
It is another evidence that we should replace org.codehaus.jackson with FasterXML/jackson.
Do you want to continue the work and create a PR :D?
Xiaoyuan
from helix.
BTW, we improved our unit test and you won't see that much OutOfMemoryException now.
from helix.
Related Issues (20)
- [Failed CI Test] testMarkEvacuationAfterEMM(org.apache.helix.integration.rebalancer.TestInstanceOperation) HOT 2
- [Failed CI Test] deleteFromResourceIdealState(org.apache.helix.rest.server.TestResourceAccessor) HOT 1
- [Failed CI Test] testGetClusters(org.apache.helix.rest.server.TestClusterAccessor) HOT 8
- [Failed CI Test] testDifferentTasks(org.apache.helix.integration.task.TestIndependentTaskRebalancer) HOT 6
- State transition not performed with shortest distance HOT 3
- [Failed CI Test] afterTest(org.apache.helix.rest.server.TestPartitionAssignmentAPI) HOT 6
- Support Simple Greedy Rebalance Strategy
- upgrade jackson-databind to 2.12.7.1 HOT 1
- Fix PreferenceList Ordering Changes during Maintenance Mode
- Provide JDK 1.8 (backward) compatibility of helix components
- NPE Issues when we add a WAGED resource without instances against the resource tag
- [Failed CI Test] testAggregateMetrics(org.apache.helix.monitoring.mbeans.TestClusterAggregateMetrics) HOT 1
- [Failed CI Test] testEnqueueJobsThrottle(org.apache.helix.task.TestTaskCreateThrottling) HOT 1
- [Failed CI Test] testNodeSwap(org.apache.helix.integration.rebalancer.TestInstanceOperation) HOT 13
- [Failed CI Test] testResourceThrottle(org.apache.helix.integration.TestPartitionMovementThrottle) HOT 1
- Add an API endpoint to allow a participant to set the current state of a partition stuck in any state to ERROR.
- Changing resource partition count via Helix Rest does not work reliably HOT 6
- [Failed CI Test] testHappyPathExtOpZkCacheBaseDataAccessor(org.apache.helix.manager.zk.TestZkCacheAsyncOpSingleThread) HOT 1
- [Failed CI Test] testInconsistentStateEventProcessing(org.apache.helix.integration.spectator.TestRoutingTableProviderFromCurrentStates) HOT 6
- [Failed CI Test] testTargetedTaskTwoCurrentStates(org.apache.helix.integration.task.TestTaskSchedulingTwoCurrentStates) HOT 30
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from helix.