Comments (13)
my pinot verision : 1.0.0
cpu : aws r7g (Graviton)
java version : 17
start script : apache-pinot-1.0.0-bin/bin/pinot-admin.sh QuickStart -type hybrid
This is the script used to insert the 3.0.0 version table.
All tables except exceptionTrace are normal and data collection is working well.
The log below is the message that appears when a segment is reloaded. It's a bit difficult to see because it's mixed in with other things.
2024/06/24 08:52:27.545 INFO [PinotHelixResourceManager] [grizzly-http-server-0] Sending reload message for segment: exceptionTrace__0__0__20240624T0630Z in table: exceptionTrace_REALTIME with forceDownload: false
2024/06/24 08:52:27.546 INFO [CriteriaEvaluator] [grizzly-http-server-0] Query returned 1 rows
2024/06/24 08:52:27.546 INFO [DefaultMessagingService] [grizzly-http-server-0] Send 1 messages with criteria instanceName=%resourceName=exceptionTrace_REALTIMEpartitionName=exceptionTrace__0__0__20240624T0630ZpartitionState=
2024/06/24 08:52:27.547 INFO [PinotHelixResourceManager] [grizzly-http-server-0] Sent 1 reload messages for segment: exceptionTrace__0__0__20240624T0630Z in table: exceptionTrace_REALTIME
2024/06/24 08:52:27.547 INFO [CallbackHandler] [ZkClient-EventThread-188-localhost:2123] 188 START: CallbackHandler 27, INVOKE /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES listener: org.apache.helix.controller.GenericHelixController@3b11deb6 type: CALLBACK
2024/06/24 08:52:27.547 INFO [CallbackHandler] [ZkClient-EventThread-188-localhost:2123] CallbackHandler 27 subscribing changes listener to path: /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES, callback type: CALLBACK, event types: [NodeChildrenChanged], listener: org.apache.helix.controller.GenericHelixController@3b11deb6, watchChild: false
2024/06/24 08:52:27.547 INFO [CallbackHandler] [ZkClient-EventThread-355-localhost:2123] 355 START: CallbackHandler 29, INVOKE /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES listener: org.apache.helix.messaging.handling.HelixTaskExecutor@5fd6d31f type: CALLBACK
2024/06/24 08:52:27.547 INFO [CallbackHandler] [ZkClient-EventThread-355-localhost:2123] CallbackHandler 29 subscribing changes listener to path: /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES, callback type: CALLBACK, event types: [NodeChildrenChanged], listener: org.apache.helix.messaging.handling.HelixTaskExecutor@5fd6d31f, watchChild: false
2024/06/24 08:52:27.548 INFO [ControllerResponseFilter] [grizzly-http-server-0] Handled request from 10.220.132.137 POST http://-/segments/exceptionTrace_REALTIME/exceptionTrace__0__0__20240624T0630Z/reload, content-type application/json; charset=UTF-8 status code 200 OK
2024/06/24 08:52:27.548 INFO [CallbackHandler] [ZkClient-EventThread-188-localhost:2123] CallbackHandler27, Subscribing to path: /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES took: 1
2024/06/24 08:52:27.548 INFO [GenericHelixController] [ZkClient-EventThread-188-localhost:2123] START: GenericClusterController.onMessage() for cluster QuickStartCluster
2024/06/24 08:52:27.548 INFO [CallbackHandler] [ZkClient-EventThread-355-localhost:2123] CallbackHandler29, Subscribing to path: /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES took: 0
2024/06/24 08:52:27.548 INFO [GenericHelixController] [ZkClient-EventThread-188-localhost:2123] END: GenericClusterController.onMessage() for cluster QuickStartCluster
2024/06/24 08:52:27.548 INFO [CallbackHandler] [ZkClient-EventThread-188-localhost:2123] 188 END:INVOKE CallbackHandler 27, /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES listener: org.apache.helix.controller.GenericHelixController@3b11deb6 type: CALLBACK Took: 1ms
2024/06/24 08:52:27.549 INFO [GenericHelixController] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Controller stopping periodical rebalance timer at period 9223372036854775807
2024/06/24 08:52:27.549 INFO [GenericHelixController] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Controller stopping periodical rebalance timer at period 9223372036854775807
2024/06/24 08:52:27.549 INFO [GenericHelixController] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] START: Invoking TASK controller pipeline for cluster: QuickStartCluster. Event type: MessageChange, ID: 2f323e54_TASK. Event session ID: 10004f0fb490005
2024/06/24 08:52:27.549 INFO [GenericHelixController] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] START: Invoking DEFAULT controller pipeline for cluster: QuickStartCluster. Event type: MessageChange, ID: 2f323e54_DEFAULT. Event session ID: 10004f0fb490005
2024/06/24 08:52:27.549 INFO [BaseControllerDataProvider] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Event 2f323e54_DEFAULT : No ideal state change for QuickStartCluster cluster, DEFAULT pipeline
2024/06/24 08:52:27.549 INFO [BaseControllerDataProvider] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Event 2f323e54_TASK : No ideal state change for QuickStartCluster cluster, TASK pipeline
2024/06/24 08:52:27.549 INFO [BaseControllerDataProvider] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Event 2f323e54_DEFAULT : No live instance change for QuickStartCluster cluster, DEFAULT pipeline
2024/06/24 08:52:27.549 INFO [BaseControllerDataProvider] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Event 2f323e54_TASK : No live instance change for QuickStartCluster cluster, TASK pipeline
2024/06/24 08:52:27.549 INFO [BaseControllerDataProvider] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Event 2f323e54_TASK : No instance config change for QuickStartCluster cluster, TASK pipeline
2024/06/24 08:52:27.549 INFO [BaseControllerDataProvider] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Event 2f323e54_DEFAULT : No instance config change for QuickStartCluster cluster, DEFAULT pipeline
2024/06/24 08:52:27.549 INFO [BaseControllerDataProvider] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Event 2f323e54_DEFAULT : No resource config change for QuickStartCluster cluster, DEFAULT pipeline
2024/06/24 08:52:27.549 INFO [BaseControllerDataProvider] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Event 2f323e54_TASK : No resource config change for QuickStartCluster cluster, TASK pipeline
2024/06/24 08:52:27.550 INFO [MessageLatencyMonitor] [ZkClient-EventThread-355-localhost:2123] The latency of message 8a579eec-c767-4459-9aa6-2edf5b9e45f9 is 5 ms
2024/06/24 08:52:27.551 INFO [PropertyCache] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Event QuickStartCluster::DEFAULT::2f323e54_DEFAULT : Refreshed 9 property StateModelDefinition took 2 ms. Selective: false
2024/06/24 08:52:27.551 INFO [HelixTaskExecutor] [ZkClient-EventThread-355-localhost:2123] Scheduling message 8a579eec-c767-4459-9aa6-2edf5b9e45f9: exceptionTrace_REALTIME:exceptionTrace__0__0__20240624T0630Z, null->null
2024/06/24 08:52:27.551 INFO [HelixTaskExecutor] [ZkClient-EventThread-355-localhost:2123] Submit task: 8a579eec-c767-4459-9aa6-2edf5b9e45f9 to pool: java.util.concurrent.ThreadPoolExecutor@50eeed6e[Running, pool size = 33, active threads = 0, queued tasks = 0, completed tasks = 33]
2024/06/24 08:52:27.551 INFO [HelixTaskExecutor] [ZkClient-EventThread-355-localhost:2123] Message: 8a579eec-c767-4459-9aa6-2edf5b9e45f9 handling task scheduled
2024/06/24 08:52:27.551 INFO [CallbackHandler] [ZkClient-EventThread-355-localhost:2123] 355 END:INVOKE CallbackHandler 29, /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES listener: org.apache.helix.messaging.handling.HelixTaskExecutor@5fd6d31f type: CALLBACK Took: 4ms
2024/06/24 08:52:27.552 INFO [PropertyCache] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Event QuickStartCluster::TASK::2f323e54_TASK : Refreshed 9 property StateModelDefinition took 3 ms. Selective: false
2024/06/24 08:52:27.552 INFO [HelixTask] [HelixTaskExecutor-message_handle_thread_233] handling task: 8a579eec-c767-4459-9aa6-2edf5b9e45f9 begin, at: 1719219147552
2024/06/24 08:52:27.552 INFO [ConstraintItemBuilder] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] overwrite existing constraint-value. old-value: null, new-value: 100000
2024/06/24 08:52:27.552 INFO [PropertyCache] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Event QuickStartCluster::DEFAULT::2f323e54_DEFAULT : Refreshed 1 property ClusterConstraint took 1 ms. Selective: false
2024/06/24 08:52:27.552 INFO [exceptionTrace_REALTIME-SegmentReloadMessageHandler] [HelixTaskExecutor-message_handle_thread_233] Handling message: ZnRecord=8a579eec-c767-4459-9aa6-2edf5b9e45f9, {CREATE_TIMESTAMP=1719219147545, EXECUTE_START_TIMESTAMP=1719219147552, MSG_ID=8a579eec-c767-4459-9aa6-2edf5b9e45f9, MSG_STATE=new, MSG_SUBTYPE=RELOAD_SEGMENT, MSG_TYPE=USER_DEFINE_MSG, PARTITION_NAME=exceptionTrace__0__0__20240624T0630Z, RESOURCE_NAME=exceptionTrace_REALTIME, RETRY_COUNT=0, SRC_CLUSTER=QuickStartCluster, SRC_INSTANCE_TYPE=PARTICIPANT, SRC_NAME=Controller_10.220.207.212_9000, TGT_NAME=Server_10.220.207.212_7050, TGT_SESSION_ID=10004f0fb490015, TIMEOUT=-1, forceDownload=false}{}{segmentNames=[exceptionTrace__0__0__20240624T0630Z]}, Stat=Stat {_version=0, _creationTime=1719219147546, _modifiedTime=1719219147546, _ephemeralOwner=0}
2024/06/24 08:52:27.552 INFO [HelixInstanceDataManager] [HelixTaskExecutor-message_handle_thread_233] Reloading multiple segments: [exceptionTrace__0__0__20240624T0630Z] in table: exceptionTrace_REALTIME
2024/06/24 08:52:27.552 WARN [HelixInstanceDataManager] [HelixTaskExecutor-message_handle_thread_233] Failed to get segment data manager for segments: [exceptionTrace__0__0__20240624T0630Z] of table: org.apache.pinot.core.data.manager.realtime.RealtimeTableDataManager@467f8cc1, skipping reloading them
2024/06/24 08:52:27.552 INFO [ConstraintItemBuilder] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] overwrite existing constraint-value. old-value: null, new-value: 100000
2024/06/24 08:52:27.552 INFO [PropertyCache] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Event QuickStartCluster::TASK::2f323e54_TASK : Refreshed 1 property ClusterConstraint took 0 ms. Selective: false
2024/06/24 08:52:27.553 INFO [InstanceMessagesCache] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] START: InstanceMessagesCache.refresh()
2024/06/24 08:52:27.553 INFO [InstanceMessagesCache] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] START: InstanceMessagesCache.refresh()
2024/06/24 08:52:27.553 INFO [HelixInstanceDataManager] [HelixTaskExecutor-message_handle_thread_233] Reloaded segments with metadata in table: exceptionTrace_REALTIME. Duration: 1
2024/06/24 08:52:27.554 INFO [HelixTask] [HelixTaskExecutor-message_handle_thread_233] Message 8a579eec-c767-4459-9aa6-2edf5b9e45f9 completed.
2024/06/24 08:52:27.554 INFO [InstanceMessagesCache] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END: InstanceMessagesCache.refresh(), 1 of Messages read from ZooKeeper. took 1 ms.
2024/06/24 08:52:27.554 INFO [InstanceMessagesCache] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Start to refresh stale message cache
2024/06/24 08:52:27.555 WARN [ZkBaseDataAccessor] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Fail to read record for paths: {/QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES/8a579eec-c767-4459-9aa6-2edf5b9e45f9=-101}
2024/06/24 08:52:27.555 INFO [InstanceMessagesCache] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] END: InstanceMessagesCache.refresh(), 1 of Messages read from ZooKeeper. took 2 ms.
2024/06/24 08:52:27.555 INFO [InstanceMessagesCache] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Start to refresh stale message cache
2024/06/24 08:52:27.555 INFO [HelixTask] [HelixTaskExecutor-message_handle_thread_233] Delete message 8a579eec-c767-4459-9aa6-2edf5b9e45f9 from zk!
2024/06/24 08:52:27.555 INFO [HelixTaskExecutor] [HelixTaskExecutor-message_handle_thread_233] message finished: 8a579eec-c767-4459-9aa6-2edf5b9e45f9, took 3
2024/06/24 08:52:27.555 INFO [HelixTask] [HelixTaskExecutor-message_handle_thread_233] Message: 8a579eec-c767-4459-9aa6-2edf5b9e45f9 (parent: null) handling task for exceptionTrace_REALTIME:exceptionTrace__0__0__20240624T0630Z completed at: 1719219147555, results: true. FrameworkTime: 2 ms; HandlerTime: 1 ms.
2024/06/24 08:52:27.555 INFO [CallbackHandler] [ZkClient-EventThread-355-localhost:2123] 355 START: CallbackHandler 29, INVOKE /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES listener: org.apache.helix.messaging.handling.HelixTaskExecutor@5fd6d31f type: CALLBACK
2024/06/24 08:52:27.555 INFO [CallbackHandler] [ZkClient-EventThread-355-localhost:2123] CallbackHandler 29 subscribing changes listener to path: /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES, callback type: CALLBACK, event types: [NodeChildrenChanged], listener: org.apache.helix.messaging.handling.HelixTaskExecutor@5fd6d31f, watchChild: false
2024/06/24 08:52:27.555 INFO [CallbackHandler] [ZkClient-EventThread-188-localhost:2123] 188 START: CallbackHandler 27, INVOKE /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES listener: org.apache.helix.controller.GenericHelixController@3b11deb6 type: CALLBACK
2024/06/24 08:52:27.555 INFO [CallbackHandler] [ZkClient-EventThread-188-localhost:2123] CallbackHandler 27 subscribing changes listener to path: /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES, callback type: CALLBACK, event types: [NodeChildrenChanged], listener: org.apache.helix.controller.GenericHelixController@3b11deb6, watchChild: false
2024/06/24 08:52:27.555 INFO [CallbackHandler] [ZkClient-EventThread-355-localhost:2123] CallbackHandler29, Subscribing to path: /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES took: 0
2024/06/24 08:52:27.556 INFO [CallbackHandler] [ZkClient-EventThread-188-localhost:2123] CallbackHandler27, Subscribing to path: /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES took: 1
2024/06/24 08:52:27.556 INFO [GenericHelixController] [ZkClient-EventThread-188-localhost:2123] START: GenericClusterController.onMessage() for cluster QuickStartCluster
2024/06/24 08:52:27.556 INFO [HelixTaskExecutor] [ZkClient-EventThread-355-localhost:2123] No Messages to process
2024/06/24 08:52:27.556 INFO [CallbackHandler] [ZkClient-EventThread-355-localhost:2123] 355 END:INVOKE CallbackHandler 29, /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES listener: org.apache.helix.messaging.handling.HelixTaskExecutor@5fd6d31f type: CALLBACK Took: 1ms
2024/06/24 08:52:27.556 INFO [GenericHelixController] [ZkClient-EventThread-188-localhost:2123] END: GenericClusterController.onMessage() for cluster QuickStartCluster
2024/06/24 08:52:27.556 INFO [CallbackHandler] [ZkClient-EventThread-188-localhost:2123] 188 END:INVOKE CallbackHandler 27, /QuickStartCluster/INSTANCES/Server_10.220.207.212_7050/MESSAGES listener: org.apache.helix.controller.GenericHelixController@3b11deb6 type: CALLBACK Took: 1ms
2024/06/24 08:52:27.557 INFO [AbstractDataCache] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Event QuickStartCluster::DEFAULT::2f323e54_DEFAULT : 0 properties refreshed from ZK.
2024/06/24 08:52:27.557 INFO [ParticipantStateCache] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Event QuickStartCluster::DEFAULT::2f323e54_DEFAULT : END: participantStateCache.refresh() for cluster QuickStartCluster, started at : 1719219147554, took 3 ms
2024/06/24 08:52:27.557 INFO [InstanceMessagesCache] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END: updateRelayMessages(), 0 of valid relay messages in cache, took 0 ms.
2024/06/24 08:52:27.557 INFO [ResourceControllerDataProvider] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Event 2f323e54_DEFAULT : No customized state config change for QuickStartCluster cluster, DEFAULT pipeline
2024/06/24 08:52:27.557 INFO [AbstractDataCache] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Event QuickStartCluster::DEFAULT::2f323e54_DEFAULT : 0 properties refreshed from ZK.
2024/06/24 08:52:27.557 INFO [ParticipantStateCache] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Event QuickStartCluster::DEFAULT::2f323e54_DEFAULT : END: participantStateCache.refresh() for cluster QuickStartCluster, started at : 1719219147557, took 0 ms
2024/06/24 08:52:27.557 INFO [ResourceControllerDataProvider] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Event 2f323e54_DEFAULT : END: ResourceControllerDataProvider.refresh() for cluster QuickStartCluster, started at 1719219147549 took 8 for DEFAULT pipeline
2024/06/24 08:52:27.557 INFO [Pipeline] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END ReadClusterDataStage for DEFAULT pipeline for cluster QuickStartCluster. took: 8 ms for event 2f323e54_DEFAULT
2024/06/24 08:52:27.557 INFO [Pipeline] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END ResourceComputationStage for DEFAULT pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_DEFAULT
2024/06/24 08:52:27.557 INFO [Pipeline] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END ResourceValidationStage for DEFAULT pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_DEFAULT
2024/06/24 08:52:27.557 INFO [Pipeline] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END CurrentStateComputationStage for DEFAULT pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_DEFAULT
2024/06/24 08:52:27.557 INFO [Pipeline] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END CustomizedStateComputationStage for DEFAULT pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_DEFAULT
2024/06/24 08:52:27.557 INFO [Pipeline] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END TopStateHandoffReportStage for DEFAULT pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_DEFAULT
2024/06/24 08:52:27.557 INFO [WagedRebalancer] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Start computing new ideal states for resources: []
2024/06/24 08:52:27.558 INFO [WagedRebalancer] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Start calculating the new best possible assignment.
2024/06/24 08:52:27.558 INFO [WagedRebalancer] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Start calculating for an assignment with algorithm ConstraintBasedAlgorithm
2024/06/24 08:52:27.558 INFO [WagedRebalancer] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Finish calculating an assignment with algorithm ConstraintBasedAlgorithm. Took: 0 ms.
2024/06/24 08:52:27.558 INFO [WagedRebalancer] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Finish calculating the new best possible assignment.
2024/06/24 08:52:27.558 INFO [WagedRebalancer] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Finish computing new ideal states for resources: []
2024/06/24 08:52:27.558 INFO [Pipeline] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END BestPossibleStateCalcStage for DEFAULT pipeline for cluster QuickStartCluster. took: 1 ms for event 2f323e54_DEFAULT
2024/06/24 08:52:27.558 INFO [Pipeline] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END MessageGenerationPhase for DEFAULT pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_DEFAULT
2024/06/24 08:52:27.558 INFO [Pipeline] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END MessageSelectionStage for DEFAULT pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_DEFAULT
2024/06/24 08:52:27.558 INFO [StateTransitionThrottleController] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] No throttle config is set!
2024/06/24 08:52:27.558 INFO [Pipeline] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END IntermediateStateCalcStage for DEFAULT pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_DEFAULT
2024/06/24 08:52:27.558 INFO [Pipeline] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END MessageThrottleStage for DEFAULT pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_DEFAULT
2024/06/24 08:52:27.558 INFO [AbstractAsyncBaseStage] [MaintenanceRecoveryWorker-QuickStartCluster] START AsyncProcess: DEFAULT::MaintenanceRecoveryStage
2024/06/24 08:52:27.558 INFO [AbstractAsyncBaseStage] [MaintenanceRecoveryWorker-QuickStartCluster] END AsyncProcess: DEFAULT::MaintenanceRecoveryStage, took 0 ms
2024/06/24 08:52:27.558 INFO [AbstractAsyncBaseStage] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Submitted asynchronous DEFAULT::MaintenanceRecoveryStage task to worker
2024/06/24 08:52:27.558 INFO [Pipeline] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END MaintenanceRecoveryStage for DEFAULT pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_DEFAULT
2024/06/24 08:52:27.558 INFO [Pipeline] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END ResourceMessageDispatchStage for DEFAULT pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_DEFAULT
2024/06/24 08:52:27.558 INFO [AbstractAsyncBaseStage] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Submitted asynchronous DEFAULT::PersistAssignmentStage task to worker
2024/06/24 08:52:27.558 INFO [Pipeline] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END PersistAssignmentStage for DEFAULT pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_DEFAULT
2024/06/24 08:52:27.558 INFO [AbstractAsyncBaseStage] [PersistAssignmentWorker-QuickStartCluster] START AsyncProcess: DEFAULT::PersistAssignmentStage
2024/06/24 08:52:27.559 INFO [AbstractAsyncBaseStage] [TargetExternalViewCalcWorker-QuickStartCluster] START AsyncProcess: DEFAULT::TargetExteralViewCalcStage
2024/06/24 08:52:27.559 INFO [AbstractAsyncBaseStage] [PersistAssignmentWorker-QuickStartCluster] END AsyncProcess: DEFAULT::PersistAssignmentStage, took 1 ms
2024/06/24 08:52:27.559 INFO [AbstractAsyncBaseStage] [TargetExternalViewCalcWorker-QuickStartCluster] END AsyncProcess: DEFAULT::TargetExteralViewCalcStage, took 0 ms
2024/06/24 08:52:27.559 INFO [AbstractAsyncBaseStage] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Submitted asynchronous DEFAULT::TargetExteralViewCalcStage task to worker
2024/06/24 08:52:27.559 INFO [Pipeline] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END TargetExteralViewCalcStage for DEFAULT pipeline for cluster QuickStartCluster. took: 1 ms for event 2f323e54_DEFAULT
2024/06/24 08:52:27.559 INFO [GenericHelixController] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] END: Invoking DEFAULT controller pipeline for event MessageChange::2f323e54_DEFAULT for cluster QuickStartCluster, took 10 ms
2024/06/24 08:52:27.559 INFO [GenericHelixController] [HelixController-pipeline-default-QuickStartCluster-(2f323e54_DEFAULT)] Callback time for event: MessageChange took: 1 ms
InQueue time for event: MessageChange took: 1 ms
TotalProcessed time for event: MessageChange took: 10 ms
2024/06/24 08:52:27.559 INFO [GenericHelixController] [HelixController-pipeline-default-QuickStartCluster-(0ba2c2af_DEFAULT)] Controller stopping periodical rebalance timer at period 9223372036854775807
2024/06/24 08:52:27.559 INFO [GenericHelixController] [HelixController-pipeline-default-QuickStartCluster-(0ba2c2af_DEFAULT)] START: Invoking DEFAULT controller pipeline for cluster: QuickStartCluster. Event type: MessageChange, ID: 0ba2c2af_DEFAULT. Event session ID: 10004f0fb490005
2024/06/24 08:52:27.559 INFO [BaseControllerDataProvider] [HelixController-pipeline-default-QuickStartCluster-(0ba2c2af_DEFAULT)] Event 0ba2c2af_DEFAULT : No ideal state change for QuickStartCluster cluster, DEFAULT pipeline
2024/06/24 08:52:27.559 INFO [BaseControllerDataProvider] [HelixController-pipeline-default-QuickStartCluster-(0ba2c2af_DEFAULT)] Event 0ba2c2af_DEFAULT : No live instance change for QuickStartCluster cluster, DEFAULT pipeline
2024/06/24 08:52:27.559 INFO [BaseControllerDataProvider] [HelixController-pipeline-default-QuickStartCluster-(0ba2c2af_DEFAULT)] Event 0ba2c2af_DEFAULT : No instance config change for QuickStartCluster cluster, DEFAULT pipeline
2024/06/24 08:52:27.559 INFO [BaseControllerDataProvider] [HelixController-pipeline-default-QuickStartCluster-(0ba2c2af_DEFAULT)] Event 0ba2c2af_DEFAULT : No resource config change for QuickStartCluster cluster, DEFAULT pipeline
2024/06/24 08:52:27.560 INFO [AbstractDataCache] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Event QuickStartCluster::TASK::2f323e54_TASK : 0 properties refreshed from ZK.
2024/06/24 08:52:27.560 INFO [ParticipantStateCache] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Event QuickStartCluster::TASK::2f323e54_TASK : END: participantStateCache.refresh() for cluster QuickStartCluster, started at : 1719219147555, took 5 ms
2024/06/24 08:52:27.560 INFO [InstanceMessagesCache] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] END: updateRelayMessages(), 0 of valid relay messages in cache, took 0 ms.
2024/06/24 08:52:27.560 INFO [PropertyCache] [HelixController-pipeline-default-QuickStartCluster-(0ba2c2af_DEFAULT)] Event QuickStartCluster::DEFAULT::0ba2c2af_DEFAULT : Refreshed 9 property StateModelDefinition took 1 ms. Selective: false
2024/06/24 08:52:27.561 INFO [ConstraintItemBuilder] [HelixController-pipeline-default-QuickStartCluster-(0ba2c2af_DEFAULT)] overwrite existing constraint-value. old-value: null, new-value: 100000
2024/06/24 08:52:27.561 INFO [PropertyCache] [HelixController-pipeline-default-QuickStartCluster-(0ba2c2af_DEFAULT)] Event QuickStartCluster::DEFAULT::0ba2c2af_DEFAULT : Refreshed 1 property ClusterConstraint took 1 ms. Selective: false
2024/06/24 08:52:27.561 INFO [AbstractDataCache] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Event QuickStartCluster::TASK::2f323e54_TASK : 0 properties refreshed from ZK.
2024/06/24 08:52:27.561 INFO [ParticipantStateCache] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Event QuickStartCluster::TASK::2f323e54_TASK : END: participantStateCache.refresh() for cluster QuickStartCluster, started at : 1719219147560, took 1 ms
2024/06/24 08:52:27.562 INFO [InstanceMessagesCache] [HelixController-pipeline-default-QuickStartCluster-(0ba2c2af_DEFAULT)] START: InstanceMessagesCache.refresh()
2024/06/24 08:52:27.572 INFO [WorkflowControllerDataProvider] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Event 2f323e54_TASK : END: WorkflowControllerDataProvider.refresh() for cluster QuickStartCluster, started at 1719219147549 took 23 for TASK pipeline
2024/06/24 08:52:27.572 INFO [Pipeline] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] END ReadClusterDataStage for TASK pipeline for cluster QuickStartCluster. took: 23 ms for event 2f323e54_TASK
2024/06/24 08:52:27.572 INFO [Pipeline] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] END ResourceComputationStage for TASK pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_TASK
2024/06/24 08:52:27.572 INFO [Pipeline] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] END ResourceValidationStage for TASK pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_TASK
2024/06/24 08:52:27.573 INFO [Pipeline] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] END CurrentStateComputationStage for TASK pipeline for cluster QuickStartCluster. took: 1 ms for event 2f323e54_TASK
2024/06/24 08:52:27.573 INFO [AssignableInstanceManager] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] AssignableInstanceManager built AssignableInstances from scratch based on CurrentState.
2024/06/24 08:52:27.573 INFO [AssignableInstanceManager] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Current quota capacity: {"Broker_10.220.207.212_8000":{"TASK_EXEC_THREAD":{"DEFAULT":"0/40"}},"Server_10.220.207.212_7050":{"TASK_EXEC_THREAD":{"DEFAULT":"0/40"}},"Minion_10.220.207.212_6000":{"TASK_EXEC_THREAD":{"DEFAULT":"0/40"}},"Controller_10.220.207.212_9000":{"TASK_EXEC_THREAD":{"DEFAULT":"0/40"}}}
2024/06/24 08:52:27.573 INFO [ControllerResponseFilter] [grizzly-http-server-1] Handled request from 10.220.132.137 GET http://-/tables/exceptionTrace_REALTIME/externalview, content-type null status code 200 OK
2024/06/24 08:52:27.574 INFO [Pipeline] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] END TaskSchedulingStage for TASK pipeline for cluster QuickStartCluster. took: 1 ms for event 2f323e54_TASK
2024/06/24 08:52:27.574 INFO [TaskPersistDataStage] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] START TaskPersistDataStage.process()
2024/06/24 08:52:27.574 INFO [TaskPersistDataStage] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] END TaskPersistDataStage.process() for cluster QuickStartCluster took 0 ms
2024/06/24 08:52:27.574 INFO [Pipeline] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] END TaskPersistDataStage for TASK pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_TASK
2024/06/24 08:52:27.574 INFO [AbstractAsyncBaseStage] [TaskJobPurgeWorker-QuickStartCluster] START AsyncProcess: TASK::TaskGarbageCollectionStage
2024/06/24 08:52:27.574 INFO [AbstractAsyncBaseStage] [TaskJobPurgeWorker-QuickStartCluster] END AsyncProcess: TASK::TaskGarbageCollectionStage, took 0 ms
2024/06/24 08:52:27.574 INFO [AbstractAsyncBaseStage] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] Submitted asynchronous TASK::TaskGarbageCollectionStage task to worker
2024/06/24 08:52:27.574 INFO [Pipeline] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] END TaskGarbageCollectionStage for TASK pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_TASK
2024/06/24 08:52:27.574 INFO [InstanceMessagesCache] [HelixController-pipeline-default-QuickStartCluster-(0ba2c2af_DEFAULT)] END: InstanceMessagesCache.refresh(), 0 of Messages read from ZooKeeper. took 12 ms.
2024/06/24 08:52:27.574 INFO [InstanceMessagesCache] [HelixController-pipeline-default-QuickStartCluster-(0ba2c2af_DEFAULT)] Start to refresh stale message cache
2024/06/24 08:52:27.574 INFO [Pipeline] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] END MessageGenerationPhase for TASK pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_TASK
2024/06/24 08:52:27.574 INFO [Pipeline] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] END TaskMessageDispatchStage for TASK pipeline for cluster QuickStartCluster. took: 0 ms for event 2f323e54_TASK
2024/06/24 08:52:27.574 INFO [GenericHelixController] [HelixController-pipeline-task-QuickStartCluster-(2f323e54_TASK)] END: Invoking TASK controller pipeline for event MessageChange::2f323e54_TASK for cluster QuickStartCluster, took 25 ms
2024/06/24 08:52:27.574 INFO [GenericHelixController] [HelixController-pipeline-task-QuickStartCluster-(0ba2c2af_TASK)] Controller stopping periodical rebalance timer at period 9223372036854775807
2024/06/24 08:52:27.574 INFO [GenericHelixController] [HelixController-pipeline-task-QuickStartCluster-(0ba2c2af_TASK)] START: Invoking TASK controller pipeline for cluster: QuickStartCluster. Event type: MessageChange, ID: 0ba2c2af_TASK. Event session ID: 10004f0fb490005
2024/06/24 08:52:27.574 INFO [BaseControllerDataProvider] [HelixController-pipeline-task-QuickStartCluster-(0ba2c2af_TASK)] Event 0ba2c2af_TASK : No ideal state change for QuickStartCluster cluster, TASK pipeline
2024/06/24 08:52:27.574 INFO [BaseControllerDataProvider] [HelixController-pipeline-task-QuickStartCluster-(0ba2c2af_TASK)] Event 0ba2c2af_TASK : No live instance change for QuickStartCluster cluster, TASK pipeline
2024/06/24 08:52:27.574 INFO [BaseControllerDataProvider] [HelixController-pipeline-task-QuickStartCluster-(0ba2c2af_TASK)] Event 0ba2c2af_TASK : No instance config change for QuickStartCluster cluster, TASK pipeline
2024/06/24 08:52:27.574 INFO [BaseControllerDataProvider] [HelixController-pipeline-task-QuickStartCluster-(0ba2c2af_TASK)] Event 0ba2c2af_TASK : No resource config change for QuickStartCluster cluster, TASK pipeline
2024/06/24 08:52:27.575 ERROR [CompletionServiceHelper] [grizzly-http-server-3] Server: Server_10.220.207.212_7050 returned error: 404
2024/06/24 08:52:27.575 WARN [CompletionServiceHelper] [grizzly-http-server-3] Finished reading information for table: exceptionTrace_REALTIME with 0/1 server responses
2024/06/24 08:52:27.575 INFO [ControllerResponseFilter] [grizzly-http-server-3] Handled request from 10.220.132.137 GET http://-/segments/exceptionTrace_REALTIME/exceptionTrace__0__0__20240624T0630Z/metadata?columns=*, content-type null status code 200 OK
2024/06/24 08:52:27.576 INFO [DebugResource] [grizzly-http-server-2] Reading segments debug info from servers: [Server_10.220.207.212_7050] for table: exceptionTrace_REALTIME
2024/06/24 08:52:27.577 INFO [CompletionServiceHelper] [grizzly-http-server-2] Finished reading information for table: exceptionTrace_REALTIME
from pinpoint.
While testing, I checked the log below. Would this help?
2024/06/24 09:36:40.188 INFO [HelixInstanceDataManager] [HelixTaskExecutor-message_handle_thread_71] Adding segment: exceptionTraceV1__0__0__20240624T0936Z to table: exceptionTraceV1_REALTIME
2024/06/24 09:36:40.190 INFO [BaseTableDataManager] [HelixTaskExecutor-message_handle_thread_71] Default time: 0 does not comply with format: 1:MILLISECONDS:EPOCH, using creation time: 1719221800160 as the default time for table: exceptionTraceV1_REALTIME
2024/06/24 09:36:40.190 INFO [LLRealtimeSegmentDataManager_exceptionTraceV1__0__0__20240624T0936Z] [HelixTaskExecutor-message_handle_thread_71] Setting sorted column name: applicationName from RealtimeDataResourceZKMetadata for segment exceptionTraceV1__0__0__20240624T0936Z
2024/06/24 09:36:40.190 ERROR [HelixStateTransitionHandler] [HelixTaskExecutor-message_handle_thread_71] Exception while executing a state transition task exceptionTraceV1__0__0__20240624T0936Z
java.lang.reflect.InvocationTargetException: null
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:350) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:278) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:97) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:49) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:840) [?:?]
Caused by: java.lang.UnsatisfiedLinkError: 'void com.yscope.clp.compressorfrontend.MessageEncoder.setVariableHandlingRuleVersions(byte[], int, byte[], int)'
at com.yscope.clp.compressorfrontend.MessageEncoder.setVariableHandlingRuleVersions(Native Method) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at com.yscope.clp.compressorfrontend.MessageEncoder.(MessageEncoder.java:36) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.apache.pinot.plugin.inputformat.clplog.CLPLogRecordExtractor.init(CLPLogRecordExtractor.java:91) ~[pinot-clp-log-1.0.0-shaded.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.apache.pinot.plugin.inputformat.clplog.CLPLogMessageDecoder.init(CLPLogMessageDecoder.java:61) ~[pinot-clp-log-1.0.0-shaded.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.apache.pinot.spi.stream.StreamDecoderProvider.create(StreamDecoderProvider.java:45) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.(LLRealtimeSegmentDataManager.java:1433) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.apache.pinot.core.data.manager.realtime.RealtimeTableDataManager.addSegment(RealtimeTableDataManager.java:444) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.apache.pinot.server.starter.helix.HelixInstanceDataManager.addRealtimeSegment(HelixInstanceDataManager.java:230) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.apache.pinot.server.starter.helix.SegmentOnlineOfflineStateModelFactory$SegmentOnlineOfflineStateModel.onBecomeConsumingFromOffline(SegmentOnlineOfflineStateModelFactory.java:80) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
... 12 more
2024/06/24 09:36:40.191 ERROR [StateModel] [HelixTaskExecutor-message_handle_thread_71] Default rollback method invoked on error. Error Code: ERROR
2024/06/24 09:36:40.192 ERROR [HelixTask] [HelixTaskExecutor-message_handle_thread_71] Message execution failed. msgId: 26fa39a8-716d-4668-a2b9-c1fedc702192, errorMsg: java.lang.reflect.InvocationTargetException
from pinpoint.
I'm glad to hear that the issue has been resolved.
Here are the details related to the issue you raised:
Apache Pinot utilizes the clp-ffi-java
library internally for its CLP-related functionalities. This library currently supports only the linux/amd64 OS X/x86-64 architecture, which is the root cause of the problem you are experiencing.
According to the issue linked below, it appears that there are plans to support binaries for multiple platforms, including aarch64, in the near future:
y-scope/clp-ffi-java#46
from pinpoint.
Thank you for pointing out the configuration error. To address this issue, I will modify the existing table and schema files.
Based on your feedback, it seems that changing the incorrect table name (exceptionTraceV1
) to exceptionTrace
resolved the error. Is that correct?
If there are any further issues, please let me know.
from pinpoint.
I need some other help.
When inserting a table with the applied schema, a segment error occurs as shown below.
All other tables are normal, but only exceptionTrace is occurring.
We are using version 1.0.0 of pinot and are operating in hybrid mode of QuickStartCluster.
from pinpoint.
The segment might not be saved correctly due to the schema change.
For more information https://docs.pinot.apache.org/users/tutorials/schema-evolution.
Try using the Reload Table Segments API to reload table segments.
Or the Delete Table Segment API to delete segment exceptionTrace__0__0__20240624T0121Z
.
These actions can be done with the Swagger Pinot Controller API.
from pinpoint.
I've tried reloading segments, rebalance servers, and deleting and regenerating segments, but all I get is a message that the segment is not normal.
from pinpoint.
I am not sure if the segment was deleted properly because the segment name is the same.
Can you explain more about how you regenerate segments?
These are some solution with more details
delete segment
-
(optional) Use forceCommit
/tables/{tableName}/forceCommit
to create a new segment and check if it is created correctly.
If you need to restore realtime segment
https://www.markhneedham.com/blog/2021/12/06/apache-pinot-copy-segment-new-table/
- download segment from deep storage(controller) or server
- Copy segment to offline table
from pinpoint.
Thanks for your guide.
Like your guide, I proceeded as follows.
![image](https://private-user-images.githubusercontent.com/80370489/342219456-2c4caee1-0512-4760-9f61-c8fc71103095.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM3OTgxMTAsIm5iZiI6MTcyMzc5NzgxMCwicGF0aCI6Ii84MDM3MDQ4OS8zNDIyMTk0NTYtMmM0Y2FlZTEtMDUxMi00NzYwLTlmNjEtYzhmYzcxMTAzMDk1LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE2VDA4NDMzMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTZiZTdhZTJmM2MzY2MxZDU2MGEzOWRhZmU0Njg2NTY3NThjMmFhZjY4ZmE3ZjcyNWNlOWY0Y2NhZTU2OThiNzkmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.2MnYAzU3z5V6Qrm8sx7xzC57PtbOHXYw_h7zrqTtMMY)
![image](https://private-user-images.githubusercontent.com/80370489/342219540-c38eae32-def3-40a2-b951-642f7aa32b06.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM3OTgxMTAsIm5iZiI6MTcyMzc5NzgxMCwicGF0aCI6Ii84MDM3MDQ4OS8zNDIyMTk1NDAtYzM4ZWFlMzItZGVmMy00MGEyLWI5NTEtNjQyZjdhYTMyYjA2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE2VDA4NDMzMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWQxMDM1MWI3OGY2MDhhZjUzZjczZjAwM2E5YWY2YTRiMzhlNWU2MjM1YmJiYjMyZWViN2I4YWM5ZjVjYWQ2MjImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.ZCHSb4n9H36tWyBnwVV9DsGBp3gmakker5xlstG1iBg)
However, segment errors continued to occur.
![image](https://private-user-images.githubusercontent.com/80370489/342220066-febd8d87-52a3-4aff-9c76-876e4c22bbec.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM3OTgxMTAsIm5iZiI6MTcyMzc5NzgxMCwicGF0aCI6Ii84MDM3MDQ4OS8zNDIyMjAwNjYtZmViZDhkODctNTJhMy00YWZmLTljNzYtODc2ZTRjMjJiYmVjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE2VDA4NDMzMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTBkNTM4MTYwNjczY2ExZDRkNTdkZWZmYzFiNjQ5NWQwYjFlNGJhZDIwZGQ4MjZlYWViZjJhNjNmNmRlZWNhOTkmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.zXh8MNq4Vo2raYHOoVFjUo4_P24r-LPf8joa8zoMhOE)
![image](https://private-user-images.githubusercontent.com/80370489/342220234-3cc5275a-1782-4fe6-a7a7-9be5ad1a5d33.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM3OTgxMTAsIm5iZiI6MTcyMzc5NzgxMCwicGF0aCI6Ii84MDM3MDQ4OS8zNDIyMjAyMzQtM2NjNTI3NWEtMTc4Mi00ZmU2LWE3YTctOWJlNWFkMWE1ZDMzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE2VDA4NDMzMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWY2YWMzOTc5OTVhMDQ3YThiZTg3ZGE1MmU1NmI3YmU2YTE2YzQxNGNkMTM4Nzc5N2MyN2ZlNzIxMmJlN2VmYTMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.GxtSVx4TgFRbciKxlxKoYZLcWUlHtW4e6C_XVaLyPkA)
from pinpoint.
I've never seen a case like this before. I'll check pinot issue to see if there are any similar issues.
If you have any logs, please share them with me.
from pinpoint.
exceptionTrace table uses CLP encoding. I am not sure CLP library support Graviton(ARM64).
reference: Pinot doesn't provide official binaries for M1 or M2 Mac systems.
https://docs.pinot.apache.org/basics/getting-started/running-pinot-locally#m1-and-m2-mac-support
from pinpoint.
Thanks for your help.
As you said, I changed it to x86 and applied it, and it worked fine.
Perhaps CLP does not support arm64 architecture.
from pinpoint.
It seems that Apache Pinot is also planning to support various platform, including aarch64.
from pinpoint.
Related Issues (20)
- MariaDB queries not visible (MariaDB JDBC version above 2.7.9)
- [FR.] application logger pattern full-replacement support
- Add Grpc Stream KeepAlive
- Fix concurrency bug in GrpcAgentConnectionRepository
- [SolutionShare] run/debug pinpoint-web error using intellij idea community edition HOT 1
- Apply gRPC MAX_CONNECTION_AGE
- Abbreviate long SQL queries in SQL metadata
- Bump hbase-client from 2.5.9 to 2.5.10
- Add validation of parentApplicationName
- improve /getApplicationHostinfo api
- Create Pinot Tables HOT 18
- Improved agent list performance
- Optimize agent lookup for agent compatibility
- pinpoint3.0.0 uses kafka to pass monitored metrics, so can stop the hbase database? HOT 1
- Add write option to AsyncBufferedMutatorBuilder
- [FR.] Add pinot systemd unit files.
- using pinpoint 3.0.0, the contextpath of most applications is lost.
- using pinpoint 3.0.0, the contextpath of most applications is lost. HOT 1
- The inspector data of the V3 version web interface is displayed incorrectly, and the values ββare all 0, but the inspector of the V2 version displays correctly.
- Create Pinot exceptionTraceTable table failed. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pinpoint.