Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimise PrimaryShardAllocator allocateUnassignedBatch method #14645

Closed
Bukhtawar opened this issue Jul 4, 2024 · 1 comment
Closed

Optimise PrimaryShardAllocator allocateUnassignedBatch method #14645

Bukhtawar opened this issue Jul 4, 2024 · 1 comment
Labels
Cluster Manager enhancement Enhancement or improvement to existing feature or request good first issue Good for newcomers ShardManagement:Performance

Comments

@Bukhtawar
Copy link
Collaborator

Bukhtawar commented Jul 4, 2024 •
edited
Loading

Is your feature request related to a problem? Please describe

 100.0% (5s out of 5s) cpu usage by thread 'opensearch[74e0b23bcf51c21918e96f38f93e1491][clusterManagerService#updateTask][T#1]'
     9/10 snapshots sharing following 25 elements
       [email protected]/java.util.ArrayList.indexOfRange(ArrayList.java:299)
       [email protected]/java.util.ArrayList.indexOf(ArrayList.java:286)
       [email protected]/java.util.ArrayList.contains(ArrayList.java:275)
       app//org.opensearch.gateway.PrimaryShardBatchAllocator.allocateUnassignedBatch(PrimaryShardBatchAllocator.java:107)
       app//org.opensearch.gateway.ShardsBatchGatewayAllocator.innerAllocateUnassignedBatch(ShardsBatchGatewayAllocator.java:255)
       app//org.opensearch.gateway.ShardsBatchGatewayAllocator.allocateAllUnassignedShards(ShardsBatchGatewayAllocator.java:227)
       app//org.opensearch.cluster.routing.allocation.AllocationService.allocateAllUnassignedShards(AllocationService.java:606)
       app//org.opensearch.cluster.routing.allocation.AllocationService.allocateExistingUnassignedShards(AllocationService.java:578)
       app//org.opensearch.cluster.routing.allocation.AllocationService.reroute(AllocationService.java:557)
       app//org.opensearch.cluster.routing.allocation.AllocationService.reroute(AllocationService.java:524)
       app//org.opensearch.node.Node$$Lambda$2602/0x0000004000a253b8.apply(Unknown Source)
       app//org.opensearch.cluster.routing.BatchedRerouteService$1.execute(BatchedRerouteService.java:136)
       app//org.opensearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:67)
       app//org.opensearch.cluster.service.MasterService.executeTasks(MasterService.java:882)
       app//org.opensearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:434)
       app//org.opensearch.cluster.service.MasterService.runTasks(MasterService.java:301)
       app//org.opensearch.cluster.service.MasterService$Batcher.run(MasterService.java:212)
       app//org.opensearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:209)
       app//org.opensearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:247)
       app//org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:863)
       app//org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedOpenSearchThreadPoolExecutor.java:283)
       app//org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedOpenSearchThreadPoolExecutor.java:246)
       [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
       [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
       [email protected]/java.lang.Thread.run(Thread.java:840)
     unique snapshot
       [email protected]/java.util.HashMap.forEach(HashMap.java:1421)
       app//org.opensearch.gateway.PrimaryShardBatchAllocator.adaptToNodeShardStates(PrimaryShardBatchAllocator.java:145)
       app//org.opensearch.gateway.PrimaryShardBatchAllocator.allocateUnassignedBatch(PrimaryShardBatchAllocator.java:112)
       app//org.opensearch.gateway.ShardsBatchGatewayAllocator.innerAllocateUnassignedBatch(ShardsBatchGatewayAllocator.java:255)
       app//org.opensearch.gateway.ShardsBatchGatewayAllocator.allocateAllUnassignedShards(ShardsBatchGatewayAllocator.java:227)
       app//org.opensearch.cluster.routing.allocation.AllocationService.allocateAllUnassignedShards(AllocationService.java:606)
       app//org.opensearch.cluster.routing.allocation.AllocationService.allocateExistingUnassignedShards(AllocationService.java:578)
       app//org.opensearch.cluster.routing.allocation.AllocationService.reroute(AllocationService.java:557)
       app//org.opensearch.cluster.routing.allocation.AllocationService.reroute(AllocationService.java:524)
       app//org.opensearch.node.Node$$Lambda$2602/0x0000004000a253b8.apply(Unknown Source)
       app//org.opensearch.cluster.routing.BatchedRerouteService$1.execute(BatchedRerouteService.java:136)
       app//org.opensearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:67)
       app//org.opensearch.cluster.service.MasterService.executeTasks(MasterService.java:882)
       app//org.opensearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:434)
       app//org.opensearch.cluster.service.MasterService.runTasks(MasterService.java:301)
       app//org.opensearch.cluster.service.MasterService$Batcher.run(MasterService.java:212)
       app//org.opensearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:209)
       app//org.opensearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:247)
       app//org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:863)
       app//org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedOpenSearchThreadPoolExecutor.java:283)
       app//org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedOpenSearchThreadPoolExecutor.java:246)
       [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
       [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
       [email protected]/java.lang.Thread.run(Thread.java:840)

Describe the solution you'd like

Use Set instead of a ArrayList for faster lookups

Related component

Cluster Manager

Describe alternatives you've considered

No response

Additional context

No response
@Bukhtawar Bukhtawar added enhancement Enhancement or improvement to existing feature or request untriaged labels Jul 4, 2024
@Bukhtawar Bukhtawar mentioned this issue Jul 4, 2024
Closed
@Bukhtawar Bukhtawar changed the title [Feature Request] Optimise PrimaryShardAllocator allocateUnassignedBatch method Optimise PrimaryShardAllocator allocateUnassignedBatch method Jul 4, 2024
@mch2 mch2 added the good first issue Good for newcomers label Jul 31, 2024
@mch2
Copy link
Member

mch2 commented Jul 31, 2024

Looks like this has been resolved with #14533

@mch2 mch2 closed this as completed Jul 31, 2024
@github-project-automation github-project-automation bot moved this from 🆕 New to ✅ Done in Cluster Manager Project Board Jul 31, 2024
@github-project-automation github-project-automation bot moved this from 🆕 New to ✅ Done in Shard Management Project Board Jul 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Cluster Manager enhancement Enhancement or improvement to existing feature or request good first issue Good for newcomers ShardManagement:Performance
Projects
Cluster Manager Project Board
Status: ✅ Done
Shard Management Project Board
Status: ✅ Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mch2 @Bukhtawar