Skip to content

Conversation

K0K0V0K
Copy link
Contributor

@K0K0V0K K0K0V0K commented Feb 21, 2024

Description of PR

Created a MultiDispatcher class which implements the Dispatcher interface.
The dispatcher replace the AsyncDispatcher in the RMStateStore.
The Dispatcher creates a separate metric object called Event metrics for "rm-state-store" where we can see

  • how many unhandled events are currently present in the event queue for the specific event type
  • how many events were handled for the specific event type
  • average execution time for the specific event

How was this patch tested?

  • Unit test
  • Deployed cluster and load test with PI example job with the following implementations
    • FileSystemRMStateStore
    • LeveldbRMStateStore
    • ZKRMStateStore
  • Check RM stop - start can't corrupt RM

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 54s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 6 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 58s Maven dependency ordering for branch
+1 💚 mvninstall 31m 49s trunk passed
+1 💚 compile 7m 32s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 compile 6m 59s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 checkstyle 1m 58s trunk passed
+1 💚 mvnsite 2m 11s trunk passed
+1 💚 javadoc 2m 11s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 2m 1s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 3m 55s trunk passed
+1 💚 shadedclient 34m 23s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 33s Maven dependency ordering for patch
+1 💚 mvninstall 1m 23s the patch passed
+1 💚 compile 6m 51s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javac 6m 51s the patch passed
+1 💚 compile 6m 57s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 javac 6m 57s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 1m 47s /results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn-project/hadoop-yarn: The patch generated 12 new + 71 unchanged - 1 fixed = 83 total (was 72)
+1 💚 mvnsite 1m 57s the patch passed
-1 ❌ javadoc 0m 58s /patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt hadoop-yarn-common in the patch failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.
-1 ❌ javadoc 0m 57s /patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt hadoop-yarn-common in the patch failed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08.
+1 💚 spotbugs 4m 7s the patch passed
+1 💚 shadedclient 34m 33s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 5m 35s hadoop-yarn-common in the patch passed.
-1 ❌ unit 106m 58s /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch passed.
-1 ❌ asflicense 0m 58s /results-asflicense.txt The patch generated 2 ASF License warnings.
288m 47s
Reason Tests
Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerHealth
hadoop.yarn.server.resourcemanager.TestResourceManager
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodes
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerApps
hadoop.yarn.server.resourcemanager.logaggregationstatus.TestRMAppLogAggregationStatus
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/1/artifact/out/Dockerfile
GITHUB PR #6569
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 2afb8c247a02 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / b75a80b
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/1/testReport/
Max. process+thread count 938 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

- fix failing test
- some test using RMStateStore without start it
* @return the semaphore
*/
default String getLockKey() {
return null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is returning null expected here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @slfan1989 !

Thanks for the review.
Yes, that is expected.
If we dont specify lockKey for an event we should return with null, so these events will be executed in sequential not parallel. The method is used in the MultiDispatcherLocks.

@K0K0V0K K0K0V0K marked this pull request as draft February 23, 2024 14:37
@K0K0V0K K0K0V0K marked this pull request as ready for review February 23, 2024 14:40
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 23s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 6 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 28m 51s Maven dependency ordering for branch
+1 💚 mvninstall 20m 55s trunk passed
+1 💚 compile 3m 46s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 compile 3m 31s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 checkstyle 1m 4s trunk passed
+1 💚 mvnsite 1m 24s trunk passed
+1 💚 javadoc 1m 18s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 15s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 2m 21s trunk passed
+1 💚 shadedclient 20m 28s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 26s Maven dependency ordering for patch
+1 💚 mvninstall 0m 49s the patch passed
+1 💚 compile 3m 26s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javac 3m 26s the patch passed
+1 💚 compile 3m 23s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 javac 3m 23s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 57s /results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn-project/hadoop-yarn: The patch generated 12 new + 71 unchanged - 1 fixed = 83 total (was 72)
+1 💚 mvnsite 1m 11s the patch passed
+1 💚 javadoc 1m 8s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 16s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 2m 33s the patch passed
+1 💚 shadedclient 20m 44s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 4m 38s hadoop-yarn-common in the patch passed.
-1 ❌ unit 67m 54s /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 40s The patch does not generate ASF License warnings.
197m 20s
Reason Tests
Failed junit tests hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceCapacity
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueParsing
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesConfigurationMutation
hadoop.yarn.server.resourcemanager.scheduler.policy.TestFairOrderingPolicy
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestSchedulingRequestContainerAllocation
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerSchedulingRequestUpdate
hadoop.yarn.server.resourcemanager.TestAppManager
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedulerConfigMutation
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSLeafQueue
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWithMultiResourceTypes
hadoop.yarn.server.resourcemanager.webapp.fairscheduler.TestRMWebServicesFairScheduler
hadoop.yarn.server.resourcemanager.webapp.fairscheduler.TestRMWebServicesFairSchedulerCustomResourceTypes
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing
hadoop.yarn.server.resourcemanager.scheduler.constraint.TestPlacementProcessor
hadoop.yarn.server.resourcemanager.TestAppManagerWithFairScheduler
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCNodeUpdates
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerSurgicalPreemption
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueues
hadoop.yarn.webapp.TestRMWithCSRFFilter
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNewQueueAutoCreationWithCapacityVectors
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppAttempt
hadoop.yarn.server.resourcemanager.nodelabels.TestRMNodeLabelsManager
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimits
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppCustomResourceTypes
hadoop.yarn.server.resourcemanager.TestRMEmbeddedElector
hadoop.yarn.server.resourcemanager.scheduler.fair.TestApplicationMasterServiceWithFS
hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceInterceptor
hadoop.yarn.server.resourcemanager.monitor.TestSchedulingMonitor
hadoop.yarn.webapp.TestRMWithXFSFilter
hadoop.yarn.server.resourcemanager.TestOpportunisticContainerAllocatorAMService
hadoop.yarn.server.resourcemanager.TestMoveApplication
hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesHttpStaticUserPermissions
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAsyncScheduling
hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceFair
hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestAbsoluteResourceConfiguration
hadoop.yarn.server.resourcemanager.TestRMHA
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestAbsoluteResourceWithAutoQueue
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerLazyPreemption
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestAMAllocatedToNonExclusivePartition
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMultiNodes
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerConfigValidator
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerFairShare
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsCustomResourceTypes
hadoop.yarn.server.resourcemanager.TestCapacitySchedulerMetrics
hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestLeveldbConfigurationStore
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWeightMode
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAmbiguousLeafs
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAutoCreatedQueuePreemption
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimitsByPartition
hadoop.yarn.server.resourcemanager.scheduler.fair.TestAppRunnability
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCResponseId
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer
hadoop.yarn.server.resourcemanager.scheduler.TestQueueMetrics
hadoop.yarn.server.resourcemanager.volume.csi.TestVolumeProcessor
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication
hadoop.yarn.server.resourcemanager.TestDecommissioningNodesWatcher
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestSchedulingRequestContainerAllocationAsync
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivities
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueMappingFactory
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppAttempts
hadoop.yarn.server.resourcemanager.TestResourceTrackerService
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAutoQueueCreation
hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
hadoop.yarn.server.resourcemanager.TestRMProxyUsersConf
hadoop.yarn.server.resourcemanager.TestApplicationCleanup
hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesContainers
hadoop.yarn.server.resourcemanager.TestLeaderElectorService
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
hadoop.yarn.server.resourcemanager.TestRMHAMetrics
hadoop.yarn.server.resourcemanager.scheduler.constraint.TestPlacementConstraintsUtil
hadoop.yarn.server.resourcemanager.scheduler.constraint.TestAllocationTagsManager
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMaxParallelApps
hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
hadoop.yarn.server.resourcemanager.resource.TestResourceProfiles
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMultiNodesWithPreemption
hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
hadoop.yarn.server.resourcemanager.scheduler.fair.TestQueueManagerRealScheduler
hadoop.yarn.server.resourcemanager.TestAMAuthorization
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedulerMixedMode
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCSAllocateCustomResource
hadoop.yarn.server.resourcemanager.rmapp.TestApplicationLifetimeMonitor
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestMixedQueueResourceCalculation
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerOvercommit
hadoop.yarn.server.resourcemanager.placement.TestPlacementManager
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestAutoCreatedQueueDeletionPolicy
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWorkflowPriorityMapping
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivitiesWithMultiNodesEnabled
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueManagementDynamicEditPolicy
hadoop.yarn.server.resourcemanager.TestSignalContainer
hadoop.yarn.server.resourcemanager.scheduler.fair.TestSchedulingUpdate
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNewQueueAutoCreation
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriorityACLs
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedLegacyQueueCreation
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServiceAppsNodelabel
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedDynamicConfig
hadoop.yarn.server.resourcemanager.nodelabels.TestRMDelegatedNodeLabelsUpdater
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestUniformQueueResourceCalculation
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions
hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler
hadoop.yarn.server.resourcemanager.federation.TestFederationRMStateStoreService
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation
hadoop.yarn.server.resourcemanager.TestClientRMService
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel
hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore
hadoop.yarn.server.resourcemanager.scheduler.constraint.algorithm.TestLocalAllocationTagsManager
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerOvercommit
hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicy
hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication
hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestZKConfigurationStore
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/4/artifact/out/Dockerfile
GITHUB PR #6569
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 6d69fd9a6f6c 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / cf3fe1c
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/4/testReport/
Max. process+thread count 5363 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/4/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@K0K0V0K K0K0V0K marked this pull request as draft February 24, 2024 11:31
- fix test
- remove locks
- add hash based dispatching
- fix thread name
- sort log records in monitor
- fix out of array exception
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 52s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 6 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 14s Maven dependency ordering for branch
+1 💚 mvninstall 36m 58s trunk passed
+1 💚 compile 9m 3s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 compile 9m 13s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 checkstyle 2m 5s trunk passed
+1 💚 mvnsite 2m 28s trunk passed
+1 💚 javadoc 2m 4s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 59s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 4m 10s trunk passed
+1 💚 shadedclient 40m 20s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 31s Maven dependency ordering for patch
+1 💚 mvninstall 1m 22s the patch passed
+1 💚 compile 8m 10s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javac 8m 10s the patch passed
+1 💚 compile 9m 47s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 javac 9m 47s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 1m 56s /results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn-project/hadoop-yarn: The patch generated 14 new + 71 unchanged - 1 fixed = 85 total (was 72)
+1 💚 mvnsite 2m 26s the patch passed
+1 💚 javadoc 1m 50s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 49s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 ❌ spotbugs 2m 10s /new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚 shadedclient 40m 2s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 5m 43s hadoop-yarn-common in the patch passed.
-1 ❌ unit 127m 31s /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 54s The patch does not generate ASF License warnings.
334m 6s
Reason Tests
SpotBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
new org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor(Logger, MultiDispatcherConfig, String) invokes org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor$MultiDispatcherExecutorThread.start() At MultiDispatcherExecutor.java:org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor$MultiDispatcherExecutorThread.start() At MultiDispatcherExecutor.java:[line 54]
Failed junit tests hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore
hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore
hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerHealth
hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
hadoop.yarn.server.resourcemanager.TestResourceManager
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/5/artifact/out/Dockerfile
GITHUB PR #6569
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux c5f4bede693b 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / f0788b5
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/5/testReport/
Max. process+thread count 1884 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/5/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 48s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 6 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 8s Maven dependency ordering for branch
+1 💚 mvninstall 37m 48s trunk passed
+1 💚 compile 10m 14s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 compile 9m 13s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 checkstyle 2m 5s trunk passed
+1 💚 mvnsite 2m 5s trunk passed
+1 💚 javadoc 2m 6s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 58s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 4m 1s trunk passed
+1 💚 shadedclient 41m 10s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 31s Maven dependency ordering for patch
+1 💚 mvninstall 1m 29s the patch passed
+1 💚 compile 9m 8s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javac 9m 8s the patch passed
+1 💚 compile 8m 26s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 javac 8m 26s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 1m 58s /results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn-project/hadoop-yarn: The patch generated 14 new + 71 unchanged - 1 fixed = 85 total (was 72)
+1 💚 mvnsite 1m 55s the patch passed
+1 💚 javadoc 1m 57s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 55s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 ❌ spotbugs 2m 9s /new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚 shadedclient 40m 48s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 5m 44s hadoop-yarn-common in the patch passed.
-1 ❌ unit 130m 39s /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 58s The patch does not generate ASF License warnings.
339m 47s
Reason Tests
SpotBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
new org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor(Logger, MultiDispatcherConfig, String) invokes org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor$MultiDispatcherExecutorThread.start() At MultiDispatcherExecutor.java:org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor$MultiDispatcherExecutorThread.start() At MultiDispatcherExecutor.java:[line 54]
Failed junit tests hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore
hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore
hadoop.yarn.server.resourcemanager.TestRMRestart
hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/6/artifact/out/Dockerfile
GITHUB PR #6569
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 2a7e5ebdfb49 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / f0788b5
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/6/testReport/
Max. process+thread count 1884 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/6/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 45s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 6 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 34s Maven dependency ordering for branch
+1 💚 mvninstall 35m 48s trunk passed
+1 💚 compile 9m 17s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 compile 7m 55s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 checkstyle 1m 58s trunk passed
+1 💚 mvnsite 2m 0s trunk passed
+1 💚 javadoc 2m 2s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 54s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 3m 55s trunk passed
+1 💚 shadedclient 41m 9s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 29s Maven dependency ordering for patch
+1 💚 mvninstall 1m 25s the patch passed
+1 💚 compile 8m 1s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javac 8m 1s the patch passed
+1 💚 compile 7m 45s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 javac 7m 45s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 1m 51s /results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn-project/hadoop-yarn: The patch generated 14 new + 71 unchanged - 1 fixed = 85 total (was 72)
+1 💚 mvnsite 1m 49s the patch passed
+1 💚 javadoc 1m 49s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 44s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 ❌ spotbugs 2m 1s /new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚 shadedclient 41m 12s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 5m 45s hadoop-yarn-common in the patch passed.
-1 ❌ unit 136m 21s /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 54s The patch does not generate ASF License warnings.
338m 42s
Reason Tests
SpotBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
new org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor(Logger, MultiDispatcherConfig, String) invokes org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor$MultiDispatcherExecutorThread.start() At MultiDispatcherExecutor.java:org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor$MultiDispatcherExecutorThread.start() At MultiDispatcherExecutor.java:[line 54]
Failed junit tests hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore
hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/7/artifact/out/Dockerfile
GITHUB PR #6569
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 97c32d4419ce 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 3b56ade
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/7/testReport/
Max. process+thread count 1830 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/7/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 46s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 6 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 15m 34s Maven dependency ordering for branch
+1 💚 mvninstall 38m 55s trunk passed
+1 💚 compile 8m 22s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 compile 7m 59s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 checkstyle 2m 0s trunk passed
+1 💚 mvnsite 2m 5s trunk passed
+1 💚 javadoc 2m 3s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 53s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 4m 30s trunk passed
+1 💚 shadedclient 41m 32s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 30s Maven dependency ordering for patch
+1 💚 mvninstall 1m 20s the patch passed
+1 💚 compile 7m 58s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javac 7m 58s the patch passed
+1 💚 compile 7m 45s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 javac 7m 45s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 1m 51s /results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn-project/hadoop-yarn: The patch generated 14 new + 71 unchanged - 1 fixed = 85 total (was 72)
+1 💚 mvnsite 1m 50s the patch passed
+1 💚 javadoc 1m 52s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 47s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 ❌ spotbugs 2m 1s /new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 💚 shadedclient 40m 33s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 5m 40s hadoop-yarn-common in the patch passed.
-1 ❌ unit 118m 47s /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 1m 27s The patch does not generate ASF License warnings.
325m 15s
Reason Tests
SpotBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
Bad attempt to compute absolute value of signed 32-bit hashcode in org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor.execute(Event, Runnable) At MultiDispatcherExecutor.java:value of signed 32-bit hashcode in org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor.execute(Event, Runnable) At MultiDispatcherExecutor.java:[line 60]
new org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor(Logger, MultiDispatcherConfig, String) invokes org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor$MultiDispatcherExecutorThread.start() At MultiDispatcherExecutor.java:org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor$MultiDispatcherExecutorThread.start() At MultiDispatcherExecutor.java:[line 54]
Failed junit tests hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore
hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/8/artifact/out/Dockerfile
GITHUB PR #6569
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux f383ab59b36f 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / e4b7b53
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/8/testReport/
Max. process+thread count 1744 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/8/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 17m 58s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 8 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 47s Maven dependency ordering for branch
+1 💚 mvninstall 36m 36s trunk passed
+1 💚 compile 8m 6s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 compile 7m 21s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 checkstyle 1m 58s trunk passed
+1 💚 mvnsite 2m 1s trunk passed
+1 💚 javadoc 2m 6s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 javadoc 1m 57s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 3m 55s trunk passed
+1 💚 shadedclient 43m 52s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 31s Maven dependency ordering for patch
+1 💚 mvninstall 1m 33s the patch passed
+1 💚 compile 9m 18s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 javac 9m 18s the patch passed
+1 💚 compile 9m 26s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 javac 9m 26s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 1m 55s /results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn-project/hadoop-yarn: The patch generated 14 new + 95 unchanged - 3 fixed = 109 total (was 98)
+1 💚 mvnsite 1m 52s the patch passed
+1 💚 javadoc 1m 52s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 javadoc 1m 48s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 ❌ spotbugs 1m 56s /new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 💚 shadedclient 40m 7s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 5m 37s hadoop-yarn-common in the patch passed.
-1 ❌ unit 104m 43s /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 55s The patch does not generate ASF License warnings.
328m 36s
Reason Tests
SpotBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
Bad attempt to compute absolute value of signed 32-bit hashcode in org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor.execute(Event, Runnable) At MultiDispatcherExecutor.java:value of signed 32-bit hashcode in org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor.execute(Event, Runnable) At MultiDispatcherExecutor.java:[line 60]
new org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor(Logger, MultiDispatcherConfig, String) invokes org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor$MultiDispatcherExecutorThread.start() At MultiDispatcherExecutor.java:org.apache.hadoop.yarn.event.multidispatcher.MultiDispatcherExecutor$MultiDispatcherExecutorThread.start() At MultiDispatcherExecutor.java:[line 54]
Failed junit tests hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/9/artifact/out/Dockerfile
GITHUB PR #6569
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 7993079112fd 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 0654211
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/9/testReport/
Max. process+thread count 1834 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/9/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 50s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 8 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 8s Maven dependency ordering for branch
+1 💚 mvninstall 36m 50s trunk passed
+1 💚 compile 8m 56s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 compile 7m 41s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 checkstyle 2m 8s trunk passed
+1 💚 mvnsite 2m 0s trunk passed
+1 💚 javadoc 2m 3s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 javadoc 1m 53s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 4m 19s trunk passed
+1 💚 shadedclient 40m 11s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 31s Maven dependency ordering for patch
+1 💚 mvninstall 1m 23s the patch passed
+1 💚 compile 8m 3s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 javac 8m 3s the patch passed
+1 💚 compile 8m 26s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 javac 8m 26s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 2m 3s /results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn-project/hadoop-yarn: The patch generated 20 new + 95 unchanged - 3 fixed = 115 total (was 98)
+1 💚 mvnsite 2m 5s the patch passed
+1 💚 javadoc 1m 53s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 javadoc 1m 59s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 4m 33s the patch passed
+1 💚 shadedclient 41m 52s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 5m 40s hadoop-yarn-common in the patch passed.
+1 💚 unit 108m 43s hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 57s The patch does not generate ASF License warnings.
313m 2s
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/10/artifact/out/Dockerfile
GITHUB PR #6569
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 2c6393f94f3c 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 465a0de
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/10/testReport/
Max. process+thread count 1597 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/10/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 54s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 8 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 4s Maven dependency ordering for branch
+1 💚 mvninstall 37m 43s trunk passed
+1 💚 compile 9m 47s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 compile 7m 57s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 checkstyle 2m 2s trunk passed
+1 💚 mvnsite 2m 1s trunk passed
+1 💚 javadoc 2m 10s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 javadoc 2m 0s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 4m 12s trunk passed
+1 💚 shadedclient 40m 24s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 31s Maven dependency ordering for patch
+1 💚 mvninstall 1m 22s the patch passed
+1 💚 compile 8m 57s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 javac 8m 57s the patch passed
+1 💚 compile 8m 18s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 javac 8m 18s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 1m 56s /results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn-project/hadoop-yarn: The patch generated 20 new + 95 unchanged - 3 fixed = 115 total (was 98)
+1 💚 mvnsite 1m 58s the patch passed
+1 💚 javadoc 1m 59s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 javadoc 1m 52s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 4m 23s the patch passed
+1 💚 shadedclient 41m 9s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 5m 42s hadoop-yarn-common in the patch passed.
+1 💚 unit 108m 8s hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 55s The patch does not generate ASF License warnings.
314m 45s
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/11/artifact/out/Dockerfile
GITHUB PR #6569
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 815f880bc435 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 465a0de
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/11/testReport/
Max. process+thread count 1650 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6569/11/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@K0K0V0K K0K0V0K marked this pull request as ready for review March 13, 2024 13:43
Copy link
Contributor

@p-szucs p-szucs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @K0K0V0K, I think it is a very nice patch! I only added some minor comments

String lockKey = event.getLockKey();
// abs of Integer.MIN_VALUE is Integer.MIN_VALUE
int threadIndex = lockKey == null || lockKey.hashCode() == Integer.MIN_VALUE ?
0 : Math.abs(lockKey.hashCode() % threads.length);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on our discussion, I think a comment or description probably would be useful to make the goal of this computation more clear

}

/**
* Maximus size of the event queue of the executor threads.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a typo, if you touch the code again anyways :)

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
_ Prechecks _
+1 💚 dupname 0m 05s No case conflicting files found.
+0 🆗 spotbugs 0m 00s spotbugs executables are not available.
+0 🆗 codespell 0m 00s codespell was not available.
+0 🆗 detsecrets 0m 00s detect-secrets was not available.
+1 💚 @author 0m 01s The patch does not contain any @author tags.
+1 💚 test4tests 0m 00s The patch appears to include 8 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 2m 26s Maven dependency ordering for branch
+1 💚 mvninstall 89m 23s trunk passed
+1 💚 compile 11m 21s trunk passed
+1 💚 checkstyle 5m 05s trunk passed
+1 💚 mvnsite 11m 06s trunk passed
+1 💚 javadoc 10m 44s trunk passed
+1 💚 shadedclient 159m 11s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 2m 28s Maven dependency ordering for patch
-1 ❌ mvninstall 3m 06s /patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch failed.
-1 ❌ compile 6m 16s /patch-compile-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn in the patch failed.
-1 ❌ javac 6m 16s /patch-compile-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn in the patch failed.
+1 💚 blanks 0m 01s The patch has no blanks issues.
+1 💚 checkstyle 3m 03s the patch passed
-1 ❌ mvnsite 3m 26s /patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch failed.
+1 💚 javadoc 7m 30s the patch passed
-1 ❌ shadedclient 85m 23s patch has errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 4m 25s The patch does not generate ASF License warnings.
375m 07s
Subsystem Report/Notes
GITHUB PR #6569
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname MINGW64_NT-10.0-17763 9bb75f535c07 3.4.10-87d57229.x86_64 2024-02-14 20:17 UTC x86_64 Msys
Build tool maven
Personality /c/hadoop/dev-support/bin/hadoop.sh
git revision trunk / 465a0de
Default Java Azul Systems, Inc.-1.8.0_332-b09
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6569/1/testReport/
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6569/1/console
versions git=2.44.0.windows.1
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

github-actions bot commented Oct 1, 2025

We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
Thanks all for your contribution.

@github-actions github-actions bot added the Stale label Oct 1, 2025
@github-actions github-actions bot closed this Oct 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants