Skip to content

YARN-11973. RM engine: comma-separated username matching in placement rules - #8656

Open
myloginid wants to merge 2 commits into
apache:trunkfrom
myloginid:YARN-11972-comma-separated-user-placement-rules
Open

YARN-11973. RM engine: comma-separated username matching in placement rules#8656
myloginid wants to merge 2 commits into
apache:trunkfrom
myloginid:YARN-11972-comma-separated-user-placement-rules

Conversation

@myloginid

@myloginid myloginid commented Aug 4, 2026 •
edited
Loading

Copy link
Copy Markdown

Summary

RM engine support for comma-separated usernames in Capacity Scheduler user placement rules.

Changes

  • Add MappingRuleMatchers.createUserMatcherFromMatches() using existing OrMatcher
  • Wire into MappingRuleCreator (JSON) and MappingRule.createLegacyRule() (legacy u: mappings)
  • Update JSON schema description for matches
  • Unit/integration tests in TestMappingRuleCreator, TestMappingRuleMatchers, TestMappingRule

Related sub-tasks (YARN-11972)

Test plan

  • mvn test -pl hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager -Dtest=TestMappingRuleCreator,TestMappingRuleMatchers,TestMappingRule -am

Sub-task of YARN-11972.
Fixes: https://issues.apache.org/jira/browse/YARN-11973

@myloginid @cursoragent
YARN-11973. RM engine: comma-separated username matching in placement…
0a60378 
… rules.

Wire OrMatcher into user rule creation via createUserMatcherFromMatches().
Applies to JSON rules and legacy u: queue mappings. Includes JSON schema
update and unit/integration tests.

Sub-task of YARN-11972.

Co-authored-by: Cursor <cursoragent@cursor.com>
@myloginid
myloginid force-pushed the YARN-11972-comma-separated-user-placement-rules branch from 5f38b43 to 0a60378 Compare August 4, 2026 13:15
@myloginid myloginid changed the title YARN-11972. Support comma-separated usernames in Capacity Scheduler user placement rules YARN-11973. RM engine: comma-separated username matching in placement rules Aug 4, 2026
This was referenced Aug 4, 2026
Open
Open
@myloginid

myloginid commented Aug 4, 2026

Copy link
Copy Markdown
Author

Please add labels: trunk, YARN

@hadoop-yetus

hadoop-yetus commented Aug 4, 2026

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 18m 50s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+0 🆗 jsonlint 0m 1s jsonlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 3 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 51m 2s trunk passed
+1 💚 compile 1m 29s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 compile 1m 29s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 checkstyle 1m 23s trunk passed
+1 💚 mvnsite 1m 38s trunk passed
+1 💚 javadoc 1m 14s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 1m 14s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 spotbugs 3m 0s trunk passed
+1 💚 shadedclient 36m 39s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 4s the patch passed
+1 💚 compile 1m 1s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javac 1m 1s the patch passed
+1 💚 compile 1m 5s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 javac 1m 5s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 59s the patch passed
+1 💚 mvnsite 1m 9s the patch passed
+1 💚 javadoc 0m 49s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 0m 50s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 spotbugs 2m 45s the patch passed
+1 💚 shadedclient 36m 36s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 121m 20s /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 37s The patch does not generate ASF License warnings.
285m 42s
Reason Tests
Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.TestMappingRuleCreator
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8656/2/artifact/out/Dockerfile
GITHUB PR #8656
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets jsonlint
uname Linux bc462c871eca 5.15.0-185-generic #195-Ubuntu SMP Fri Jun 19 17:11:50 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 0a60378
Default Java Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.11+10-1-24.04.2-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8656/2/testReport/
Max. process+thread count 930 (vs. ulimit of 10000)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8656/2/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

hadoop-yetus commented Aug 4, 2026

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 20m 11s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+0 🆗 jsonlint 0m 1s jsonlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 3 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 51m 36s trunk passed
+1 💚 compile 1m 24s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 compile 1m 25s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 checkstyle 1m 21s trunk passed
+1 💚 mvnsite 1m 34s trunk passed
+1 💚 javadoc 1m 14s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 1m 12s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 spotbugs 2m 56s trunk passed
+1 💚 shadedclient 37m 37s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 7s the patch passed
+1 💚 compile 1m 1s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javac 1m 1s the patch passed
+1 💚 compile 1m 1s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 javac 1m 1s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 57s the patch passed
+1 💚 mvnsite 1m 10s the patch passed
+1 💚 javadoc 0m 49s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 0m 52s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 spotbugs 2m 46s the patch passed
+1 💚 shadedclient 35m 48s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 121m 16s /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 36s The patch does not generate ASF License warnings.
287m 34s
Reason Tests
Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.TestMappingRuleCreator
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8656/1/artifact/out/Dockerfile
GITHUB PR #8656
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets jsonlint
uname Linux 73cb72eeb703 5.15.0-185-generic #195-Ubuntu SMP Fri Jun 19 17:11:50 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 0a60378
Default Java Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.11+10-1-24.04.2-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8656/1/testReport/
Max. process+thread count 902 (vs. ulimit of 10000)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8656/1/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@myloginid @cursoragent
YARN-11973. Reject empty usernames in comma-separated match strings.
8bcf90f 
Use splitPreserveAllTokens so values like alice,,bob are rejected instead
of silently collapsing to alice,bob.

Co-authored-by: Cursor <cursoragent@cursor.com>
@hadoop-yetus

hadoop-yetus commented Aug 5, 2026

Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 54s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 jsonlint 0m 0s jsonlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 3 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 48m 41s trunk passed
+1 💚 compile 1m 20s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 compile 1m 21s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 checkstyle 1m 18s trunk passed
+1 💚 mvnsite 1m 28s trunk passed
+1 💚 javadoc 1m 12s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 1m 10s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 spotbugs 2m 41s trunk passed
+1 💚 shadedclient 35m 56s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 0s the patch passed
+1 💚 compile 0m 54s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javac 0m 54s the patch passed
+1 💚 compile 0m 59s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 javac 0m 59s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 51s the patch passed
+1 💚 mvnsite 1m 3s the patch passed
+1 💚 javadoc 0m 44s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 0m 45s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 spotbugs 2m 23s the patch passed
+1 💚 shadedclient 35m 44s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 114m 40s hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 35s The patch does not generate ASF License warnings.
255m 24s
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8656/3/artifact/out/Dockerfile
GITHUB PR #8656
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets jsonlint
uname Linux 77ac0c0c0087 5.15.0-185-generic #195-Ubuntu SMP Fri Jun 19 17:11:50 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 8bcf90f
Default Java Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.11+10-1-24.04.2-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8656/3/testReport/
Max. process+thread count 883 (vs. ulimit of 10000)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8656/3/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@slfan1989 slfan1989 self-assigned this Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@slfan1989 slfan1989

Labels

trunk YARN

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@myloginid @hadoop-yetus @slfan1989