Skip to content

HADOOP-19950. Capture surefire diagnostics on fork timeout in CI - #8651

Open
pan3793 wants to merge 1 commit into
apache:trunkfrom
pan3793:HADOOP-19950
Open

HADOOP-19950. Capture surefire diagnostics on fork timeout in CI#8651
pan3793 wants to merge 1 commit into
apache:trunkfrom
pan3793:HADOOP-19950

Conversation

@pan3793

@pan3793 pan3793 commented Aug 2, 2026

Copy link
Copy Markdown
Member

Description of PR

Set forkedProcessExitTimeoutInSeconds (60s) in the root surefire
configuration so that a hung fork is force-killed after a grace window
and surefire writes the dumpstream (thread dump) naming the non-daemon
thread keeping the fork alive. Without this, the only signal is a
generic "timeout in the fork" error.

Expand the test-logs artifact upload globs from:

**/target/*.log
**/target/*.xml

to:

**/target/**/*.log
**/target/surefire-reports/**

The original globs only matched files directly under target/
(site.xml, plugin-enhanced.xml), missing the actual test reports and
thread dumps in nested directories. The new globs capture TEST-*.xml,
*.dump, and *.dumpstream surefire output needed to diagnose fork
hangs.

How was this patch tested?

Verified via GitHub Actions CI on a fork branch. With the previous
globs, the uploaded artifact was 7KB (2 incidental files). With the new
globs, the artifact is ~112MB and contains the full surefire-reports/
directory, including the *.dump thread dump that named the culprit
test keeping a fork alive.

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (HADOOP-19950)?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

AI Tooling

If an AI tool was used:

Contains content generated by GLM 5.2

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves CI diagnostics for hung Maven Surefire forks by adding a forced-exit grace window and by expanding the GitHub Actions artifact upload globs to actually include Surefire reports and dumpstreams/thread dumps.

Changes:

  • Add forkedProcessExitTimeoutInSeconds (60s) to the root Surefire configuration to force-kill stuck forks after the existing timeout triggers.
  • Expand CI artifact upload paths to capture nested target/**/*.log and full target/surefire-reports/** contents (including TEST-*.xml, *.dump, *.dumpstream).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
hadoop-project/pom.xml Adds a Surefire exit-timeout property and wires it into the global Surefire plugin configuration to produce better diagnostics on fork timeouts.
.github/workflows/tmpl_build_and_test.yml Updates failure-only artifact globs to include full Surefire reports/dumps needed to debug fork hangs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 19m 29s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 yamllint 0m 0s yamllint was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 1m 44s Maven dependency ordering for branch
+1 💚 mvninstall 41m 38s trunk passed
+1 💚 compile 16m 26s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 compile 16m 33s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 mvnsite 18m 35s trunk passed
+1 💚 javadoc 9m 48s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 9m 36s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 shadedclient 142m 33s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 28s Maven dependency ordering for patch
+1 💚 mvninstall 35m 57s the patch passed
+1 💚 compile 16m 22s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javac 16m 22s the patch passed
+1 💚 compile 16m 35s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 javac 16m 35s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 18m 20s the patch passed
+1 💚 javadoc 9m 49s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 9m 37s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 shadedclient 67m 18s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 836m 51s /patch-unit-root.txt root in the patch failed.
+1 💚 asflicense 1m 53s The patch does not generate ASF License warnings.
1140m 15s
Reason Tests
Failed junit tests hadoop.yarn.server.router.webapp.TestRouterWebServicesREST
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8651/1/artifact/out/Dockerfile
GITHUB PR #8651
Optional Tests dupname asflicense codespell detsecrets yamllint compile javac javadoc mvninstall mvnsite unit shadedclient xmllint
uname Linux 675c45a373f2 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 1531c26
Default Java Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.11+10-1-24.04.2-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8651/1/testReport/
Max. process+thread count 3619 (vs. ulimit of 10000)
modules C: hadoop-project . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8651/1/console
versions git=2.43.0 maven=3.9.15
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

Set forkedProcessExitTimeoutInSeconds so surefire force-kills hung forks
after a 60s grace window and writes the dumpstream naming the non-daemon
thread keeping the fork alive. Expand the test-logs artifact to cover
surefire-reports/ (TEST-*.xml, *.dump, *.dumpstream) and nested .log
files, which the previous *.log/*.xml globs missed.

Assisted-by: GLM 5.2

@ajfabbri ajfabbri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (assuming clean CI run)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants