HADOOP-19776: trunk pre-commits for native code still try to use Java 8 #8166

cnauroth · 2026-01-05T23:04:05Z

Description of PR

Pre-commits contain some steps specifically triggered for native code patches to check cross-platform compatibility. These are still trying to run with Java 8 targets. (Example: #8151 .) This patch introduces Java 17 and 21 in the Rocky 8 image.

How was this patch tested?

Relying on pre-submit results. I'm temporarily including the native code change from #8151 to try to trigger it. I won't include that when I commit this.

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hadoop-yetus · 2026-01-06T02:06:14Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	33m 51s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	44m 12s		trunk passed
-1 ❌	compile	3m 17s	/branch-compile-root.txt	root in trunk failed.
+1 💚	mvnsite	1m 45s		trunk passed
+1 💚	shadedclient	30m 13s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 9s		the patch passed
-1 ❌	compile	3m 6s	/patch-compile-root.txt	root in the patch failed.
-1 ❌	cc	3m 6s	/patch-compile-root.txt	root in the patch failed.
-1 ❌	golang	3m 6s	/patch-compile-root.txt	root in the patch failed.
-1 ❌	javac	3m 6s	/patch-compile-root.txt	root in the patch failed.
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	mvnsite	1m 30s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	shadedclient	30m 25s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	27m 8s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		180m 19s

Reason	Tests
Failed junit tests	hadoop.security.token.delegation.TestZKDelegationTokenSecretManager

Subsystem	Report/Notes
Docker	ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8166/1/artifact/out/Dockerfile
GITHUB PR	#8166
Optional Tests	dupname asflicense codespell detsecrets shellcheck shelldocs compile cc mvnsite javac unit golang
uname	Linux c7d8ae3cf4b3 5.15.0-160-generic #170-Ubuntu SMP Wed Oct 1 10:06:56 UTC 2025 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `2653f7b`
Default Java	Debian-17.0.17+10-Debian-1deb12u1
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8166/1/testReport/
Max. process+thread count	1271 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8166/1/console
versions	git=2.39.5 maven=3.9.11 shellcheck=0.9.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

pan3793 · 2026-01-06T04:17:58Z

dev-support/Jenkinsfile

        // This stage serves as a means of cross platform validation, which is
        // really needed to ensure that any C++ related/platform change doesn't
-        // break the Hadoop build on Rocky Linux 8.
-        stage ('precommit-run Rocky Linux 8') {


can we install openjdk 17 instead of removing rocky linux 8? I think we should at least keep one rhel-like OS in pre-commit, along-side with other debian-like OS

I've updated the patch to keep Rocky 8 and install Java 17. We can reconsider modernizing the OS targets separately from this patch.

dev-support/Jenkinsfile

cnauroth · 2026-01-09T00:26:09Z

These new build targets I'm trying to set up are failing, related to the Fuse dependency:

[WARNING] --   Package 'fuse', required by 'virtual:world', not found
[WARNING] -- Failed to find Linux FUSE libraries or include files.  Will not build FUSE client.
[WARNING] CMake Error at CMakeLists.txt:215 (message):
[WARNING]   Required component fuse_dfs could not be built.

I think I'm going to switch gears to take the simplest path to fixing pre-submit by getting Java 17 into the existing build targets. Then, we can look at modernizing the platforms in a subsequent patch.

…Java 17

cnauroth · 2026-01-09T18:01:01Z

I have updated this patch to stick with the original build targets, but make sure they are using Java 17.

hadoop-yetus · 2026-01-09T18:04:16Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8166/2/console in case of problems.

hadoop-yetus · 2026-01-09T19:14:13Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	34m 58s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shellcheck	0m 0s		Shellcheck was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+0 🆗	hadolint	0m 0s		hadolint was not available.
+0 🆗	jsonlint	0m 0s		jsonlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
-1 ❌	mvninstall	1m 46s	/branch-mvninstall-root.txt	root in trunk failed.
-1 ❌	compile	0m 32s	/branch-compile-root.txt	root in trunk failed.
+1 💚	mvnsite	2m 38s		trunk passed
-1 ❌	shadedclient	6m 36s		branch has errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 12s		the patch passed
-1 ❌	compile	0m 32s	/patch-compile-root.txt	root in the patch failed.
-1 ❌	cc	0m 32s	/patch-compile-root.txt	root in the patch failed.
-1 ❌	golang	0m 32s	/patch-compile-root.txt	root in the patch failed.
-1 ❌	javac	0m 32s	/patch-compile-root.txt	root in the patch failed.
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	mvnsite	1m 31s		the patch passed
-1 ❌	shadedclient	3m 4s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	22m 26s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 49s		The patch does not generate ASF License warnings.
		72m 3s

Subsystem	Report/Notes
Docker	ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8166/2/artifact/out/Dockerfile
GITHUB PR	#8166
Optional Tests	dupname asflicense codespell detsecrets shellcheck shelldocs hadolint jsonlint compile cc mvnsite javac unit golang
uname	Linux dbaeb456cb0d 5.15.0-160-generic #170-Ubuntu SMP Wed Oct 1 10:06:56 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `0aa0912`
Default Java	Red Hat, Inc.-1.8.0_472-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8166/2/testReport/
Max. process+thread count	3097 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8166/2/console
versions	git=2.43.7 maven=3.9.11
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

cnauroth · 2026-01-09T20:39:46Z

jenkins.sh overrides the Yetus --java-home to a path with a -amd64 suffix, which doesn't exist in the Rocky image. Then it ends up defaulting back to Java 8. I might have to fake this with symlinks.

[2026-01-09T18:38:26.880Z] WARNING: JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 does not exist. Dockermode: attempting to switch to another.
[2026-01-09T18:38:26.880Z] WARNING: Setting /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.472.b08-1.el8.x86_64 as the JAVA_HOME.

edwardcapriolo · 2026-01-09T20:59:11Z

Modern maven seems to leverage:
~/.m2/toolchains.xml

However based on maven versions it is no magic.

… by jenkins.sh

hadoop-yetus · 2026-01-09T21:13:54Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8166/3/console in case of problems.

hadoop-yetus · 2026-01-10T01:31:09Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	32m 16s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shellcheck	0m 0s		Shellcheck was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+0 🆗	hadolint	0m 0s		hadolint was not available.
+0 🆗	jsonlint	0m 0s		jsonlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	43m 40s		trunk passed
+1 💚	compile	19m 19s		trunk passed with JDK Red Hat, Inc.-21.0.9+10-LTS
+1 💚	compile	20m 9s		trunk passed with JDK Red Hat, Inc.-17.0.17+10-LTS
+1 💚	mvnsite	3m 4s		trunk passed
+1 💚	shadedclient	120m 55s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 24s		the patch passed
+1 💚	compile	17m 56s		the patch passed with JDK Red Hat, Inc.-21.0.9+10-LTS
+1 💚	cc	17m 56s		the patch passed
+1 💚	golang	17m 56s		the patch passed
+1 💚	javac	17m 56s		the patch passed
+1 💚	compile	20m 7s		the patch passed with JDK Red Hat, Inc.-17.0.17+10-LTS
+1 💚	cc	20m 7s		the patch passed
+1 💚	golang	20m 7s		the patch passed
+1 💚	javac	20m 7s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	mvnsite	3m 6s		the patch passed
+1 💚	shadedclient	37m 59s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	23m 39s		hadoop-common in the patch passed.
+1 💚	asflicense	2m 19s		The patch does not generate ASF License warnings.
		260m 28s

Subsystem	Report/Notes
Docker	ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8166/3/artifact/out/Dockerfile
GITHUB PR	#8166
Optional Tests	dupname asflicense codespell detsecrets shellcheck shelldocs hadolint jsonlint compile cc mvnsite javac unit golang
uname	Linux 43372fbc8381 5.15.0-160-generic #170-Ubuntu SMP Wed Oct 1 10:06:56 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `e256ba6`
Default Java	Red Hat, Inc.-17.0.17+10-LTS
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-21.0.9.0.10-1.el8.x86_64:Red Hat, Inc.-21.0.9+10-LTS /usr/lib/jvm/java-17-openjdk-17.0.17.0.10-1.el8.x86_64:Red Hat, Inc.-17.0.17+10-LTS
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8166/3/testReport/
Max. process+thread count	3109 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8166/3/console
versions	git=2.43.7 maven=3.9.11
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

ayushtkn

LGTM

There is a blank line warning by the CI, Once fixed we can merge this

ayushtkn · 2026-01-10T06:31:25Z

hadoop-common-project/hadoop-common/src/main/native/src/exception.c

 {
-// MT-Safe under Solaris or glibc >= 2.32 not supporting sys_errlist/sys_nerr
-#if defined(__sun)
-  #define USE_STR_ERROR
-#elif defined(__GLIBC_PREREQ)
-  #if __GLIBC_PREREQ(2, 32)
-    #define USE_STR_ERROR
+/* STD_ERROR is the new standard. Alpine musc does not want to be 'detected' it want to be pure and modern. Thus we detect the old glib and handle. */  
+#ifdef __GLIBC__
+  #if defined(__GLIBC_PREREQ)
+    #if __GLIBC_PREREQ(2, 32)
+      #define USE_STR_ERROR
+    #endif
  #endif
+#else
+  #define USE_STR_ERROR
 #endif

+
 #if defined(USE_STR_ERROR)
  return strerror(errnum); 


are we overlapping with:
#8151

@edwardcapriolo can you check once?

Edward's PR is what made me first notice this problem. I had temporarily included it here just so Yetus would trigger a native code patch run. I'm going to commit #8151 separately.

cnauroth · 2026-01-12T18:04:29Z

@pan3793 and @ayushtkn , thank you for the reviews. I committed this to trunk.

cnauroth · 2026-01-12T18:07:31Z

I forgot to remove the exception.c change that was only part of testing, so you'll see one extra commit. I reverted it so I can give the real author proper credit in a separate commit.

cnauroth added 2 commits January 5, 2026 22:56

HADOOP-19776: trunk pre-commits for native code still try to use Java 8

ea7f1fd

HADOOP-19776: temporarily include a native code change

2653f7b

github-actions bot added trunk Common Infra Native labels Jan 5, 2026

cnauroth requested review from brumi1024, slfan1989, vinayakumarb and zhtttylz and removed request for brumi1024 January 5, 2026 23:04

This was referenced Jan 5, 2026

HADOOP-19773. [JDK17] Use JDK17 as default in precommit Docker image. #8162

Closed

HADOOP-19756 make std_error adopters have less friction with hadoop n… #8151

Closed

pan3793 reviewed Jan 6, 2026

View reviewed changes

dev-support/Jenkinsfile Outdated Show resolved Hide resolved

HADOOP-19776: keep the original build targets but make sure they use …

0aa0912

…Java 17

HADOOP-19776: Add Java 21 too and symlinks for --java-home paths used…

e256ba6

… by jenkins.sh

ayushtkn approved these changes Jan 10, 2026

View reviewed changes

ayushtkn reviewed Jan 10, 2026

View reviewed changes

pan3793 approved these changes Jan 11, 2026

View reviewed changes

cnauroth closed this in 6fae1cf Jan 12, 2026

cnauroth mentioned this pull request Jan 12, 2026

YARN-11919 segfault with get_user_info #8177

Open

HADOOP-19776: trunk pre-commits for native code still try to use Java 8 #8166

HADOOP-19776: trunk pre-commits for native code still try to use Java 8 #8166

Conversation

cnauroth commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

How was this patch tested?

For code changes:

Uh oh!

hadoop-yetus commented Jan 6, 2026

Uh oh!

pan3793 Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

cnauroth Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cnauroth commented Jan 9, 2026

Uh oh!

cnauroth commented Jan 9, 2026

Uh oh!

hadoop-yetus commented Jan 9, 2026

Uh oh!

hadoop-yetus commented Jan 9, 2026

Uh oh!

cnauroth commented Jan 9, 2026

Uh oh!

edwardcapriolo commented Jan 9, 2026

Uh oh!

hadoop-yetus commented Jan 9, 2026

Uh oh!

hadoop-yetus commented Jan 10, 2026

Uh oh!

ayushtkn left a comment

Choose a reason for hiding this comment

Uh oh!

ayushtkn Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

cnauroth Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

cnauroth commented Jan 12, 2026

Uh oh!

cnauroth commented Jan 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

cnauroth commented Jan 5, 2026 •

edited

Loading