Upgrade to Gradle 8.14 and Java 24 minimum #14533

dweiss · 2025-04-22T17:09:58Z

This is an initial set of changes that:

update gradle to 8.14 (rc2), supporting Java 24,
bump minimum Java version to Java 24 (build support: java 24 #14379)
upgrade to an ECJ version that supports Java 24 (none yet) and revert the enabled=false from the patch
upgrade to gradle 8.14 (final)

Showstoppers for now:

ECJ doesn't have a version that supports Java 24 yet. I disabled ecj checks for now to verify other things work.
we should probably wait for the final version of gradle 8.14.

…va 24.

rmuir · 2025-04-23T02:13:02Z

upgrade to an ECJ version that supports Java 24 (none yet) and revert the enabled=false from the patch

I don't understand this one: their language server works fine with java 24 and i'm able to navigate the new JDK24 classfile api, etc. I guess I haven't used it extensively, due to the gradle issue, and they've got some bugfixes since the last release.

dweiss · 2025-04-23T07:06:12Z

I don't understand this one: their language server works fine with java 24 and i'm able to navigate the new JDK24 classfile api, etc. I guess I haven't used it extensively, due to the gradle issue, and they've got some bugfixes since the last release.

I don't fully understand it either - the latest artifact for the batch ecj compiler (what we use) seems to be from march:
https://repo1.maven.org/maven2/org/eclipse/jdt/ecj/3.41.0/

this doesn't support Java 24. I've just downloaded the latest Eclipse and it seems to contain the same batch compiler. Their website says this:

There are some marketplace extensions to support Java 24 but these seem to be external - so... I don't think Eclipse supports Java 24 yet?

rmuir · 2025-04-23T09:10:52Z

I'm using https://github.com/eclipse-jdtls/eclipse.jdt.ls which, is just this same eclipse stuff packaged in a different way?

It had the java 24 support like a week after the java 24 release. Sorry, I don't understand eclipse ecosystem.

dweiss · 2025-04-23T10:03:59Z

I've no idea either. All I know is that if I download eclipse-jdt, it doesn't have support for Java 24 yet. There is a milestone version which supposedly has it - but not an official stable version, as far as I can see. And the maven repo definitely doesn't have the batch compiler supporting java24. That eclipse-jdls has a fork of jdt... which is also weird.

dweiss · 2025-04-23T10:12:03Z

Eclipse 4.36M1 seems to have Java 24 support in the batch compiler. This is marked as "stable" but artifacts for this release are not on maven central yet.

rmuir · 2025-04-23T10:17:56Z

Yeah I think "big old eclipse" lags behind, whereas the jdtls is "the latest" and sees more usage (e.g. all VSCode users and other editors).

There's even a plugin to allow use of jdtls in "big old eclipse": https://github.com/redhat-developer/eclipseide-jdtls/

very confusing ecosystem, but that's my read on it.

dweiss · 2025-04-23T10:38:13Z

I believe they use p2 repositories for interim releases and only do a maven push once in a quarter for stable final releases.

If this becomes a problem, we can pull ecj directly from p2 drops, like here:
https://www.eclipse.org/downloads/download.php?file=/eclipse/downloads/drops4/S-4.36M1-202504031800/ecj-4.36M1.jar&r=1

…from Eclipse's p2 repositories.

dweiss · 2025-04-29T07:59:58Z

I've replaced ecj batch compiler dependency with a direct download from Eclipse's p2 drop repositories. I don't think anybody will even notice the difference and p2 repositories seem to have a lot eager release cadence compared to maven eclipse artifacts (~3 months).

uschindler · 2025-04-29T09:10:32Z

Please give a short notice to me directly before your merge this to main branch so I can update Jenkins Jobs on Policeman.

uschindler · 2025-04-29T09:12:35Z

I am not really happy about downloading from the non-maven repo, but as this won't affect any release, I am fine. If we release main branch we should rellay switch to an offical Maven artifact for ECJ.

dweiss · 2025-04-29T09:15:44Z

Please give a short notice to me directly before your merge this to main branch so I can update Jenkins Jobs on Policeman.

We can update anytime, really. I don't have the pressure to do this but everything seems to work fine so why wait.

uschindler · 2025-04-29T09:32:30Z

Thats fine, we should just make sure (by comment or wahtever) that we can easily switch back in a documented way to official Maven artifacts before we release Main branch.

uschindler · 2025-04-29T09:35:39Z

lucene/core/src/java/org/apache/lucene/store/MMapDirectory.java

@@ -365,7 +365,9 @@ private static <A> MMapIndexInputProvider<A> lookupProvider() {
      // access through the lookup:
      final var constr = lookup.findConstructor(cls, MethodType.methodType(void.class, int.class));
      try {
-        return (MMapIndexInputProvider<A>) constr.invoke(maxPermits);
+        @SuppressWarnings("unchecked")


LOL, whats the reason for this?

We should anways get rid on the whole code here in main branch and merge all files to the main compilation unit (except vectors, of course).

The compiler complains about it. Try reverting and see for yourself.

It does look odd, I agree.

Execution failed for task ':lucene:core:compileJava'. > Compilation failed; see the compiler output below. Note: Recompile with -Xlint:deprecation for details. error: warnings found and -Werror specified /home/.../lucene/core/src/java/org/apache/lucene/store/MMapDirectory.java:368: warning: [unchecked] unchecked cast return (MMapIndexInputProvider<A>) constr.invoke(maxPermits); ^ required: MMapIndexInputProvider<A> found: MMapIndexInputProvider where A is a type-variable: A extends Object declared in method <A>lookupProvider() Note: Some input files use or override a deprecated API. 1 error 1 warning

I opened #14564 to get rid of all this shit.

At moment the other PR is based on Java 23, but it is easy to update to 24. At moment we forgot to change the code, so Java 21 was still in business in Main!

In future, when we update Java version we also need the updates in build system and sourcesets for vectors. So we need to change the mrJarVersions property in Gradle build, too and rename the sourceset folder java23 -> java24

uschindler · 2025-04-29T15:18:26Z

@dweiss: I opened #14564 to get rid of MMapDirexctoy abstractions. I can wait with merging this until this one is merged. The update to Java 24 will be done before I commit.

For the future: Whenever we update Java versions, we also need to make sure to update the minimal versions in the mrJar code parts. At moment, it produces a MR-JAR file with Java 21.... (because that's the version where the APIJARs were extracted from). I fixed this in the other PR, no need to take care of it here. But for the update to Java 25 we should make sure that we update the APIJAR stub extraction also to Java 25.

dweiss · 2025-04-29T19:18:49Z

Ok, I've applied this to main. I will backport gradle upgrade selectively to 10x tomorrow (can't be cherry picked automatically).

dweiss · 2025-04-29T19:18:54Z

Ok, I've applied this to main. I will backport gradle upgrade selectively to 10x tomorrow (can't be cherry picked automatically).

dweiss · 2025-04-29T19:19:01Z

Ok, I've applied this to main. I will backport gradle upgrade selectively to 10x tomorrow (can't be cherry picked automatically).

dweiss · 2025-04-29T19:19:23Z

Ok, I've applied this to main. I will backport gradle upgrade selectively to 10x tomorrow (can't be cherry picked automatically).

uschindler · 2025-04-29T19:27:59Z

I disabled Policeman Jenkins Main Jobs, working on Upgrade. Can you take care of ASF Jenkins?

uschindler · 2025-04-29T19:37:27Z

When testing this I figured out the following warning when starting with Java 24:

WARNING: A restricted method in java.lang.System has been called
WARNING: java.lang.System::load has been called by net.rubygrapefruit.platform.internal.NativeLibraryLoader in an unnamed module (file:/C:/Users/Uwe%20Schindler/.gradle/wrapper/dists/gradle-8.14-bin/38aieal9i53h9rfe7vjup95b9/gradle-8.14/lib/native-platform-0.22-milestone-28.jar)
WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for callers in this module
WARNING: Restricted methods will be blocked in a future release unless native access is enabled

I think we need to change the gradlew / gradlew.cmd files to pass the additional command line option --enable-native-access=ALL-UNNAMED

uschindler · 2025-04-29T19:50:23Z

I updated the other PR and generated the APIJAR for 24. For the next update, basically the same linke this commit needs to be done: 2ad382a

dweiss · 2025-04-29T19:54:14Z

I think we need to change the gradlew / gradlew.cmd files to pass the additional command line option --enable-native-access=ALL-UNNAMED

I didn't touch the launcher scripts... could be that the latest from gradle already does it but we have quite a few tweaks in there that'd have to be ported. I'll look into this tomorrow, hopefully.

dweiss · 2025-04-29T19:54:54Z

I disabled Policeman Jenkins Main Jobs, working on Upgrade. Can you take care of ASF Jenkins?

I've switched all jobs on asf jenkins to Java 24. Hopefully didn't miss anything.

uschindler · 2025-04-29T20:01:00Z

I think we need to change the gradlew / gradlew.cmd files to pass the additional command line option --enable-native-access=ALL-UNNAMED

I didn't touch the launcher scripts... could be that the latest from gradle already does it but we have quite a few tweaks in there that'd have to be ported. I'll look into this tomorrow, hopefully.

I think adding that as command line option should be easy to do. Basically the parameter is available since Java 20 or 21 (since Panama exists), so it should also work with launching from older java versions and it won't fail with error message before ours.

uschindler · 2025-04-29T20:18:47Z

Policeman Jenkins was also updated. I hope the randomization and RUNTIME_JAVA_HOME with 25 works, too. If something breaks, I will keep an eye on it.

uschindler · 2025-04-29T20:23:05Z

I also updated the 10.x release tester job to use Java 24, too. This will affect the next release. Of course Gradle backport is needed.

dweiss · 2025-04-30T08:52:52Z

I think we need to change the gradlew / gradlew.cmd files to pass the additional command line option --enable-native-access=ALL-UNNAMED

gradle is now launched with --jar and contains a manifest entry for this (no need to tweak the scripts). Should be fine with #14592

uschindler · 2025-04-30T10:55:09Z

I got a strange error:

What went wrong:
Execution failed for task ':downloadEcjDrop'.

A failure occurred while executing de.undercouch.gradle.tasks.download.internal.DefaultWorkerExecutorHelper$DefaultWorkAction
javax.net.ssl.SSLHandshakeException: (certificate_unknown) PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

But on second try it worked.

Another problem is that the ECJ compiler config is not using 24 but still 23 in this PR. I will fix this in a commit.

uschindler · 2025-04-30T10:55:26Z

Fixed: 0550cc6

uschindler · 2025-04-30T10:55:35Z

One information about Exclipse: With the recent commit all works fine. It just sometimes fails with a compilation error (but that's not new) regarding: "Permitted type LeafReaderContext does not declare org.apache.lucene.index.IndexReaderContext as a direct supertype"

This error is bullshit but something seems to confuse the compile inside the Eclipse IDE.

Anyways: If you want to use exclipse, you need to install the Milestone 1 of 2025-06 release, otherwise it won't work (same reason like discussed above). Luckily on starting Eclipse it asks you if you want to try the milestone release.

* main: (51 commits) Fix ECJ compiler config for Java 24 (also affects Eclipse IDE) Correct shebang in gradlew. apache#14592 Overwrite gradlew scripts with up-to-date ones and apply Lucene customizations. (apache#14592) Remove abstractions of MMapDirectory and its "fake" MR-JAR support and move it to main sourceSet (vectors untouched) (apache#14564) deps(java): bump org.gradle.toolchains.foojay-resolver-convention (apache#14573) deps(java): bump com.gradle.develocity from 3.18.2 to 4.0.1 (apache#14585) deps(java): bump nl.littlerobots.version-catalog-update (apache#14568) Bump expected base Java version to 24. apache#14533 Create file open hints on IOContext to replace ReadAdvice (apache#14482) deps(java): bump com.github.ben-manes.versions from 0.51.0 to 0.52.0 (apache#14574) deps(java): bump org.owasp.dependencycheck from 7.2.0 to 12.1.1 (apache#14584) deps(java): bump org.eclipse.jgit:org.eclipse.jgit (apache#14579) deps(java): bump com.diffplug.spotless from 6.9.1 to 7.0.3 (apache#14588) deps(java): bump com.gradle.common-custom-user-data-gradle-plugin (apache#14587) deps(java): bump net.ltgt.errorprone from 4.1.0 to 4.2.0 (apache#14567) deps(java): bump org.locationtech.jts:jts-core from 1.17.0 to 1.20.0 (apache#14586) deps(java): bump org.apache.opennlp:opennlp-tools from 2.5.3 to 2.5.4 (apache#14580) deps(java): bump asm from 9.6 to 9.8 (apache#14578) deps(java): bump commons-codec:commons-codec from 1.17.2 to 1.18.0 (apache#14581) deps(java): bump net.java.dev.javacc:javacc from 7.0.12 to 7.0.13 (apache#14576) ... # Conflicts: # lucene/CHANGES.txt

dweiss · 2025-04-30T17:34:36Z

Thank you, Uwe.

dweiss added 2 commits April 22, 2025 19:04

Initial set of updates to gradle 8.14-rc-2. Minimum Java bumped to Ja…

067835f

…va 24.

No ECJ version supports Java 24 yet.

74f6694

github-project-automation bot added this to OpenSearch Lucene & Core Performance Tracking Apr 22, 2025

github-project-automation bot moved this to Open in OpenSearch Lucene & Core Performance Tracking Apr 22, 2025

dweiss self-assigned this Apr 22, 2025

dweiss added this to the 11.0.0 milestone Apr 22, 2025

github-actions bot added the module:core/store label Apr 22, 2025

dweiss removed the module:core/store label Apr 22, 2025

Bump Java to 24 in gh workflows.

5e51f62

github-actions bot added the module:core/store label Apr 22, 2025

This was referenced Apr 22, 2025

Upgrade to gradle 8.14-rc-2 #14519

Closed

build support: java 24 #14379

Closed

dweiss linked an issue Apr 22, 2025 that may be closed by this pull request

build support: java 24 #14379

Closed

dweiss added 2 commits April 22, 2025 19:22

Upgrade groovy to work with java 24.

4c7a50e

./gradlew regenerate --rerun-tasks (update TLDs).

b4dce8b

github-actions bot added the module:analysis label Apr 22, 2025

dweiss added 2 commits April 29, 2025 08:51

Merge branch 'main' into gradle-8.14

94508cd

Update to gradle 8.14, official.

5928186

dweiss changed the title ~~Initial upgrade to Gradle 8.14 (rc2)~~ Upgrade to Gradle 8.14 Apr 29, 2025

Replace ecj compiler dependency with a download of ecj compiler drop …

6bd4658

…from Eclipse's p2 repositories.

dweiss marked this pull request as ready for review April 29, 2025 09:15

uschindler reviewed Apr 29, 2025

View reviewed changes

uschindler changed the title ~~Upgrade to Gradle 8.14~~ Upgrade to Gradle 8.14 and Java 24 minimum Apr 29, 2025

dweiss merged commit 9c67088 into apache:main Apr 29, 2025
10 checks passed

github-project-automation bot moved this from Open to Merged in OpenSearch Lucene & Core Performance Tracking Apr 29, 2025

dweiss deleted the gradle-8.14 branch April 29, 2025 19:18

dweiss mentioned this pull request Apr 30, 2025

Revert jenkins to use lucene label instead of sticky lucene-solr-1 #14590

Closed

dweiss added a commit that referenced this pull request Apr 30, 2025

Bump expected base Java version to 24. #14533

d0f2c9e

dweiss mentioned this pull request Apr 30, 2025

Upgrade to gradle 8.14 on branch_10x (backport). #14591

Merged

Upgrade to Gradle 8.14 and Java 24 minimum #14533

Upgrade to Gradle 8.14 and Java 24 minimum #14533

Uh oh!

Conversation

dweiss commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rmuir commented Apr 23, 2025

Uh oh!

dweiss commented Apr 23, 2025

Uh oh!

rmuir commented Apr 23, 2025

Uh oh!

dweiss commented Apr 23, 2025

Uh oh!

dweiss commented Apr 23, 2025

Uh oh!

rmuir commented Apr 23, 2025

Uh oh!

dweiss commented Apr 23, 2025

Uh oh!

dweiss commented Apr 29, 2025

Uh oh!

uschindler commented Apr 29, 2025

Uh oh!

uschindler commented Apr 29, 2025

Uh oh!

dweiss commented Apr 29, 2025

Uh oh!

uschindler commented Apr 29, 2025

Uh oh!

uschindler Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

dweiss Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

dweiss Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

uschindler Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

uschindler Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

uschindler commented Apr 29, 2025

Uh oh!

Uh oh!

dweiss commented Apr 29, 2025

Uh oh!

dweiss commented Apr 29, 2025

Uh oh!

dweiss commented Apr 29, 2025

Uh oh!

dweiss commented Apr 29, 2025

Uh oh!

uschindler commented Apr 29, 2025

Uh oh!

uschindler commented Apr 29, 2025

Uh oh!

uschindler commented Apr 29, 2025

Uh oh!

dweiss commented Apr 29, 2025

Uh oh!

dweiss commented Apr 29, 2025

Uh oh!

uschindler commented Apr 29, 2025

Uh oh!

uschindler commented Apr 29, 2025

Uh oh!

uschindler commented Apr 29, 2025

Uh oh!

dweiss commented Apr 30, 2025

Uh oh!

uschindler commented Apr 30, 2025

Uh oh!

uschindler commented Apr 30, 2025

Uh oh!

uschindler commented Apr 30, 2025

dweiss commented Apr 22, 2025 •

edited

Loading