gradle-worker-classpath never cleaned up (polluting temp)

[dweiss]

Classpath passed via options file is never cleaned up from temp.

https://github.com/gradle/gradle/blob/master/subprojects/core/src/main/java/org/gradle/process/internal/worker/child/ApplicationClassesInSystemClassLoaderWorkerImplementationFactory.java#L100

This creates literally hundreds of files on each build for large, multi-module projects.

Is there any workaround? This is really annoying.

[martinda]

I have over 700 /tmp/gradle-worker-classpath**txt files after only a few days. Something is wrong.

[dweiss]

Well, yes -- the above snippet I included is wrong. :) No mystery about it.

[mikemccand]

FYI this bug is rather serious, at least for my use case.

I discovered my /tmp directory was littered with ~675K temporary files that gradle failed to delete! We are discussing the problem and workarounds on this Apache Lucene issue.

Is the above snippet easy to fix? Hmm, the snippet looks wrong, because the source has presumably changed since @dweiss opened this issue. @dweiss is it possible to link to a stable source (to a specific githash) instead?

[dweiss]

it's still the same code area -- temporary file creation without proper deletion.

[dweiss]

@adammurdoch Just pinging to bring your attention to this issue - would you happen to know if it's a bug or a feature we can somehow avoid?

[dweiss]

Similar issue occurs with tests - this time they leave behind temp files under the task's tmp, which is at least project-local.

gradle/subprojects/testing-jvm/src/main/java/org/gradle/api/internal/tasks/testing/junit/JUnitTestFramework.java

Lines 42 to 47 in c626475

	public JUnitTestFramework(Test testTask, DefaultTestFilter filter) {
	this.filter = filter;
	options = new JUnitOptions();
	detector = new JUnitDetector(new ClassFileExtractionManager(testTask.getTemporaryDirFactory()));
	}

Related Lucene issues:
https://issues.apache.org/jira/browse/LUCENE-9471
https://issues.apache.org/jira/browse/LUCENE-9468

[strasharo]

Any workaround for this one?

[ikhoon]

I've encountered the same issue. Our CI machine's disk filled up with garbage files (15G ) gradle-worker-classpath...txt...

[BonusLord]

For anyone still suffering from this problem, here's a simple gradle task kludge I've been using as a workaround:

// Gradle has no built-in mechanism for rolling it's log files so we have to manually do it ourselves to prevent 
// gigs of logs from accumulating on disk. >.>

// Additionally, modern versions of gradle now also leak "gradle-worker-classpath" files into the tmp dir, which can
// lead to performance issues after thousands of them accumulate (which has occurred on our Jenkins worker nodes),
// so we clean those up here as well zzz (can remove the classpath file logic if they ever address this: 
// https://github.com/gradle/gradle/issues/12020)

task deleteOldLogFiles {
  doLast {

    long now = System.currentTimeMillis()
    def gradle = project.getGradle()

    def filesToCheck = []

    filesToCheck += new File("${gradle.getGradleUserHomeDir().getAbsolutePath()}/daemon/${gradle.getGradleVersion()}").listFiles()
      .findAll {it.name.endsWith('.out.log')}

    filesToCheck += new File(System.getProperty("java.io.tmpdir")).listFiles()
      .findAll {it.name.startsWith('gradle-worker-classpath')}	
		
    for (File booGradleFile in filesToCheck) {
      long lastMod = booGradleFile.lastModified()
      long ageMs = now - lastMod;
      
      // Delete tmp files that older than 4 days; they are almost certainly just leaks
      if (ageMs < 1000*60*60*24*4) {
        continue
      }
      
      println "Deleting old daemon log file: $booGradleFile"
      booGradleFile.delete()
    }
  }
}

[mikemccand]

This issue is still biting me (Gradle 8.10 on Arch Linux 6.12.4-arch1-1) .. I now have ~93 K of these leftover files from gradle:

raptorlake:trunk[main]$ ls ~/.gradle/.tmp | wc
  93427   93427 4334325

I will remove them manually but the root cause bug really should be fixed?

[ov7a]

Sorry for the late reply.

This issue will be closed as a duplicate of

• Clean up temporary files created via TempFiles #15367

Please add your use case and 👍 to that issue.

If you think our analysis is wrong, please provide us with more detailed information explaining why.

---