Skip to content

v0.3.3
Compare
Choose a tag to compare
@cloventt cloventt released this 05 Mar 20:18
· 10 commits to main since this release
v0.3.3
3e8832c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's Changed

  • setting a CPU request no longer also sets a limit automatically
  • if you need to set a CPU limit on a driver/executor, call set_executor_cores_limit() or set_driver_cores_limit() on the builder
  • those methods can also be explicitly passed None to disable limits
  • otherwise the value passed must be an int that is at least 1 and greater than the requested CPUs

Why the change

Most k8s clusters are running pods as Docker containers, and so CPU limits are implemented using the kernel's ability to schedule processes on the CPU. In our experience, this can cause instability and poor performance for some applications. Most importantly, it can also result in poor resource utilisation on the node. Without the limit, the pods can use as much CPU as is available on the node, which can make workloads perform better.

Assets 2
Loading