Skip to content

Commit 520e171

Browse files
committed
Simplify the iterator adaptive splitting strategy
Before, when an iterator job was stolen, we would reset the split count all the way back to `current_num_threads` to adaptively split jobs more aggressively when threads seem to need more work. This ends up splitting a lot farther than a lot of people expect, especially in the tail end of a computation when threads are fighting over what's left. Excess splitting can also be harmful for things like `fold` or `map_with` that want to share state as much as possible. We can get a much lazier "adaptive" effect by just not updating the split count when we split a stolen job, effectively giving it only _one_ extra boost of splitting.
1 parent 6a01573 commit 520e171

File tree

1 file changed

+2
-3
lines changed

1 file changed

+2
-3
lines changed

src/iter/plumbing/mod.rs

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -273,9 +273,8 @@ impl Splitter {
273273
let Splitter { splits } = *self;
274274

275275
if stolen {
276-
// This job was stolen! Reset the number of desired splits to the
277-
// thread count, if that's more than we had remaining anyway.
278-
self.splits = cmp::max(crate::current_num_threads(), self.splits / 2);
276+
// This job was stolen! Leave the `splits` count alone while we go ahead and split
277+
// anyway, so we dynamically get more splitting for jobs that are moving a lot.
279278
true
280279
} else if splits > 0 {
281280
// We have splits remaining, make it so.

0 commit comments

Comments
 (0)