-
Notifications
You must be signed in to change notification settings - Fork 36
Improve Revs Performance by Avoiding Repeated Tree Checkouts and Adding Version Indexing #1577
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Improve Revs Performance by Avoiding Repeated Tree Checkouts and Adding Version Indexing #1577
Conversation
@smoelius, here’s a summary of the changes I made in this PR:
|
Thanks, @suratkhan! I'll do my best to review this this week. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I left some comments below. I think they may require significant changes, so I'm going to pause here and let you reflect on them.
I really appreciate you working on this.
.next() | ||
.unwrap_or_else(|| { | ||
Err(anyhow!("Could not determine latest `clippy_utils` version")) | ||
})?, | ||
} | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this declaration of rev
is the only thing in this file that should have to change.
Many of the other changes seem like unnecessary reformatting.
If I'm not mistaken in thinking that, could you please put everything else back the way it was?
dylint/src/package_options/mod.rs
Outdated
.next() | ||
.unwrap_or_else(|| { | ||
Err(anyhow!("Could not determine latest `clippy_utils` version")) | ||
})?, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I would prefer to not introduce another iterator implementation.
For this case (where we cannot use binary search), could you instead just have a function that does a linear search to find the commit where the most recent Rust version became active?
dylint/src/package_options/revs.rs
Outdated
} | ||
|
||
// Sort versions for binary search | ||
versions.sort_by(|a, b| a.version.cmp(&b.version)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I probably wasn't sufficiently clear in what I was imagining, but it was that the binary search would be over the commits rather than the versions.
I see that the current code visits every commit and extracts the Cargo.toml and rust-toolchain blobs. I expect that to be faster than full checkouts, but it still means the code does work at every commit, which I think we can avoid.
Would it be possible to adjust the code so that vector holds commits rather than versions?
Also, could adding to the vector be done in a lazy, doubling manner? E.g.:
- read 1 commit and add it to the vector
- if the oldest read commit is not sufficiently old, read 2 more commits and add them to the vector
- if the oldest read commit is not sufficiently old, read 4 more commits and add them to the vector
- etc.
Eventually, a commit will be found with a Rust version older than the desired Rust version. At that point, the binary search can begin over all of the commits accumulated in the vector.
Please let me know if my description is too informal.
hello @smoelius, I have applied the things you mentioned in the comments. Here is a summary of what I did:
|
This PR solves #1559 by refactoring the
Revs
implementation to improve performance by:Rev
structs from commit trees.checkout_tree
calls.find_version
lookups.