Skip to content

feat(clone): turbo clone #9904

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Mar 5, 2025
Merged

feat(clone): turbo clone #9904

merged 8 commits into from
Mar 5, 2025

Conversation

NicholasLYang
Copy link
Contributor

@NicholasLYang NicholasLYang commented Feb 5, 2025

Description

Implements turbo clone, essentially a small wrapper around git clone that does simple optimizations like blobless clones for local development and treeless clones for CI.

Testing Instructions

Wrote some tests but not convinced they're actually effective. See comments below

Copy link

vercel bot commented Feb 5, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
examples-basic-web ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 4, 2025 8:03pm
examples-designsystem-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 4, 2025 8:03pm
examples-gatsby-web ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 4, 2025 8:03pm
examples-kitchensink-blog ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 4, 2025 8:03pm
examples-native-web ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 4, 2025 8:03pm
examples-nonmonorepo ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 4, 2025 8:03pm
examples-svelte-web ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 4, 2025 8:03pm
examples-tailwind-web ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 4, 2025 8:03pm
examples-vite-web ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 4, 2025 8:03pm

@@ -1274,7 +1285,6 @@ pub async fn run(
};

cli_args.command = Some(command);
cli_args.cwd = Some(repo_root.as_path().to_owned());
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as far as I can tell, we don't actually use the cwd as repo root anywhere. can correct if this isn't true

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a necessary change for this PR? repo_root looks like a bad name since it can get it's value from cli_args.cwd. This value will now no longer be made absolute if it is given as a relative value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately it is necessary, although I can just save the old cwd instead. We don't want the repo root to be our cwd and in fact, we probably don't want to be in a repo in the first place for turbo clone. This is a pretty big departure from our other commands.

Make sure we allow partial clones

Do a blobless clone
$ ${TURBO} clone file://$(pwd)/basic-monorepo.t basic-monorepo-blobless --local
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not convinced we're actually doing a real blobless clone here. If anybody has ideas for better testing that doesn't involve hitting a remote git repo (unless we really have to ig), I'm open to ideas.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like the config claims it's blobless but my suspicion is that git only does a blobless clone if you really need it. For trivial repos it just doesn't make sense. I tried adding a test that adds a file, then deletes in in a second commit, and then does a blobless clone, with the idea that the file should not be included in the clone since it's not a blob that is reachable from HEAD. But the file was indeed cloned, so who knows.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're not convinced this test is actually testing the behavior we want than we shouldn't include it. Only suggestion for testing would be to set up a local git server and clone from that, but that feels a bit like overkill.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could try to clone against GitHub but that's a little sketchy for a test

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer not to do that, but if we need a specific response from a git server we probably either need to hit GitHub or somehow run a similar server locally. Since we're still exploring this I think it's fine to test it out where we want to use it instead of investing into a testing strategy. I do still stand that if you don't think this test is testing the behavior, we should remove it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I think this test is still worth doing because it's testing that we have indeed set up the repo to be a treeless/blobless clone. From there, it's up to the server to respect this config, but we can't control that.

Copy link
Member

@chris-olszewski chris-olszewski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new tests seem to be unhappy

@@ -1274,7 +1285,6 @@ pub async fn run(
};

cli_args.command = Some(command);
cli_args.cwd = Some(repo_root.as_path().to_owned());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a necessary change for this PR? repo_root looks like a bad name since it can get it's value from cli_args.cwd. This value will now no longer be made absolute if it is given as a relative value.

Make sure we allow partial clones

Do a blobless clone
$ ${TURBO} clone file://$(pwd)/basic-monorepo.t basic-monorepo-blobless --local
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're not convinced this test is actually testing the behavior we want than we shouldn't include it. Only suggestion for testing would be to set up a local git server and clone from that, but that feels a bit like overkill.

@NicholasLYang NicholasLYang merged commit cf5a1f9 into main Mar 5, 2025
39 checks passed
@NicholasLYang NicholasLYang deleted the nicholasyang/turbo-clone branch March 5, 2025 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants