Working with LFS repositries

[ghareeb-falazi]

Hi there,

Can copybara operate with repositories with files managed by Git LFS? My scenario is as follows:

• I have a private repository on github that has many artifacts (some of which are large files which are managed by LFS). This repository acts as a source repository.
• I also have another private repository on github which acts as the target repository.
• The source repository gets frequent pushes, and from time to time we need to selectively copy certain files from it to the target repository.
• when using copybara to do so, we get an error message during the task: "Git Destination: Checking out master":

Error downloading object: (...): batch request: missing protocol: ""
Errors logged to (...)
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: (...): smudge filter lfs failed

which seems as a problem with relevant to git-lfs.

So does copybara support repositories with LFS-managed files?

[mikelalcon]

Seems we don't :). We've never tested.

Is is possible to share the stacktrace?

[mikelalcon]

We might need to add something like lfs.url = xxxxx either in the repo config or global config.

[ghareeb-falazi]

So as a workaround I found out that if the .gitconfig file does not include the following section:

[filter "lfs"]
	clean = git-lfs clean -- %f
	smudge = git-lfs smudge -- %f
	process = git-lfs filter-process
	required = true

Then copybara handles lfs repositories just fine.

[kjellander-unity]

To help future readers: newer versions of git-lfs (e.g. when installing ubuntu:20.04 docker image) will setup system-wide Git LFS configuration in /etc/gitconfig for you, making Copybara error out when working with Git LFS enabled repos. Simplest workaround seems to be just deleting the file.

[CookieSalad]

To help future readers: newer versions of git-lfs (e.g. when installing ubuntu:20.04 docker image) will setup system-wide Git LFS configuration in /etc/gitconfig for you, making Copybara error out when working with Git LFS enabled repos. Simplest workaround seems to be just deleting the file.

I couldn't find the file on my system, but git lfs uninstall did the job instead.

[adamhowardprice]

@mikelalcon Is Copybara ever going to support Git-LFS?

[mikelalcon]

It is not in our roadmap, and we don't have the bandwidth to implement/support it :( . But we would welcome contributions :)

[kjellander-unity]

In our setup we've worked around it by basically doing cloning the source repos and running:

git lfs fetch --all
git lfs push --all <destination repo URL>

before doing so, it's a good idea to align .gitattributes in the source and destination repo, plus renormalize files in the source repo (git add --renormalize). We also, to be sure, run a similar renormalize step in the destination repo after each sync.

[vanurag]

Instead of uninstalling git-lfs, one could also use the GIT_LFS_SKIP_SMUDGE environment variable.

GIT_LFS_SKIP_SMUDGE=1 copybara copy.bara.sky --force

[Matthiasfranck]

This issue seems not be solved for repos other than github repositories?

[rpavlik]

Hmm, that does look right - the enable_lfs is not exposed except for GitHubOrigin and GitHubPR.