A pub git references that specifies a specific ref does an entire git clone

[dnfield]

e.g. the following:

dependencies:
  vm_service:
    git:
      url: https://dart.googlesource.com/sdk
      ref: refs/changes/80/156980/3
      path: pkg/vm_service

ends up doing:

[  +34 ms] IO  : Spawning "git --version" in /Users/dnfield/src/flutter/plugins/packages/integration_test/.
[  +17 ms] IO  : Finished git. Exit code 0.
[        ]     | stdout:
[        ]     | | git version 2.18.0
[        ]     | Nothing output on stderr.
[        ] FINE: Determined git command git.
[   +1 ms] IO  : Spawning "git rev-parse --is-inside-git-dir" in /Users/dnfield/.pub-cache/git/cache/sdk-40ed25bf474d78ecf3880eb744f1165252eb170d
[  +34 ms] IO  : Finished git. Exit code 128.
[        ]     | Nothing output on stdout.
[        ]     | stderr:
[        ]     | | fatal: not a git repository (or any of the parent directories): .git
[   +4 ms] IO  : Deleting directory /Users/dnfield/.pub-cache/git/cache/sdk-40ed25bf474d78ecf3880eb744f1165252eb170d.
[   +4 ms] IO  : Spawning "git clone --mirror https://dart.googlesource.com/sdk /Users/dnfield/.pub-cache/git/cache/sdk-40ed25bf474d78ecf3880eb744f1165252eb170d" in /Users/dnfield/src/flutter/plugins/packages/integration_test/.

Which goes on to fetch the entire history of a very large repository when all I'll really need is a specific ref. As of this writing, it results in cloning multiple gigabytes of data into my .pub-cache which I will have to manually clean up, and takes over a minute to complete.

This would be multitudes more efficient if it did something like:

final sdkDir = createSdkCacheDir();
runProcess('git init .', workingDirectory: sdkDir);
runProcess('git remote add upstream https://dart.gogolesource.com/sdk', workingDirectory: sdkDir);
runProcess('git fetch upstream $ref', workingDirectory: sdkDir);
runProcess('git checkout FETCH_HEAD', workingDirectory: sdkDir); // or git checkout $ref is fine too

It should be faster.

[sigurdm]

Checking out a single ref would also get the entire history leading to that ref (as I understand it).

I think what we should be using is a shallow clone: https://linuxhint.com/git-shallow-clone-and-clone-depth/ .

I wonder if it has any downsides (will it be more expensive to change the ref?) @jonasfj WDYT?

[jonasfj]

I think this is nice to have.. we already do some caching, so the second time you use the same repo, it's not cloned again.

[mralbertchen]

I think shallow clone is the way to go for this. At my company we are currently combining unity with flutter through https://pub.dev/packages/flutter_unity_widget and we host the unity widget dependency on a git repo. It gets big really quickly and we have to keep deleting tags. Can someone point out where this logic is currently done? Happy to take a stab at it and submit a PR.

[2shrestha22]

Any update on this?

[sigurdm]

closing this in favor of #3556