Command: git test

Description

Available since v0.6.0.

git test is used to run a user-provided command on a given set of commits.

Here’s a demo of formatting the commits in a stack (skip to 0:35 to see just the demonstration of git test fix):

git-test-fix.mov

Usage

In its most basic form, run git test run --exec '<command>'. This will execute <command> on each commit in your current commit stack and display a summary of the results.

You can run on a different set of commits by specifying a different revset. For example: git test run -x '<command>' draft().

Invoking

Command aliases

You can pass --command <command> to run a pre-defined command alias instead of specifying the full command with --exec. Define a new alias with git config branchless.test.alias.<name> <command>, then run it with git test run --command <name>.

Default command

If neither --command nor --exec is provided, the default command alias is used.

Execution

Running outside the current working copy

There are two strategies for running the provided command:

• --strategy working-copy: Run in the working copy.
  • Only one instance of the command can run concurrently.
  • Useful if the working copy has build artifacts which should be preserved for each run.
  • Doesn't create any additional repository checkouts on disk.
• --strategy worktree: Run in a worktree managed by git-branchless.
  • Multiple instances of the command can run concurrently.
  • Untracked files such as build artifacts won't be shared between the working copy or different worktrees. However, build artifacts may be preserved in the same worktree for the next run.
  • Creates up to one extra repository checkout on disk per job.

You can set the default strategy with git config branchless.test.strategy <strategy>.

Running in parallel

You can pass --jobs N to run the command in parallel on different commits. A value of 0 indicates to select the number of jobs to run based on the number of CPUs on the machine. A value greater than one implies the worktree strategy.

You can set the default number of jobs with git config branchless.test.jobs <N>.

Caching

Each command invocation is cached using the command and the tree ID of the commit it ran on. If the command would run on a commit with the same tree again, the cached exit code and test output are immediately returned. This means that test commands won't re-run if only the commit message or ancestry have changed, but not the contents of the commit.

To delete cached results for a given commit, run git test clean <revset>.

Available since v0.7.0: To run git test run without reading or writing the cache, pass the --no-cache option.

Showing output

To show the results of previous test runs, run git test show -c/-x <command> <revset>. By default, only the passed/failed status is shown. To show the test output, pass -v/--verbose or -vv/--verbose --verbose for more detail.

You can also pass -v/-vv directly to git test run to show the output immediately after running.

Environment variables

When executing the provided command, all environment variables are inherited from the calling process, and the following environment variables are additionally set:

• BRANCHLESS_TEST_COMMIT: The hash of the commit being tested. Currently, this is always equivalent to HEAD.
• BRANCHLESS_TEST_COMMAND: The command being executed.

Fixing formatting and linting issues

Available since v0.7.0.

git test can be used not just to detect issues in commits, but also fix them. The git test fix subcommand can run command such as a formatter or linter and apply its resulting changes to each provided commit. For example: git test fix --exec 'cargo fmt --all'.

Preventing merge conflicts

It can be difficult to apply formatter or linter changes by hand (with e.g. git rebase --exec) because it necessarily rebases the descendants of fixed commits. This oftentimes causes merge conflicts that are about to be resolved by formatting or linting the next commit. (As a workaround, some people try fixing the commits in reverse topological order to minimize the chances of merge conflicts.)

git test fix never produces merge conflicts because it fixes each commit in isolation by replacing its tree OID directly, and leaves descendant commits otherwise unchanged. (You can do the same thing by hand by calling git amend with the --reparent option.)

Fixing in parallel

You can run git test fix with the --jobs option to run the fix command on each commit in the stack in parallel. This is best for I/O-bound commands such as formatters or for single-threaded commands.

Searching

Available since v0.7.0.

It often happens that you want to find the first commit which starts failing tests. You can run git test run with the -S <option>/--search <strategy> to search commits in a certain order. Then, when the first failing commit is found, the search stops and prints summary output.

The valid values for strategy are:

• linear: start with the oldest commits and proceed to old newest.
  • This can be useful if the cost of switching to a distant commit is high, or if distant switching tends to invalidate many build artifacts.
• reverse: start with the newest commits and proceed to the oldest.
  • This can be useful when you have fresh build artifacts that you want to preserve during incremental rebuilds, and you think that the failing commit is nearby.
• binary: start somewhere in the middle and apply binary search.
  • This assumes monotonicity in order to calculate the correct result: a passing test cannot be a descendant of a failing test.
  • This will generally find the failing commit in the shortest number of tests, assuming that each commit is equally likely to be the first failing commit.
  • As a shorthand, instead of writing --search binary, you can pass the -b/--bisect option.

Searching interactively

If you prefer to perform the search manually by examining each commit, rather than running a specific command, you can pass -i/--interactive instead of -x/--exec or -c/--command. For each commit, git-branchless will launch your shell and print instructions on how to mark the commit as passing or failing (or skip the commit or abort the test process).

Caching

Just as with git test run without the --search option, test results and exit codes for each command invocation are cached. This means that you can interrupt the search and resume it later without losing any work.

Note that even interactive results are cached. These will be marked with an interactive tag in summary output.

If the result of testing a certain commit was wrong, you can clear the cache for it with git test clean.

Searching in parallel

You can run the search with the --jobs option to speculatively test multiple commits at once as part of the search. Note that you may not gain performance if the command is CPU-bound rather than IO-bound.