RFC: Deferring merge conflicts

[arxanas]

It's clear that the workflows around resolving merge conflicts are clumsy in Git. What's the best way to fix this?

I think there are main fundamental issues that impede these workflows:

• It's not clear ahead of time whether an option will result in a merge conflict. If it does, then the user may wish to defer the operation until they're ready to resolve it.
  • I think the ideal workflow would be in a GUI tool where you simply drag a commit stack from one place to another, and the tool speculatively carries out the rebase and reports whether it would cause conflicts. This is also nice for performance, since if the user does decide to carry out the rebase, then it's already been performed in the background, and we only need to update a few references.
  • Carrying out the above procedure even in a TUI tool seems difficult.
• Once a merge conflict has started, it's hard to pause it and come back to it later.
  • Fixing this issue is one of the main goals of Jujutsu (cc @martinvonz).
  • There's some design at Support undoing staged changes #46 (comment), but it needs a lot of work to realize in a performant way.
• If a merge conflict was resolved wrongly, it's hard to re-do it. This is ameliorated somewhat by git undo.

For git-branchless to integrate well with Git, I think there are a few options.

Don't merge

Don't resolve merge conflicts by default when carrying out operations. The user should have to opt-in via a --merge flag. Personally, I don't like how rebase operations sometimes rudely introduce merge conflict state that I then have to abort.

In my opinion, this solution is probably worth doing regardless of whether we pick one of the below solutions as well.

Merge as much as possible

When updating local commit stacks to be based on the latest main branch, move the stack as far as possible along the main branch. This is an option available in https://github.com/Timmmm/autorebase (cc @Timmmm), and also mentioned in https://github.com/epage/git-stack/issues/25 (cc @epage).

In the above solution, we try to rebase the entire commit stack as much as possible. It would also make sense to rebase as much as of the commit stack as possible. So you would start with a graph like this:

O def001 main branch commit 1
|\
| o abc001 good commit
| |
| o abc002 bad commit
| |
| o abc003 descendant commit
|
O def002 main branch commit 2

And then, rather than forcing merge conflict resolution to happen immediately, we would end up with a graph like this:

O def001 main branch commit 1
|\
| x abc001 (rewritten as abc004) good commit
| |
| o abc002 bad commit
| |
| o abc003 descendant commit
|
O def002 main branch commit 2
|
o abc004 good commit

It would definitely be good to know how much of the move operation succeeded, but I don't like the idea that I might accidentally fail to move an entire stack of commits when I intended to.

Mark conflicting commits in the smartlog

Like the above solution, but explicitly mark details of the conflict in the smartlog:

O def001 main branch commit 1
|\
| x abc001 (rewritten as abc004) good commit
| |
| ~ abc002 (1 conflict) bad commit
| |
| ~ abc003 (parent has conflicts) descendant commit
|
O def002 main branch commit 2
|
o abc004 good commit

If you check out to abc002, then the post-checkout hook would do one of the following:

• Warn you that the commit needs to be restacked, and to run git restack, so that you can initiate merge conflict resolution.
  • This gives maximum control to the user, and allows them to precisely decide when they want to start resolving the merge conflict.
• Actually initiate the merge conflict resolution by starting the rebase process.
  • This might be problematic for scripted checkouts? If you perform an on-disk rebase, you probably don't want to enter a nested level of merge conflict resolution (or at least, not without significantly more support from the version control system).
  • Besides that, you have to be quite aware that performing a checkout may have side-effects.

But then I'm not sure how we would dissociate the commit with its merge conflict. What if we want to move it somewhere else in the commit graph, or simply silence the warning in the smartlog?

Simulate commits with conflicts

If a merge conflict would occur as a result of an operation, pretend that the operation succeeded, but that the rebased commit is marked with a conflict. We would be simulating the behavior of Jujutsu as much as possible within the Git design. So you would start with a graph like this:

O def001 main branch commit 1
|\
| o abc001 good commit
| |
| o abc002 bad commit
| |
| o abc003 descendant commit
|
O def002 main branch commit 2

Then after trying to rebase the stack and encountering a conflict in abc002, the smartlog would look like this:

O def001 main branch commit 1
|
O def002 main branch commit 2
|
o abc004 good commit
|
~ abc002 (1 conflict) bad commit
|
~ abc003 (parent has conflicts) descendant commit

Note that the commit hash abc002 remains unchanged. This rendering above is only for the smartlog.

This is paradigm-shifting, and introduces a compelling alternative to the current solution. The problem I have with the design is that I often want to use the pre-conflicted version (for example, to test to see if a certain test fails before versus after rebasing on top of the main branch). It's not clear how to do this under the Jujutsu model (@martinvonz do you have a solution for this workflow?).

And this is a leaky abstraction when implemented on top of Git. If usability is a goal for git-branchless, is it worth creating commands with a brand new set of difficult-to-inspect state? We might be able to preserve the abstraction within the universe of git-branchless commands, but what if someone runs git checkout abc002~? Would we conceptually like to put them at abc001 or abc004, and does that fall in line with user expectations?

One solution to fixing the abstraction a little is to actually carry out the rebase by keeping conflict markers in the conflicting files and committing that. Then, the contents of commit are sensible if you try to operate on it; we just have to be sure to precisely track that the commit is associated with a previous patch which needs to be merged properly. (It also gets complicated for descendant commits, which may or may not conflict with the first conflicting commit, depending on its actual resolution. I think Jujutsu has a principled solution for this, but I don't know what it is.)

It might also be possible to use some kind of synthetic commit identifier to make sure that the user can't accidentally mix up conflicted commits with real commits. If the commit identifier is instead ghijkl, then surely the user won't use it in Git commands inappropriately. (And let's just hope that ghijkl isn't the name of a branch...) Or we can use an arbitrary commit hash, and automatically tag it with a useful branch, like conflict-1, to make it easier for the user to address.

[martinvonz]

Jujutsu saves conflicts as special files (if you check out the commit with Git, you'll see a foo.jj-conflict file with some weird JSON contents). These files do not have conflict markers, they just have information about how to reproduce the conflict state. That information is a list of states to remove and a list of states to add. In the common case of a 3-way merge of file contents B and C, using contents A in the merge base, the state (JSON file in Git) will say something like {removes: [A], adds: [B, C]}. Various commands "materialize" the conflict using conflict markers. That includes jj checkout and jj diff.

I had documentation of the format in the README a long time ago (before I started targeting users instead of developers in the documentation). You can read it here. I need to recover that information and put it in a more accessible place. This discussion between and me and @quark-zju may also help explain it.

[arxanas]

Attaching metadata to Git commits seems to be very hard, since Git will garbage collect objects which aren't referenced. It seems like the only type of valid edge from a commit object is a parent edge to another commit object, so any metadata would have to be stored either in a special parent commit object, or an object pointed to by a ref, or out of band somewhere. If I store it out of band, then it risks being garbage collected in the future. (git undo explicitly handles this by saying that certain commits/events aren't available.)

[black7375]

There seems to be a way to add metadata.
https://tylercipriani.com/blog/2022/11/19/git-notes-gits-coolest-most-unloved-feature/

[arxanas]

I believe Jujutsu used to store conflict information in Git notes, but it was very slow, I believe. In any event, I decided not to attempt storing merge conflicts in git-branchless, and leave that for Jujutsu, whose implementation is bound to be better anyways.

[martinvonz]

Yes, you need to create refs pointing to the data the conflicts point to. You don't need to have references to the conflicts themselves, because they are pointed to directly by the tree objects they're in. You just need references to the objects involved in the conflict. I need to fix that in jj (martinvonz/jj#12). I'm careful not to run git's gc until then :)

[arxanas]

Do you have the tree objects point to the conflicts by storing tag objects in the tree, and if so, does git fsck allow that?

[martinvonz]

The conflicts are actually just blobs (files) containing JSON data. Here's an example of a conflict in my repo (only in my local repo, because it makes no sense to push this to GitHub):

diff --git a/lib/src/revset.rs b/lib/src/revset.rs
deleted file mode 100644
index c9c1f644..00000000
--- a/lib/src/revset.rs
+++ /dev/null
@@ -1,167 +0,0 @@
<skipped content>
diff --git a/lib/src/revset.rs.jjconflict b/lib/src/revset.rs.jjconflict
new file mode 100644
index 00000000..e070f77b
--- /dev/null
+++ b/lib/src/revset.rs.jjconflict
@@ -0,0 +1 @@
+{"adds":[{"value":{"file":{"executable":false,"id":"c9c1f644c929663e807399b1efb5619b4bd1c7a3"}}},{"value":{"file":{"executable":false,"id":"fd903b5aae2132d05c2299f21a249475a43a4fc3"}}}],"removes":[{"value":{"file":{"executable":false,"id":"4e731cd3141df44526b186fa0909184a041b2e16"}}}]}
\ No newline at end of file

The objects pointed to by the JSON data there are of course not understood by Git, so those are what I need to add refs to.

[martinvonz]

Sorry, that's of course the diff for a commit introducing a conflict. I figured it would helpful to see how jj replaces the file by a weird <filename>.jjconflict file to indicate that it's actually a conflict. I don't remember if I tried abusing the file mode bits for that (I think I did and libgit2 didn't like it).

[epage]

git-stack will rebase as many sub-trees as possible, leaving the rest to the user to have to manually rebase. If the user didn't like this, they can use git branch-stash to revert the entire tree back to a prior state (we push to the branch-stash on every mutation)

This is visually presented to the user by showing a commit as the shared base

clap on  user-error is 📦 v3.0.0-beta.5 took 9s 
❯ git stack
afd9efc Merge #2887
├── o
|   └── make (ready) chore(ci): Re-use CI logic through make
└── o
    ├── user-error (no remote), master Merge #2909
    |   ├── o
    |   |   └── derive_error (ready) fix(derive): Error, don't Panic
    |   ├── o
    |   |   └── pos WIP
    |   ├── o
    |   |   ├── 2b01b9f fix: Ease clap2->3 transition with renamed Settings
    |   |   ├── dc3af35 fix: Allow using deprecated color AppSettings
    |   |   └── 2to3-settings (10 ahead, 3 behind) fix: Ease transition for now-default settings
    |   ├── o
    |   |   ├── 5dc8f58 docs(yaml): Consistently place layout descriptio
    |   |   └── docs (9 ahead, 2 behind) docs: Clarify corner caseses with default values
    |   └── o
    |       └── derive_vec WIP
    └── possible (pushed) fix!: Rename ArgValue to PossibleValue

So far I've considered this "good enough" because its already streamlined so much of my workflow I don't mind the occasional hand done rebases.