Concatenate prefixes with bodies

[jvcoutinho]

Currently, prefixes store all of the special tokens (white spaces and comments) that occur prior to a token. On the other hand, the bodies store all of the tokens of a node. This is in some way useful to analyse indentation and spacing, but it has some weird colateral effects:

Suppose base has only one attribute int a. Left pushes a 4 spaces and right pushes a 8 spaces.
If the merge ignores whitespaces, right is chosen as it has more characters in the prefix than left does. Otherwise, a conflict is reported. But it is reported in this way:

<<<<<<< MINE
=======
    
>>>>>>> YOURS
int a;

The conflicting contents are only whitespaces and a lost its indentation. It should be reported like this:

<<<<<<< MINE
    int a;
=======
         int a;
>>>>>>> YOURS

Now, suppose the same scenario but instead of an attribute we have a method

void m() {

}

Left pushes all of m's lines 4 spaces and right pushes them 8 spaces. Considering whitespaces ignored, we have:

        void m() {

}

That means right's prefix was chosen (that's why the first line is indented 8 spaces), but the rest was decided by textual merge (that's why the third line is not indented at all).

So, instead of manipulating prefix and body to solve this specific scenario, I propose concatenating them at instancing level. That means the body will store all of the special tokens that occur prior to the first token + all of the node's content. Textual merge will take care of handling all of the spacing.
This solves these scenarios, makes whitespaces ignoration more responsive and eliminates the necessity of gambiarras. In the future, it can also make the output of a diff tool more intuitive.

[guilhermejccavalcanti]

So, you want to move all the "\t\r\n etc." that appears on FSTTerminal.getPrefix() to (concatenate with) FSTTerminal.getBody() ?

[jvcoutinho]

@guilhermejccavalcanti Exactly.

[pauloborba]

@jvcoutinho try a few examples with comments. seems to work for whitespace. not so sure about the implications on how comments are handled.

[pauloborba]

@jvcoutinho what about one developer changes the last line of a comment, and the other changes the first line of a method declaration?