LocalVariable() doesn't work with Go's calling convention

[dgryski]

The standard C prologue

pushl %ebp
movl %esp, %ebp
subl %esp, <frame size>

doesn't work with Go assembly functions. After adjusting the stack pointer, references to arguments are still relative to the FP pseudoregister (%esp in our case), but since %esp has been decremented, these offsets are now wrong.

Looking at objdump, it appears the Go prologue/epilogue for a small sub (including the stack growth check) is:

        bench_test.go:8 0x46dea0        64488b0c25f8ffffff      FS MOVQ FS:0xfffffff8, CX
        bench_test.go:8 0x46dea9        483b6110                CMPQ 0x10(CX), SP
        bench_test.go:8 0x46dead        762f                    JBE 0x46dede
        bench_test.go:8 0x46deaf        4883ec18                SUBQ $0x18, SP
        bench_test.go:8 0x46deb3        48896c2410              MOVQ BP, 0x10(SP)
        bench_test.go:8 0x46deb8        488d6c2410              LEAQ 0x10(SP), BP
        bench_test.go:8 0x46debd        488b442420              MOVQ 0x20(SP), AX
        bench_test.go:8 0x46dec2        48890424                MOVQ AX, 0(SP)
        bench_test.go:8 0x46dec6        48c744240808000000      MOVQ $0x8, 0x8(SP)
        bench_test.go:8 0x46decf        e8ac010000              CALL github.com/dgryski/go-highway.benchmarkHash(SB)
        bench_test.go:8 0x46ded4        488b6c2410              MOVQ 0x10(SP), BP
        bench_test.go:8 0x46ded9        4883c418                ADDQ $0x18, SP
        bench_test.go:8 0x46dedd        c3                      RET
        bench_test.go:8 0x46dede        e8fd6afeff              CALL runtime.morestack_noctxt(SB)
        bench_test.go:8 0x46dee3        ebbb                    JMP github.com/dgryski/go-highway.BenchmarkHighway8(SB)

What there is documented is at https://golang.org/doc/asm

This will probably be easier once golang/go#16922 is resolved.

I'm not sure the status of the "red zone" w/r/t Go on AMD64. The limitation for syso files is to keep the stack usage below ~100 bytes, so that seems like a reasonable warning level to not need the stackgrowth check.

[Maratyszcza]

I will think about it, but it seems there's no way to support local variables while generating Golang assembly listing. There are two problems:

1. PeachPy expects LocalVariable objects to have abide alignment restrictions, and in Golang assembly there's no way to specify particular alignment for the stack frame.

2. Local variables in Golang assembly must be addressed through special FP register. Since FP is a Golang assembly abstraction, and not a real machine register, it can't be encoded in the instructions not supported by Golang assembly (PeachPy emits such instructions via BYTE directive, but since FP is not a real machine register, PeachPy can't encode such instruction).

[dgryski]

I'll play with this and see what I can come up with.

[dgryski]

Here are some notes:

1. Local variables should use the negative offsets from the SP pseudo-register: [−framesize, 0): x-8(SP), y-4(SP), and so on. (0 is not a valid offset).

2. PeachPy currently does the following:

 if stack_size == 0:
                text_arguments.append("$0")
            else:
                text_arguments.append("$0-%d" % stack_size)

The 0 indicates the size of the local variables. If that value is non-zero, the Go assembler will insert a function prologue/epilogue that does the appropriate BP/SP adjustments. It also will also correct adjust the offsets relative to FP (for function arguments) so the final assembled instructions are still correct.

For a local variable size of 24 bytes, the following prologue is emitted (note that an extra adjustment of 8 bytes is added to save BP). (This is the disassembled dump, so here SP refers to the real SP register. Also, on x86, positive offsets from SP refer to the real SP register, not the pseudo-register.

SUBQ $0x20, SP                  
MOVQ BP, 0x18(SP)               
LEAQ 0x18(SP), BP               

and epilogue:

MOVQ 0x18(SP), BP               
ADDQ $0x20, SP                  

So, if seems that if we can adjust the 0 in the TEXT line and track the offsets of the local variables and output them relative to the SP pseudo-register, everything else "should just work" (at least with reference to the assembly source code output. I realize this won't work for generated object files directly.)

[Maratyszcza]

@dgryski Thank you for the thorough analysis. However, there still a problem:

In PeachPy LocalVariable objects have not only size, but also alignment. I don't see how we could guarantee alignment within Golang assembly syntax. Note: in other ABIs alignment is enforced by explicitly aligning RSP with the ANDQ instruction. However, I don't think it is safe to do in Golang:

• It may break references to arguments generated by 6a.
• It may break garbage collector if it finds a different stack frame structure than generated by 6a.

[dgryski]

Good points. I'll see if I can get more info from the Go team.

[dgryski]

Interesting macros from the Go runtime:

// GO_ARGS indicates that the Go prototype for this assembly function
// defines the pointer map for the function's arguments.
// GO_ARGS should be the first instruction in a function that uses it.
// It can be omitted if there are no arguments at all.
// GO_ARGS is inserted implicitly by the linker for any function
// that also has a Go prototype and therefore is usually not necessary
// to write explicitly.
#define GO_ARGS FUNCDATA $FUNCDATA_ArgsPointerMaps, go_args_stackmap(SB)

// GO_RESULTS_INITIALIZED indicates that the assembly function
// has initialized the stack space for its results and that those results
// should be considered live for the remainder of the function.
#define GO_RESULTS_INITIALIZED  PCDATA $PCDATA_StackMapIndex, $1

// NO_LOCAL_POINTERS indicates that the assembly function stores
// no pointers to heap objects in its local stack variables.
#define NO_LOCAL_POINTERS       FUNCDATA $FUNCDATA_LocalsPointerMaps, runtime·no_pointers_stackmap(SB)