Fix tests/codegen/mangling.d on x86

[the-horo]

The test codegen/mangling.d has been failing for me on x86 with a segmentation fault because of stack corruption due to the function:

ldc/tests/codegen/inputs/mangling_definitions.d

Line 49 in 93afbfa

int naked_dFunc(double a) { asm { naked; ret; } }

not respecting the stdcall convention which is set as the default for functions with D linkage on x86.

I've tried to fix this by changing the default calling convention for naked assembly functions on x86 and adding an attribute in the mangling.d file to specify the C calling convention.

I'm not sure how this test has been passing on windows x86.

[kinke]

I very much doubt we want to change the low-level calling convention, not matching up with the frontend CC. And definitely not for working around a test failure (where the the test might need adjustments, but I don't want to verify myself right now).

[the-horo]

Yes, I was hesitant to go further with touching gen/abi but, on the other hand, having naked assembly blocks change calling convention when going from dmd to ldc2 isn't ideal either. I won't mind the solution so long as we, eventually, get the test to pass.

[kinke]

AFAICT, DMD doesn't automagically fix up anything either, or change the calling convention - it doesn't fixup the ret to a ret 8, nor does it seem to call a naked function differently than a non-naked one.

[the-horo]

Well, there seems to be plenty of segfaults in the 32bit tests so I think we can safely rule out changing the calling convention

AFAICT, DMD doesn't automagically fix up anything either, or change the calling convention - it doesn't fixup the ret to a ret 8, nor does it seem to call a naked function differently than a non-naked one.

Yes, dmd doesn't do anything special to naked functions, nor does ldc2. They both keep the default calling convention for the architecture, which is C for dmd and stdcall for ldc2. I don't have a problem with this. My only issues was that, while calling conventions can mostly be ignored, you can't do that with naked assembly functions and I didn't want to have the same code that works with dmd be completely broken with ldc2, though we are talking about naked assembly so maybe I'm making a bigger deal then it needs to be.

Either way, given the test failures I think it would be better to just change the test file to make it pass and live with the differences between ldc2 and dmd when it comes to naked assembly functions on x86.

[kinke]

They both keep the default calling convention for the architecture, which is C for dmd and stdcall for ldc2.

Nope, only for 32-bit x86, there's a D-specific ABI/CC, where LDC follows DMD (except for NOT reversing the args IIRC, but not 100% sure), which is very similar to stdcall, except that it tries to use one register: https://dlang.org/spec/abi.html#function_calling_conventions

[kinke]

See e.g. here, required for DMD too: https://github.com/dlang/phobos/blob/master/std/internal/math/biguintx86.d#L623

[the-horo]

They both keep the default calling convention for the architecture, which is C for dmd and stdcall for ldc2.

Nope, only for 32-bit x86, there's a D-specific ABI/CC, where LDC follows DMD (except for NOT reversing the args IIRC, but not 100% sure), which is very similar to stdcall, except that it tries to use one register: https://dlang.org/spec/abi.html#function_calling_conventions

Neat! Having a formal document describing the spec seems to make it easier to understand as opposed to looking at the code and working my way backwards.

[kinke]

Having a formal document describing the spec seems to make it easier to understand as opposed to looking at the code and working my way backwards.

Hah! :D - Well I do it that way most of the time too, not trusting potentially outdated docs/specs. ;)

Edit: E.g., the spec mentions that this is the CC for Windows x86. In reality, it's the one for all 32-bit x86 targets. Well, not really, Android is different, of course...

[kinke]

I'd suggest changing the param to a 32-bit integer, which is passed in a register on 32-bit x86, and so the plain ret should still work.

[the-horo]

I've looked over this again with a rested mind and I think I finally understand what you mean.

First of all, ldc is consistent with dmd when it comes to the calling convention of naked assembly functions on x86 (like you told me), on most operating system that being the one described in the spec. What threw my off was:

1. the fact that the code wasn't segfauling on dmd, which is because it restores ESP when returning from a function

2.  11bb:       83 ec 08                sub    $0x8,%esp
    11be:       6a 00                   push   $0x0
    11c0:       6a 02                   push   $0x2
    11c2:       8b 5d fc                mov    -0x4(%ebp),%ebx
    11c5:       e8 d6 ff ff ff          call   11a0 <_D1a3fooFlZv>
    11ca:       83 c4 08                add    $0x8,%esp
   

   Making me think that the add was for cleaning the stack when it is not, it is just the counterpart to the sub at the beggining.

All in all, I just wanted to thank you for being patient with me and taking the time to explain it.

[kinke]

Oh, you're very welcome, thank you for cleaning up this stuff showing up when running the tests on non-CI-tested platforms, investigating and fixing it all yourself! 👍

The fact that the code wasn't segfauling on dmd, which is because it restores ESP when returning from a function

Oh okay, I was expecting it'd segfault too. No wonder this caused a bit of confusion.