Fix strip issue with 'nekotools boot' created executables on Linux ELF... #86
Conversation
executables made via 'nekotools boot <bytecode>' on Linux systems (really systems using ELF ABI) will be irreprably damaged if stripped by the 'strip' toolchain program. The issue is described in detail here... HaxeFoundation#75 The solution here is to modify the Neko VM and the nekotools programs to be more ELF-aware. This new code is activated by compiling with ABI_ELF (now the default for Linux builds). The Neko VM will now attempt to find a '.nekobytecode' section in the executable by rifling through the ELF section headers. If such a section is found, it is inspected, looking for the customary bytecode signature at the end of the section. If the section is not found, the code will fall back to the old algorithm and search at the end of the file. (This algorithm was necessary since tools like strip may actually place other sections after the bytecode at the end of the file, even if the bytecode section was initially at the end of the file.) Additionally, the 'nekotools' executable now understands how to inject bytecode into the '.nekobytecode' section, resizing that section as appropriate. This allows this data to survive strip attempts. Below are some measured performance and executable size impacts of these changes as measured on my Fedora Core 17, x86_64 box that uses Xeon processors... perf : 'neko -version' execution slowed by 32% or ~4.4ms/execution on my 2.53Ghz Xeon perf : 'nekoc -help' execution slowed by 4% or ~13.8ms/execution on my 2.53Ghz Xeon size : neko vm grew 2,880 bytes or ~23% larger; still under 16KB size : nekotools boot executables can be stripped now, making them 2%-8% smaller on x86_64
|
Oops. Right, Windows. Looks like I didn't add the new files to the Windows project file. Not sure how well I can fix that without really being able to test it, but I'll have a look tmw. In the meantime, feel free to look at the code here. ;) |
project files. I am unable to test this, but it seems right. I'll let AppVeyor check it for me.
|
Ok, this is getting ugly. I'm using checkins to run builds. Ick. I think I've placed the elf_update.c code in the wrong place (libneko when it should be in std.ndll). Somehow that worked fine on Linux, but not on Windows. I'm trying to sort it out now. |
interpreter into libneko. Move bytecode injection logic into std.ndll where it belongs so that it can be visible to nekoboot.neko. Also, adjust symbol exposure of elf helper functions in libneko so that they are properly accessible on Windows (hopefully).
|
Yeah. Well that was more fun than expected. ;) |
|
@kulick thanks for the contribution ! could you confirm it works with both 32 and 64 bits ELF ? |
|
Designed for both. I tested 64-bit. I'll see if I can validate 32-bit real quick...brb. |
|
Ok, just tested on a 32-bit VM image and all looks good to me. Any specific concern or just checking? |
|
Just checking. Merging now :) |
Fix strip issue with 'nekotools boot' created executables on Linux ELF...
|
@kulick Any idea? |
|
Andy, Sorry about this. I can't really test Mac OS X or Windows build workflows. The problem here is that we shouldn't be building on Mac with -DABI_ELF. Hopefully the fix is simply a matter of just refactoring the addition of that compiler argument so that it only occurs on Linux. Maybe something like this... ...? Alternatively, we could require 'make os=linux' on Linux or detect Linux with something like "uname | grep '^Linux'". Lots of solutions. If you like my 'filter-out' solution, feel free to pull the change from my neko repo. Or let me know if you'd like a pull request. Thanks and, again, sorry for the hassles... |
|
Thanks for the info! I have no idea about the ELF stuffs at all. I've just refactored the makefile as you've suggested (#87). |
We may want to discuss what, if any, other precautions we'd like to take to better support other ABIs and other platforms. Bryan and I did look at Mac OS X and it seems that attempts to strip 'nekoc' there result in the strip tool just giving up (without damaging the executable) since it doesn't understand the extra bytes at the end of the file...