Vagrant on ARM64 (Apple Silicon and VMware Fusion compatibility)

[JobeanSmith]

Hello,

I am running Vagrant on an MacBook Pro with an Apple Silicon chip and macOS Monterey.
I would like to run it with VMware Fusion (because it's the only VM ARM compatible) but it seems that vagrant is not made for ARM architecture (as well as Vagrant VMware Desktop and Vagrant VMware Utility).

May I ask you to recompile it on arm64 to work on Apple Silicon devices ?

Thank you a lot

[fatso83]

@JobeanSmith Vagrant runs fine on Apple Silicon (ARM). You need to download the Technology Preview of VMWare Fusion, symlink/rename the dir and use a box that is built for ARM (VMWare for ARM will not run x86 images). You will find a full discussion leading up to these conclusions, along with links to gists (and Ubuntu ARM boxes) that will get you up and running in the following issue for the vmware provider: hashicorp/vagrant-vmware-desktop#22

This issue should be closed in that regard. What is missing from a UX perspective (IMHO), is metadata on boxes that says which architecture the box is for. Currently, you need to follow some kind of (inofficial) naming convention where you append -arm64 or something like that to the box names for anyone to find them, hence #12610.

[fatso83]

@JobeanSmith I might have misunderstood you here. Vagrant is currently working on Apple Silicon, but I have no idea if I am running something compiled for ARM or using x86 via Rosetta. Is this issue just about compiling a native version? I would think the possible performance hit of Rosetta would be very negligible, considering all the real work is done by other programs.

[tcurdt]

@fatso83 The homebrew download still seems to be intel only - or do you mean "vagrant is working via rosetta"?

$ brew install vagrant
Running `brew update --preinstall`...
==> Auto-updated Homebrew!
Updated 6 taps (homebrew/cask-versions, homebrew/core, homebrew/cask, homebrew/cask-fonts, homebrew/services and homebrew/cask-drivers).
==> New Formulae
mist
==> Updated Formulae
Updated 83 formulae.
==> Updated Casks
Updated 41 casks.

==> Downloading https://releases.hashicorp.com/vagrant/2.2.19/vagrant_2.2.19_x86_64.dmg

[fatso83]

@tcurdt I think that's answered by my last comment 😄

have no idea if I am running something compiled for ARM or using x86 via Rosetta.

I am pretty sure I downloaded the official version, so I am probably running x86 via Rosetta. Hence my question to Jo, as it is working (i.e. the Vagrant CLI lets me run ARM boxes on VMWare Fusion Tech Preview), just not using a "native" binary.

[JobeanSmith]

@fatso83 Thank you for your answer ! you have understood well what I meant. I had an issue finding the ARM box (reason why I thought it was because of ARM) I temporary switched to Docker because of time. Thank you for your help !

[jcsalem]

FYI... Vagrant currently runs via the Rosetta x86 translation on the Apple M1 Macs. There are ARM boxes available on Vagrant Cloud.
I had good success with Parallels on the Mac. It's been out longer than VMWare Fusion.

[Amar1729]

For reference, you can check which architecture a particular binary is by using file -

~ $ file /opt/homebrew/opt/python3/bin/python3

/opt/homebrew/opt/python3/bin/python3: Mach-O 64-bit executable arm64

~ $ file /usr/bin/python

/usr/bin/python: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
/usr/bin/python (for architecture x86_64):	Mach-O 64-bit executable x86_64
/usr/bin/python (for architecture arm64e):	Mach-O 64-bit executable arm64e

~ $ file $(which vagrant)

/usr/local/bin/vagrant: Mach-O 64-bit executable x86_64

[trinitronx]

Yes, it appears there is no native aarch64 / arm64 binary package for Vagrant... yet.

It seems to work fine for most functioning virtualization tasks that are currently possible on Apple Silicon with VMWare Fusion Tech Preview. (Of course... after following the symlink hack / workaround here)

However, it still is a blocker for other plugins such as vagrant-libvirt:

$ vagrant plugin install vagrant-libvirt
Installing the 'vagrant-libvirt' plugin. This can take a few minutes...
Building native extensions. This could take a while...
Vagrant failed to properly resolve required dependencies. These
errors can commonly be caused by misconfigured plugin installations
or transient network issues. The reported error is:

ERROR: Failed to build gem native extension.

    current directory: ~/.vagrant.d/gems/2.7.4/gems/ruby-libvirt-0.8.0/ext/libvirt
/opt/vagrant/embedded/bin/ruby -I /opt/vagrant/embedded/lib/ruby/2.7.0 -r ./siteconf20220119-6421-koqs6m.rb extconf.rb
Looking for libvirt using pkg-config
checking for virConnectOpen() in -lvirt... no
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
	--with-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
	--ruby=/opt/vagrant/embedded/bin/$(RUBY_BASE_NAME)
	--with-libvirt-include
	--without-libvirt-include
	--with-libvirt-lib
	--without-libvirt-lib
	--with-libvirt-config
	--without-libvirt-config
	--with-pkg-config
	--without-pkg-config
	--with-virt-dir
	--without-virt-dir
	--with-virt-include
	--without-virt-include=${virt-dir}/include
	--with-virt-lib
	--without-virt-lib=${virt-dir}/lib
	--with-virtlib
	--without-virtlib
extconf.rb:44:in `<main>': No working libvirt installation found (RuntimeError)

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  ~/.vagrant.d/gems/2.7.4/extensions/x86_64-darwin-19/2.7.0/ruby-libvirt-0.8.0/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in ~/.vagrant.d/gems/2.7.4/gems/ruby-libvirt-0.8.0 for inspection.
Results logged to ~/.vagrant.d/gems/2.7.4/extensions/x86_64-darwin-19/2.7.0/ruby-libvirt-0.8.0/gem_make.out

Checking in the mkmf.log, it's showing that the x86_64 vagrant cannot link against aarch64 version of libvirt (yep, Homebrew does support this now, and it works without manual patches!. If only we could get fully working hvf in mainline...)

The real error is: ld: symbol(s) not found for architecture x86_64, which ld warns us about trying to link against aarch64: ld: warning: ignoring file /opt/homebrew/Cellar/libvirt/7.10.0_1/lib/libvirt.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64

So, it's still worth getting a native build of Vagrant for linking against plugins that need native gem extensions. There's work being done to get full hvf support for qemu + libvirt on the Apple M1, so the requirements for supporting this are moving along steadily as developers are upgrading to newer MacBooks.

Full mkmf.log:

"pkg-config --exists libvirt"
| pkg-config --libs libvirt
=> "-L/opt/homebrew/Cellar/libvirt/7.10.0_1/lib -lvirt\n"
"gcc -o conftest -I/opt/vagrant/embedded/include/ruby-2.7.0/x86_64-darwin19 -I/opt/vagrant/embedded/include/ruby-2.7.0/ruby/backward -I/opt/vagrant/embedded/include/ruby-2.7.0 -I. -I/opt/vagrant/embedded/include -mmacosx-version-min=10.9 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/opt/vagrant/embedded/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT   -I/opt/vagrant/embedded/include -mmacosx-version-min=10.9 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I./include -O3 -std=c99 -fno-common -pipe conftest.c  -L. -L/opt/vagrant/embedded/lib -L/opt/vagrant/embedded/lib -L.  -mmacosx-version-min=10.9 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -fstack-protector-strong -L/opt/vagrant/embedded/lib     -lruby.2.7   "
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

"gcc -o conftest -I/opt/vagrant/embedded/include/ruby-2.7.0/x86_64-darwin19 -I/opt/vagrant/embedded/include/ruby-2.7.0/ruby/backward -I/opt/vagrant/embedded/include/ruby-2.7.0 -I. -I/opt/vagrant/embedded/include -mmacosx-version-min=10.9 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/opt/vagrant/embedded/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT   -I/opt/vagrant/embedded/include -mmacosx-version-min=10.9 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I./include -O3 -std=c99 -fno-common -pipe conftest.c  -L. -L/opt/vagrant/embedded/lib -L/opt/vagrant/embedded/lib -L.  -mmacosx-version-min=10.9 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -fstack-protector-strong -L/opt/vagrant/embedded/lib     -lruby.2.7 -L/opt/homebrew/Cellar/libvirt/7.10.0_1/lib -lvirt  "
ld: warning: ignoring file /opt/homebrew/Cellar/libvirt/7.10.0_1/lib/libvirt.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

| pkg-config --cflags-only-I libvirt
=> "-I/opt/homebrew/Cellar/libvirt/7.10.0_1/include\n"
| pkg-config --cflags-only-other libvirt
=> "\n"
| pkg-config --libs-only-l libvirt
=> "-lvirt\n"
package configuration for libvirt
incflags: -I/opt/homebrew/Cellar/libvirt/7.10.0_1/include
cflags: 
ldflags: -L/opt/homebrew/Cellar/libvirt/7.10.0_1/lib
libs: -lvirt

have_library: checking for virConnectOpen() in -lvirt... -------------------- no

"gcc -o conftest -I/opt/vagrant/embedded/include/ruby-2.7.0/x86_64-darwin19 -I/opt/vagrant/embedded/include/ruby-2.7.0/ruby/backward -I/opt/vagrant/embedded/include/ruby-2.7.0 -I. -I/opt/homebrew/Cellar/libvirt/7.10.0_1/include -I/opt/vagrant/embedded/include -mmacosx-version-min=10.9 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/opt/vagrant/embedded/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT   -I/opt/vagrant/embedded/include -mmacosx-version-min=10.9 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I./include -O3 -std=c99 -fno-common -pipe  conftest.c  -L. -L/opt/vagrant/embedded/lib -L/opt/vagrant/embedded/lib -L.  -mmacosx-version-min=10.9 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -fstack-protector-strong -L/opt/vagrant/embedded/lib -L/opt/homebrew/Cellar/libvirt/7.10.0_1/lib     -lvirt -lruby.2.7 -lvirt  -lvirt  "
ld: warning: ignoring file /opt/homebrew/Cellar/libvirt/7.10.0_1/lib/libvirt.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
Undefined symbols for architecture x86_64:
  "_virConnectOpen", referenced from:
      _t in conftest-916dbf.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
checked program was:
/* begin */
 1: #include "ruby.h"
 2: 
 3: #include <libvirt/libvirt.h>
 4: 
 5: /*top*/
 6: extern int t(void);
 7: int main(int argc, char **argv)
 8: {
 9:   if (argc > 1000000) {
10:     int (* volatile tp)(void)=(int (*)(void))&t;
11:     printf("%d", (*tp)());
12:   }
13: 
14:   return !!argv[argc];
15: }
16: int t(void) { void ((*volatile p)()); p = (void ((*)()))virConnectOpen; return !p; }
/* end */

"gcc -o conftest -I/opt/vagrant/embedded/include/ruby-2.7.0/x86_64-darwin19 -I/opt/vagrant/embedded/include/ruby-2.7.0/ruby/backward -I/opt/vagrant/embedded/include/ruby-2.7.0 -I. -I/opt/homebrew/Cellar/libvirt/7.10.0_1/include -I/opt/vagrant/embedded/include -mmacosx-version-min=10.9 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/opt/vagrant/embedded/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT   -I/opt/vagrant/embedded/include -mmacosx-version-min=10.9 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I./include -O3 -std=c99 -fno-common -pipe  conftest.c  -L. -L/opt/vagrant/embedded/lib -L/opt/vagrant/embedded/lib -L.  -mmacosx-version-min=10.9 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -fstack-protector-strong -L/opt/vagrant/embedded/lib -L/opt/homebrew/Cellar/libvirt/7.10.0_1/lib     -lvirt -lruby.2.7 -lvirt  -lvirt  "
conftest.c:16:13: error: conflicting types for 'virConnectOpen'
extern void virConnectOpen();
            ^
/opt/homebrew/Cellar/libvirt/7.10.0_1/include/libvirt/libvirt-host.h:579:25: note: previous declaration is here
virConnectPtr           virConnectOpen          (const char *name);
                        ^
conftest.c:17:30: error: too few arguments to function call, single argument 'name' was not specified
int t(void) { virConnectOpen(); return 0; }
              ~~~~~~~~~~~~~~ ^
/opt/homebrew/Cellar/libvirt/7.10.0_1/include/libvirt/libvirt-host.h:579:25: note: 'virConnectOpen' declared here
virConnectPtr           virConnectOpen          (const char *name);
                        ^
2 errors generated.
checked program was:
/* begin */
 1: #include "ruby.h"
 2: 
 3: #include <libvirt/libvirt.h>
 4: 
 5: /*top*/
 6: extern int t(void);
 7: int main(int argc, char **argv)
 8: {
 9:   if (argc > 1000000) {
10:     int (* volatile tp)(void)=(int (*)(void))&t;
11:     printf("%d", (*tp)());
12:   }
13: 
14:   return !!argv[argc];
15: }
16: extern void virConnectOpen();
17: int t(void) { virConnectOpen(); return 0; }
/* end */

--------------------

[trinitronx]

Circling back on this... I've been able to work around the linker issue by installing vagrant as a gem via RVM gemset + bundler (happily ignoring its complaints). Following the process here, I was able to install vagrant-libvirt plugin and even get test-kitchen working with it.

So it's possible to build & use vagrant natively on Apple Silicon, and link it against native libraries. It's just a matter of when official packages will be built.

[snadorp]

@trinitronx hfv support has just been merged to mainline: https://gitlab.com/libvirt/libvirt/-/issues/147#note_821234039

[jottr]

Is there a specific reason why there is no release available that specifically targets the arm64 platform?
As @trinitronx correctly pointed out, running the packaged x86/64 ruby through Rosetta on Apple Silicon causes build issues for anything that requires native packages.

[jottr]

@JobeanSmith please provide context on why you closed this issue. Thank you.

[JobeanSmith]

It's now working for me since last comment so I close this issue, thanks !

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.