-
-
Notifications
You must be signed in to change notification settings - Fork 432
[Hetzner Bug]: ARM snapshot failing #732
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
@fmorato Use the |
Just FYI, maybe this maintenance has not ended yet? https://status.hetzner.com/incident/6531f1d2-8cf5-4ecc-959f-f0f9d7e77c70 That's why, maybe wait. |
The same error here. |
Facing the same issue. |
Same here now. Yesterday it worked, Hetzner seems to have changed something. Alternatively the MicroOS image for ARM has changed... Did a test - this is a Hetzner problem - server stays a looong time in NFSv3 mount phase, then it boots finally in rescue mode but seems not to be reachable packer output log
I opened a support request @ hetzner. |
@ifeulner Thanks for confirming that the problem comes from Hetzner. Folks, don't hesitate to open tickets with them too explaining that you are trying to create a snapshot via rescue mode but the ARM node is never coming online. |
It probably comes from the failed "Fetch Hetzner Robot Config". |
After trying to get this working, I finally tracked it down to the Robot config service failing, due to the service being unable to resolve Excerpt from journalctl output of an ARM VM I created on the console and booted it into recovery:
Looking at Doing some further testing in the VM, it seems the DNS setup is broken for the ARM linux64 rescue image: Creating the required directories and appending
|
@Diftraku great findings, I opened a support ticket with referring to this issue conversation here, so I do expect that Hetzner is fixing this quickly... |
Is there a way to use other versions of the install script? tmp_script=$(mktemp) && curl -sSL -o "${tmp_script}" https://raw.githubusercontent.com/kube-hetzner/terraform-hcloud-kube-hetzner/v2.0.8/scripts/create.sh && chmod +x "${tmp_script}" && "${tmp_script}" && rm "${tmp_script}" Enter the name of the folder you want to create (leave empty to use the current directory instead, useful for upgrades):
The snapshot is required and deployed using packer. If you need specific extra packages, you need to choose no and edit hcloud-microos-snapshot.pkr.hcl file manually. This is not needed in 99% of cases, as we already include the most common packages.
Do you want to create the MicroOS snapshot with packer now? (yes/no): yes
Enter your HCLOUD_TOKEN: xxxx
Running: packer build packer build hcloud-microos-snapshot.pkr.hcl
Error: Argument or block definition required
on hcloud-microos-snapshot.pkr.hcl line 1:
1: 404: Not Found
An argument or block definition is required here.
Before running 'terraform apply', go through the kube.tf file and complete your desired values there.
To activate the hcloud CLI for this project, run 'hcloud context create <project-name>'. It is a lot more practical than using the Hetzner UI, and allows for easy cleanup or debugging. |
@aDingil You can use old versions of the project (without ARM), yes. The install script is just for convenience and not relevant here. |
How to revert to a specific version like 2.0.8
mkdir /path/to/your/new/folder
cd /path/to/your/new/folder
curl -sL https://raw.githubusercontent.com/kube-hetzner/terraform-hcloud-kube-hetzner/v2.0.8/kube.tf.example -o kube.tf
curl -sL https://raw.githubusercontent.com/kube-hetzner/terraform-hcloud-kube-hetzner/v2.0.8/packer-template/hcloud-microos-snapshot.pkr.hcl -o hcloud-microos-snapshots.pkr.hcl
export HCLOUD_TOKEN="xxxxxx"
packer build hcloud-microos-snapshot.pkr.hcl
hcloud context create <project-name>
version = "2.0.8" |
Thanks @ifeulner for letting Hetzner know! 🙏 |
Cool, so I didn't get yet an answer by Hetzner... |
Ok, thanks for the info. At least it seems fixed now, they must have seen your ticket for sure, and probably gotten other reports too. |
Nice to have action and discussion here! I had also reported the issue to Hetzner, and I can confirm that the snapshot creation works. However, the issue I was reporting was that terraform doesn't run without either snapshot present. As I wrote in the OP and title of this issue. Apparently the title of the issue was changed to make the Hetzner bug more evident. Are we to expect both snapshots should always be present, even if we're not interested in using one of them? |
Now I got an answer from Hetzner:
So it works again, but there seem to be still some limitations with the ARM based server's rescue system. |
@fmorato We do expect both snapshots to be present, this is by design, as they are dirt cheap to hold and it's just very practical to have both. So to recap, it is a required initial phase in the setup. |
Description
Hi, I found this project yesterday, and was trying it out today.
Following the steps and running the script for generating the snapshots created the
x86
snapshot, but failed for the ARM snapshot. I ran it many times, but it hasn't worked. Tried settingssh_timeout
to longer time (10m), and if I go to the server console I do see that the rescue system booted a while before it times out at 10m, but packer still errors out. It does take longer than the default timeout for the rescue system to boot.I pasted the log lines for the
arm
build below.Without the ARM snapshot, terraform doesn't run
Kube.tf file
Screenshots
No response
Platform
Linux
The text was updated successfully, but these errors were encountered: