Skip to content

Commit d304850

Browse files
bgilbertbaude
andcommitted
Add support for Hyper-V platform
Read the Ignition config from the Hyper-V Data Exchange Service ("KVP"), which can be configured on the host via WMI. KVP on Linux is normally handled by a combination of a kernel module (hv_utils) and a daemon shipped with the kernel source (hv_kvp_daemon). The latter writes world-readable files to /var/lib/hyperv; these contain an array of binary structs. The host-guest protocol has the guest OS as the passive peer; hv_kvp_daemon connects and is sent all the host's key-value pairs. We don't want to require hv_kvp_daemon as a dependency, so we use a pure Go reimplementation (libhvee) that supports just enough of the protocol to receive the KVPs. However, if we disconnect and then hv_kvp_daemon reconnects later, the host won't re-send the KVPs, so we also need to save them to /var/lib/hyperv in the same format used by the daemon. Unlike the daemon, we set the files to mode 600, since the Ignition config is potentially sensitive; the daemon does not change the permissions of existing files. Small configs can be stored in the `ignition.config` KVP. However, individual values have severe length limitations (~1 KiB of UTF-8), so we also support concatenating `ignition.config.0`, `ignition.config.1`, etc. to form a larger config. See also: coreos/fedora-coreos-tracker#1411 coreos/fedora-coreos-tracker#1424 Almost all of this work was done by Brent Baude; I just added the final polish. Thanks Brent! Co-authored-by: Brent Baude <[email protected]>
1 parent 153df40 commit d304850

File tree

11 files changed

+609
-0
lines changed

11 files changed

+609
-0
lines changed

docs/release-notes.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ Starting with this release, ignition-validate binaries are signed with the
1414

1515
### Features
1616

17+
- Support Hyper-V platform
1718

1819
### Changes
1920

docs/supported-platforms.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ Ignition is currently only supported for the following platforms:
1515
* [DigitalOcean] (`digitalocean`) - Ignition will read its configuration from the droplet userdata. Cloud SSH keys and network configuration are handled separately.
1616
* [Exoscale] (`exoscale`) - Ignition will read its configuration from the instance userdata. Cloud SSH keys are handled separately.
1717
* [Google Cloud] (`gcp`) - Ignition will read its configuration from the instance metadata entry named "user-data". Cloud SSH keys are handled separately.
18+
* [Microsoft Hyper-V] (`hyperv`) - Ignition will read its configuration from the `ignition.config` key in pool 0 of the Hyper-V Data Exchange Service (KVP). Values are limited to approximately 1 KiB of text, so Ignition can also read and concatenate multiple keys named `ignition.config.0`, `ignition.config.1`, and so on.
1819
* [IBM Cloud] (`ibmcloud`) - Ignition will read its configuration from the instance userdata. Cloud SSH keys are handled separately.
1920
* [KubeVirt] (`kubevirt`) - Ignition will read its configuration from the instance userdata via config drive. Cloud SSH keys are handled separately.
2021
* Bare Metal (`metal`) - Use the `ignition.config.url` kernel parameter to provide a URL to the configuration. The URL can use the `http://`, `https://`, `tftp://`, `s3://`, `arn:`, or `gs://` schemes to specify a remote config.
@@ -41,6 +42,7 @@ For most cloud providers, cloud SSH keys and custom network configuration are ha
4142
[DigitalOcean]: https://www.digitalocean.com/products/droplets/
4243
[Exoscale]: https://www.exoscale.com/compute/
4344
[Google Cloud]: https://cloud.google.com/compute
45+
[Microsoft Hyper-V]: https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/
4446
[IBM Cloud]: https://www.ibm.com/cloud/vpc
4547
[KubeVirt]: https://kubevirt.io
4648
[Nutanix]: https://www.nutanix.com/products/ahv

dracut/30ignition/module-setup.sh

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,3 +96,8 @@ install() {
9696
# needed for openstack config drive support
9797
inst_rules 60-cdrom_id.rules
9898
}
99+
100+
installkernel() {
101+
# required by hyperv platform to read kvp from the kernel
102+
instmods -c hv_utils
103+
}

go.mod

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ require (
77
cloud.google.com/go/storage v1.30.1
88
github.com/aws/aws-sdk-go v1.44.284
99
github.com/beevik/etree v1.2.0
10+
github.com/containers/libhvee v0.2.0
1011
github.com/coreos/go-semver v0.3.1
1112
github.com/coreos/go-systemd/v22 v22.5.0
1213
github.com/coreos/vcontext v0.0.0-20230201181013-d72178a18687

go.sum

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,8 @@ github.com/cncf/udpa/go v0.0.0-20210930031921-04548b0d99d4/go.mod h1:6pvJx4me5XP
2525
github.com/cncf/xds/go v0.0.0-20210805033703-aa0b78936158/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
2626
github.com/cncf/xds/go v0.0.0-20210922020428-25de7278fc84/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
2727
github.com/cncf/xds/go v0.0.0-20211011173535-cb28da3451f1/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
28+
github.com/containers/libhvee v0.2.0 h1:6h7LdSvBt176oIdMXKkgxdoT/IVP+o/gkwgyjWzvEAo=
29+
github.com/containers/libhvee v0.2.0/go.mod h1:Zr2Qhnl5THW/HQjF1o8HmxXWjvHfJb8fvd0ThTzHMys=
2830
github.com/coreos/go-json v0.0.0-20230131223807-18775e0fb4fb h1:rmqyI19j3Z/74bIRhuC59RB442rXUazKNueVpfJPxg4=
2931
github.com/coreos/go-json v0.0.0-20230131223807-18775e0fb4fb/go.mod h1:rcFZM3uxVvdyNmsAV2jopgPD1cs5SPWJWU5dOz2LUnw=
3032
github.com/coreos/go-semver v0.3.1 h1:yi21YpKnrx1gt5R+la8n5WgS0kCrsPp33dmEyHReZr4=

internal/providers/hyperv/kvp.go

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
// Copyright 2023 Red Hat
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
package hyperv
16+
17+
import (
18+
"fmt"
19+
"os/exec"
20+
"path/filepath"
21+
22+
"github.com/containers/libhvee/pkg/kvp"
23+
"github.com/coreos/ignition/v2/config/shared/errors"
24+
"github.com/coreos/ignition/v2/config/v3_5_experimental/types"
25+
"github.com/coreos/ignition/v2/internal/distro"
26+
"github.com/coreos/ignition/v2/internal/platform"
27+
"github.com/coreos/ignition/v2/internal/providers/util"
28+
"github.com/coreos/ignition/v2/internal/resource"
29+
"github.com/coreos/vcontext/report"
30+
)
31+
32+
const singleKey = "ignition.config"
33+
34+
// Prefix for multiple config fragments to reassemble. The suffix is a
35+
// sequential integer starting from 0.
36+
const splitKeyPrefix = "ignition.config."
37+
38+
func init() {
39+
platform.Register(platform.Provider{
40+
Name: "hyperv",
41+
FetchWithFiles: fetchConfig,
42+
})
43+
}
44+
45+
func fetchConfig(f *resource.Fetcher) ([]types.File, types.Config, report.Report, error) {
46+
var kvpFiles []types.File
47+
48+
// To read key-value pairs from the Windows host, the hv_util kernel
49+
// module must be loaded to create the kernel device
50+
_, err := f.Logger.LogCmd(exec.Command(distro.ModprobeCmd(), "hv_utils"), "loading hv_utils kernel module")
51+
if err != nil {
52+
return nil, types.Config{}, report.Report{}, fmt.Errorf("loading hv_utils kernel module: %w", err)
53+
}
54+
55+
keyValuePairs, err := kvp.GetKeyValuePairs()
56+
if err != nil {
57+
return nil, types.Config{}, report.Report{}, fmt.Errorf("reading key-value pairs: %w", err)
58+
}
59+
60+
var ign string
61+
kv, err := keyValuePairs[kvp.DefaultKVPPoolID].GetValueByKey(singleKey)
62+
if err == nil {
63+
f.Logger.Debug("found single KVP key")
64+
ign = kv.Value
65+
} else if err != kvp.ErrKeyNotFound {
66+
return nil, types.Config{}, report.Report{}, fmt.Errorf("looking up single KVP key: %w", err)
67+
}
68+
69+
if ign == "" {
70+
ign, err = keyValuePairs.GetSplitKeyValues(splitKeyPrefix, kvp.DefaultKVPPoolID)
71+
if err == nil {
72+
f.Logger.Debug("found concatenated KVP keys")
73+
} else if err != kvp.ErrNoKeyValuePairsFound {
74+
return nil, types.Config{}, report.Report{}, fmt.Errorf("reassembling split config: %w", err)
75+
}
76+
}
77+
78+
// hv_kvp_daemon writes pools to the filesystem in /var/lib/hyperv.
79+
// We've already read the pool data, and the host won't send it again
80+
// on this boot, so we need to write the files ourselves.
81+
for poolID := range keyValuePairs {
82+
// hv_kvp_daemon writes the pool files with mode 644 in a
83+
// directory with mode 755. This isn't safe for us, since
84+
// it leaks the config to non-root users, including on
85+
// subsequent boots.
86+
// - There's no API that lets us delete the KVPs from the host.
87+
// - We could filter out the KVPs when writing the pools,
88+
// but if hv_kvp_daemon runs on subsequent boots, it could
89+
// re-add them.
90+
// - The caller doesn't give us a way to create directory
91+
// entries, only files; and we probably shouldn't set
92+
// restrictive permissions on /var/lib/hyperv because it
93+
// hypothetically might be used for other purposes.
94+
// Avoid the issue by setting the files to mode 600.
95+
// hv_kvp_daemon won't change the mode afterward.
96+
poolPath := filepath.Join(kvp.DefaultKVPFilePath, fmt.Sprintf("%s%d", kvp.DefaultKVPBaseName, poolID))
97+
kvpFiles = append(kvpFiles, util.MakeProviderOutputFile(poolPath, 0600, keyValuePairs.EncodePoolFile(poolID)))
98+
}
99+
100+
if ign == "" {
101+
return kvpFiles, types.Config{}, report.Report{}, errors.ErrEmpty
102+
}
103+
104+
c, r, err := util.ParseConfig(f.Logger, []byte(ign))
105+
return kvpFiles, c, r, err
106+
}

internal/register/providers.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ import (
2424
_ "github.com/coreos/ignition/v2/internal/providers/exoscale"
2525
_ "github.com/coreos/ignition/v2/internal/providers/file"
2626
_ "github.com/coreos/ignition/v2/internal/providers/gcp"
27+
_ "github.com/coreos/ignition/v2/internal/providers/hyperv"
2728
_ "github.com/coreos/ignition/v2/internal/providers/ibmcloud"
2829
_ "github.com/coreos/ignition/v2/internal/providers/kubevirt"
2930
_ "github.com/coreos/ignition/v2/internal/providers/metal"

vendor/github.com/containers/libhvee/LICENSE

Lines changed: 201 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)