url: support IPv4 and IPv6 for IMDS

[tormath1]

Hi,

This PR is another approach to try solving the IPv4 / IPv6 question on Ignition for Scaleway and OpenStack providers. The idea is to implement a helper: resource.FetchConfigDualStack that will try to fetch Ignition configuration from multiple endpoints (IPv4 and IPv6). The first one to return a configuration will win the race.

This allows to have a common implementation as each provider can decide if it wants to use or not IPv4, IPv6 or both and by still implementing its own fetching logic.

Related to: #1897, flatcar/Flatcar#1696 and #1909

Locally tested on Scaleway and currently tested on Flatcar CI (flatcar/scripts#2824) but it's pointless as we don't have yet tests for IPv6 on Scaleway or OpenStack:

 # IPv4 ONLY
core@scw-confident-keldysh ~ $ sudo ./ignition --platform scaleway --log-to-stdout --stage fetch --config-cache ./config.json
INFO     : Ignition v2.21.0-4-gb111af12-dirty
INFO     : Stage: fetch
INFO     : no config dir at "/usr/lib/ignition/base.d"
INFO     : no config dir at "/usr/lib/ignition/base.platform.d/scaleway"
DEBUG    : parsed url from cmdline: ""
INFO     : no config URL provided
INFO     : reading system config file "/usr/lib/ignition/user.ign"
INFO     : no config at "/usr/lib/ignition/user.ign"
INFO     : GET http://169.254.42.42/user_data/cloud-init: attempt #1
INFO     : GET http://[fd00:42::42]/user_data/cloud-init: attempt #1
INFO     : GET error: Get "http://[fd00:42::42]/user_data/cloud-init": dial tcp :381->[fd00:42::42]:80: connect: network is unreachable
INFO     : GET result: OK
DEBUG    : got configuration from: http://169.254.42.42/user_data/cloud-init
DEBUG    : parsing config with SHA512: 7d25de56ceb1cb987792f3ceba9494086376e6b640421d3ffdae0cd29660bd8ae852bdd77040e205b4eeabbc0c899a6ac571e3f65d1a6753c84091766c8a2e81
INFO     : fetch: fetch complete
INFO     : fetch: fetch passed
INFO     : Ignition finished successfully


# IPv4 and IPv6
core@scw-confident-keldysh ~ $ sudo ./ignition --platform scaleway --log-to-stdout --stage fetch --config-cache ./config.json
INFO     : Ignition v2.21.0-4-gb111af12-dirty
INFO     : Stage: fetch
INFO     : no config dir at "/usr/lib/ignition/base.d"
INFO     : no config dir at "/usr/lib/ignition/base.platform.d/scaleway"
DEBUG    : parsed url from cmdline: ""
INFO     : no config URL provided
INFO     : reading system config file "/usr/lib/ignition/user.ign"
INFO     : no config at "/usr/lib/ignition/user.ign"
INFO     : GET http://169.254.42.42/user_data/cloud-init: attempt #1
INFO     : GET http://[fd00:42::42]/user_data/cloud-init: attempt #1
INFO     : GET result: OK
INFO     : GET result: OK
DEBUG    : got configuration from: http://169.254.42.42/user_data/cloud-init
DEBUG    : parsing config with SHA512: 7d25de56ceb1cb987792f3ceba9494086376e6b640421d3ffdae0cd29660bd8ae852bdd77040e205b4eeabbc0c899a6ac571e3f65d1a6753c84091766c8a2e81
INFO     : fetch: fetch complete
INFO     : fetch: fetch passed
INFO     : Ignition finished successfully
core@scw-confident-keldysh ~ $ sudo ./ignition --platform scaleway --log-to-stdout --stage fetch --config-cache ./config.json
INFO     : Ignition v2.21.0-4-gb111af12-dirty
INFO     : Stage: fetch
INFO     : no config dir at "/usr/lib/ignition/base.d"
INFO     : no config dir at "/usr/lib/ignition/base.platform.d/scaleway"
DEBUG    : parsed url from cmdline: ""
INFO     : no config URL provided
INFO     : reading system config file "/usr/lib/ignition/user.ign"
INFO     : no config at "/usr/lib/ignition/user.ign"
INFO     : GET http://[fd00:42::42]/user_data/cloud-init: attempt #1
INFO     : GET http://169.254.42.42/user_data/cloud-init: attempt #1
INFO     : GET error: Get "http://169.254.42.42/user_data/cloud-init": dial tcp :995->169.254.42.42:80: bind: address already in use
INFO     : GET result: OK
DEBUG    : got configuration from: http://[fd00:42::42]/user_data/cloud-init
DEBUG    : parsing config with SHA512: 7d25de56ceb1cb987792f3ceba9494086376e6b640421d3ffdae0cd29660bd8ae852bdd77040e205b4eeabbc0c899a6ac571e3f65d1a6753c84091766c8a2e81
INFO     : fetch: fetch complete
INFO     : fetch: fetch passed
INFO     : Ignition finished successfully

# IPv6 only
$ sudo ./ignition --platform scaleway --log-to-stdout --stage fetch --config-cache ./config.json
INFO     : Ignition v2.21.0-4-gb111af12-dirty
INFO     : Stage: fetch
INFO     : no config dir at "/usr/lib/ignition/base.d"
INFO     : no config dir at "/usr/lib/ignition/base.platform.d/scaleway"
DEBUG    : parsed url from cmdline: ""
INFO     : no config URL provided
INFO     : reading system config file "/usr/lib/ignition/user.ign"
INFO     : no config at "/usr/lib/ignition/user.ign"
INFO     : GET http://169.254.42.42/user_data/cloud-init: attempt #1
INFO     : GET error: Get "http://169.254.42.42/user_data/cloud-init": dial tcp :731->169.254.42.42:80: connect: network is unreachable
INFO     : GET http://[fd00:42::42]/user_data/cloud-init: attempt #1
INFO     : GET result: OK
DEBUG    : got configuration from: http://[fd00:42::42]/user_data/cloud-init
DEBUG    : parsing config with SHA512: 7d25de56ceb1cb987792f3ceba9494086376e6b640421d3ffdae0cd29660bd8ae852bdd77040e205b4eeabbc0c899a6ac571e3f65d1a6753c84091766c8a2e81
INFO     : fetch: fetch complete
INFO     : fetch: fetch passed
INFO     : Ignition finished successfully

[prestist]

Sorry if this is a dumb question, If neither succeed what happens? is there a default timeout?

[tormath1]

Not yet, the PR is still in "draft" as I wanted first to have users feedback before on the overall logic. I still need to polish the error handling and the case you mentioned. I'm thinking of having something similar to the OpenStack dispatch implementation here:

ignition/internal/providers/openstack/openstack.go

Line 117 in 2e40256

if dispatchCount == 0 {

[prestist]

No worries :) I was just making sure I was not missing it. Yeah I think that would work well here.

[prestist]

@tormath1 this looks really good; thank you for working on this.

These changes make a lot of sense.

[yasminvalim]

Hey @tormath1, nice work! Thanks for working on that! I cherry-picked your commit that implement the helper resource.FetchConfigDualStack and will use it in my PR for Openstack #1909. WDYT?

[tormath1]

Hey @tormath1, nice work! Thanks for working on that! I cherry-picked your commit that implement the helper resource.FetchConfigDualStack and will use it in my PR for Openstack #1909. WDYT?

Your call on this, that would be good indeed to see how it fits on OpenStack - maybe you can create separate branch to keep your initial proposal around?

[yasminvalim]

Hey @tormath1, nice work! Thanks for working on that! I cherry-picked your commit that implement the helper resource.FetchConfigDualStack and will use it in my PR for Openstack #1909. WDYT?

Your call on this, that would be good indeed to see how it fits on OpenStack - maybe you can create separate branch to keep your initial proposal around?

Hey! I have a local backup with the initial proposal, but I think yours might be a better fit for what we need. I went ahead and updated my PR to use the helper Still working on it, but feel free to take a look :)