- Sponsor
-
Notifications
You must be signed in to change notification settings - Fork 293
Expand block reset logic #1560
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Expand block reset logic #1560
+234
−47
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a more comprehensive approach to storage reseting than what's currently present in clearDiskData as it now uses a set of markers to confirm that the device was correctly reset. It's also expanded to try a variety of different discarding options (from fastest to slowest). Suggested-by: Takero Funaki <[email protected]> Signed-off-by: Stéphane Graber <[email protected]>
Signed-off-by: Stéphane Graber <[email protected]>
hallyn
approved these changes
Jan 6, 2025
It seems that clearBlock() is only executed when a volume is created (lvcreate for LVM driver). |
@takefun hmm, good point. Good thing that blkdiscard support being passed an offset, that'd have made things quite a bit more annoying if it didn't... |
tmeijn
pushed a commit
to tmeijn/dotfiles
that referenced
this pull request
Jan 28, 2025
This MR contains the following updates: | Package | Update | Change | |---|---|---| | [lxc/incus](https://github.com/lxc/incus) | minor | `v6.8.0` -> `v6.9.0` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>lxc/incus (lxc/incus)</summary> ### [`v6.9.0`](https://github.com/lxc/incus/releases/tag/v6.9.0): Incus 6.9 [Compare Source](lxc/incus@v6.8.0...v6.9.0) #### What's Changed - Translations update from Hosted Weblate by [@​weblate](https://github.com/weblate) in lxc/incus#1506 - Refactor scriptlet loader and improve checks by [@​bensmrs](https://github.com/bensmrs) in lxc/incus#1507 - Translations update from Hosted Weblate by [@​weblate](https://github.com/weblate) in lxc/incus#1511 - incusd/storage/drivers: Force blkdiscard and ignore errors by [@​stgraber](https://github.com/stgraber) in lxc/incus#1513 - incusd/auth: fix FGA online data race by [@​breml](https://github.com/breml) in lxc/incus#1515 - incusd/storage/drivers: Log on blkdiscard failure by [@​stgraber](https://github.com/stgraber) in lxc/incus#1517 - Fix issue with live-migration between storage pools by [@​presztak](https://github.com/presztak) in lxc/incus#1518 - Add uplink addresses for OVN network by [@​winiciusallan](https://github.com/winiciusallan) in lxc/incus#1512 - Add `--description` flag to create type CLI commands by [@​luissimas](https://github.com/luissimas) in lxc/incus#1498 - client: Propagate HTTP TLS dialer to websocket by [@​stgraber](https://github.com/stgraber) in lxc/incus#1519 - Fix QEMU scriptlet parameter name by [@​bensmrs](https://github.com/bensmrs) in lxc/incus#1524 - Add shell.nix by [@​MadnessASAP](https://github.com/MadnessASAP) in lxc/incus#1526 - client: add proxy host to skopoe inspect by [@​winiciusallan](https://github.com/winiciusallan) in lxc/incus#1525 - incusd/cluster: Add missing error handling by [@​stgraber](https://github.com/stgraber) in lxc/incus#1528 - incusd/instance/qemu: Fix QMP arguments typing by [@​bensmrs](https://github.com/bensmrs) in lxc/incus#1531 - Encode mc alias for storage buckets to match allowed characters by [@​monkz](https://github.com/monkz) in lxc/incus#1535 - doc: Contributing section restructuring by [@​bensmrs](https://github.com/bensmrs) in lxc/incus#1534 - internal/server: allow creation of OVN network with no uplink by [@​winiciusallan](https://github.com/winiciusallan) in lxc/incus#1533 - Translations update from Hosted Weblate by [@​weblate](https://github.com/weblate) in lxc/incus#1540 - Translations update from Hosted Weblate by [@​weblate](https://github.com/weblate) in lxc/incus#1541 - lxd-to-incus: Clear volatile.uuid from instances by [@​stgraber](https://github.com/stgraber) in lxc/incus#1546 - Translations update from Hosted Weblate by [@​weblate](https://github.com/weblate) in lxc/incus#1548 - Translations update from Hosted Weblate by [@​weblate](https://github.com/weblate) in lxc/incus#1550 - Translations update from Hosted Weblate by [@​weblate](https://github.com/weblate) in lxc/incus#1551 - Translations update from Hosted Weblate by [@​weblate](https://github.com/weblate) in lxc/incus#1553 - Tweak Github Actions and add arm64 tests by [@​stgraber](https://github.com/stgraber) in lxc/incus#1555 - Translations update from Hosted Weblate by [@​weblate](https://github.com/weblate) in lxc/incus#1556 - incusd/networks: Tweak dnsmasq startup condition by [@​alex14641](https://github.com/alex14641) in lxc/incus#1544 - Tweak namespace attach logic in forkproxy and forkfile by [@​stgraber](https://github.com/stgraber) in lxc/incus#1557 - Expand block reset logic by [@​stgraber](https://github.com/stgraber) in lxc/incus#1560 - Discard blocks on LVM resize by [@​stgraber](https://github.com/stgraber) in lxc/incus#1561 - incusd/storage/lvm: Run ClearBlock on all new thick block volumes by [@​stgraber](https://github.com/stgraber) in lxc/incus#1565 - Improve format flag by [@​breml](https://github.com/breml) in lxc/incus#1564 - devcontainer: initial version by [@​breml](https://github.com/breml) in lxc/incus#1566 - incusd/network/ovn: Skip remote peers when iterating peer networks by [@​stgraber](https://github.com/stgraber) in lxc/incus#1568 - Fix discard logic on LVM resize and optimize image unpacking by [@​stgraber](https://github.com/stgraber) in lxc/incus#1569 - incusd/main_cluster: Tweak to have help refer to correct command name by [@​stgraber](https://github.com/stgraber) in lxc/incus#1576 - Docs: Fixed link to point to correct manpage. by [@​foxtrotcz](https://github.com/foxtrotcz) in lxc/incus#1581 - incusd/device/disk: Better handle partitions by [@​stgraber](https://github.com/stgraber) in lxc/incus#1578 - incusd/storage/drivers: Handle 4k sector sizes by [@​stgraber](https://github.com/stgraber) in lxc/incus#1585 - lxd-to-incus: Be more lenient on database changes by [@​stgraber](https://github.com/stgraber) in lxc/incus#1582 - incusd/dnsmasq: Don't put the project name in the DNS record by [@​stgraber](https://github.com/stgraber) in lxc/incus#1586 - Remove virtfs-proxy-helper dependency by [@​bensmrs](https://github.com/bensmrs) in lxc/incus#1547 - Add new `config` hook by [@​bensmrs](https://github.com/bensmrs) in lxc/incus#1577 - github: Switch to Github's hosted arm64 runners by [@​stgraber](https://github.com/stgraber) in lxc/incus#1589 - Support for ACLs for bridge NIC device when using nftables driver by [@​stgraber](https://github.com/stgraber) in lxc/incus#1587 - doc: typo by [@​eebssk1](https://github.com/eebssk1) in lxc/incus#1591 - Added unit for minimum ZFS blocksize by [@​Spitfireap](https://github.com/Spitfireap) in lxc/incus#1593 - incus/file/pull: Port to SFTP by [@​HassanAlsamahi](https://github.com/HassanAlsamahi) in lxc/incus#1592 - incusd/device/disk: Update correct mount options by [@​stgraber](https://github.com/stgraber) in lxc/incus#1594 - doc: Force white background for swagger API by [@​stgraber](https://github.com/stgraber) in lxc/incus#1599 - Implement a VM memory dump API by [@​SpiffyEight77](https://github.com/SpiffyEight77) in lxc/incus#1596 - gomod: Update dependencies by [@​stgraber](https://github.com/stgraber) in lxc/incus#1600 - incus/init: Add support for storage volumes in preseed init by [@​megheaiulian](https://github.com/megheaiulian) in lxc/incus#1595 - incusd/instance/qemu: Fix device_id warning on 9.2.x by [@​stgraber](https://github.com/stgraber) in lxc/incus#1601 - incus: Fix example description for debug dump-memory command by [@​SpiffyEight77](https://github.com/SpiffyEight77) in lxc/incus#1602 #### New Contributors - [@​winiciusallan](https://github.com/winiciusallan) made their first contribution in lxc/incus#1512 - [@​luissimas](https://github.com/luissimas) made their first contribution in lxc/incus#1498 - [@​MadnessASAP](https://github.com/MadnessASAP) made their first contribution in lxc/incus#1526 - [@​monkz](https://github.com/monkz) made their first contribution in lxc/incus#1535 - [@​alex14641](https://github.com/alex14641) made their first contribution in lxc/incus#1544 - [@​foxtrotcz](https://github.com/foxtrotcz) made their first contribution in lxc/incus#1581 - [@​eebssk1](https://github.com/eebssk1) made their first contribution in lxc/incus#1591 - [@​Spitfireap](https://github.com/Spitfireap) made their first contribution in lxc/incus#1593 - [@​megheaiulian](https://github.com/megheaiulian) made their first contribution in lxc/incus#1595 **Full Changelog**: lxc/incus@v6.8.0...v6.9.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMzcuMiIsInVwZGF0ZWRJblZlciI6IjM5LjEzNy4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In Incus 6.8 (and 6.0.3), we started performing block discarding prior to writing the initial volume data. This is primarily useful on thick LVM volumes and in some other situations where an existing volume is being reset (some snapshot operations, volume refreshes, ...) in order to avoid prior data from showing up in the resulting volume.
It's particularly problematic on thick LVM as someone unaware of LVM's design may find themselves getting access to deleted data from another instance, effectively causing a data leakage. That's part of LVM's design for thick volumes as they are just an offset on the underlying PVs and not something unique to Incus, but still something that may catch some users off guard.
Our initial implementation of the reset logic works perfectly for file backed storage and appears to work as expected on the majority of storage options tested.
However we've had reports of some environments, primarily multi-device VGs or devices with broken block discard support where the current approach is insufficient.
With this PR, we end up with a much more thorough implementation of block device resetting as we now:
This should catch all cases and goes from safest and easiest all the way to safe but extremely slow as the ultimate fallback.
Reported-by: Takero Funaki [email protected]