Raw (encrypted) send failure

[zdykstra]

When attempting to use znapzend to send a snapshot on a pool that's fully encrypted to a remote pool that is unencrypted, I'm receiving an error, even though the snapshot(s) do arrive on the destination without issue.

Sender details: Void Linux, ZFS 0.8.4

# zfs get encryptionroot,encryption,keyformat,keylocation,keystatus zroot
NAME   PROPERTY        VALUE                      SOURCE
zroot  encryptionroot  zroot                      -
zroot  encryption      aes-256-gcm                -
zroot  keyformat       passphrase                 -
zroot  keylocation     file:///etc/zfs/zroot.key  local
zroot  keystatus       available                  -

# zfs list
NAME                         USED  AVAIL     REFER  MOUNTPOINT
zroot                       28.4G   433G      192K  none
zroot/ROOT                  16.0G   433G      192K  none
zroot/ROOT/void.2020.06.22  8.24G   433G     8.24G  /
zroot/home                  12.4G   433G     12.4G  /home

# zfs list -t snapshot
no datasets available

# znapzendzetup list
*** backup plan: zroot/ROOT ***
           dst_0 = root@rethe:zroot/backups/auri
      dst_0_plan = 60days=>4hours
         enabled = on
         mbuffer = off
    mbuffer_size = 1G
   post_znap_cmd = off
    pre_znap_cmd = off
       recursive = on
             src = zroot/ROOT
        src_plan = 30days=>4hours
        tsformat = %Y-%m-%d-%H%M%S
      zend_delay = 0

Receiver details: Void Linux, ZFS 0.8.4. For the purposes of this test, zroot/backups has been destroyed, so we're starting from scratch.

Full log of a run

# znapzend --loglevel=debug --features=sendRaw --autoCreation --runonce=zroot/ROOT
[2020-06-29 21:11:22.78305] [23015] [info] znapzend (PID=23015) starting up ...
[2020-06-29 21:11:22.78330] [23015] [info] refreshing backup plans for dataset "zroot/ROOT" ...
cannot open 'zroot/backups/auri': dataset does not exist
cannot open 'zroot/backups/auri': dataset does not exist
[2020-06-29 21:11:24.62637] [23015] [info] creating destination dataset 'root@rethe:zroot/backups/auri'...
[2020-06-29 21:11:24.62740] [23015] [info] found a valid backup plan for zroot/ROOT...
[2020-06-29 21:11:24.63057] [23015] [info] znapzend (PID=23015) initialized -- resuming normal operations.
[2020-06-29 21:11:24.63409] [23015] [debug] snapshot worker for zroot/ROOT spawned (23046)
[2020-06-29 21:11:24.63684] [23046] [info] creating recursive snapshot on zroot/ROOT
[2020-06-29 21:11:24.69364] [23046] [info] checking for explicitly excluded ZFS dependent datasets under 'zroot/ROOT'
[2020-06-29 21:11:24.74121] [23015] [debug] snapshot worker for zroot/ROOT done (23046)
[2020-06-29 21:11:24.74296] [23015] [debug] send/receive worker for zroot/ROOT spawned (23084)
[2020-06-29 21:11:24.74378] [23084] [info] starting work on backupSet zroot/ROOT
[2020-06-29 21:11:24.75973] [23084] [debug] sending snapshots from zroot/ROOT to root@rethe:zroot/backups/auri
cannot receive new filesystem stream: zfs receive -F cannot be used to destroy an encrypted filesystem or overwrite an unencrypted one with an encrypted one
[2020-06-29 21:11:25.11679] [23084] [warn] ERROR: cannot send snapshots to zroot/backups/auri on root@rethe
[2020-06-29 21:11:25.11771] [23084] [debug] sending snapshots from zroot/ROOT/void.2020.06.22 to root@rethe:zroot/backups/auri/void.2020.06.22
cannot open 'zroot/backups/auri/void.2020.06.22': dataset does not exist
[2020-06-29 21:14:16.76205] [23084] [warn] ERROR: suspending cleanup source dataset because 1 send task(s) failed:
[2020-06-29 21:14:16.76263] [23084] [warn]  +-->   ERROR: cannot send snapshots to zroot/backups/auri on root@rethe
[2020-06-29 21:14:16.76283] [23084] [info] done with backupset zroot/ROOT in 172 seconds
[2020-06-29 21:14:16.76750] [23015] [debug] send/receive worker for zroot/ROOT done (23084)

Filesystems on remote host

# zfs list | grep backups
zroot/backups                                                                       8.24G  1.64T       96K  none
zroot/backups/auri                                                                  8.24G  1.64T       96K  none
zroot/backups/auri/void.2020.06.22                                                  8.24G  1.64T     8.24G  none
# zfs list -t snapshot | grep backups
zroot/backups/auri/void.2020.06.22@2020-06-29-211124                                             0B      -     8.24G  -

Is there a step I'm missing or a configuration option I'm otherwise not enabling? Any insight into this would be greatly appreciated.

Edit: As an aside, it's great to see Perl and Mojolicious in use in the wild!

[crabique]

See https://github.com/oetiker/znapzend/blob/master/lib/ZnapZend/ZFS.pm#L355

     -F      Force a rollback of the file system to the most recent snap-
             shot before performing the receive operation. If receiving an
             incremental replication stream (for example, one generated by
             "zfs send -R -Fi -iI"), destroy snapshots and file systems
             that do not exist on the sending side.

The -F argument to zfs recv is always used by znapzend, so it is not possible to back up encrypted datasets into non-encrypted destination, as can be seen from your log:

cannot receive new filesystem stream: zfs receive -F cannot be used to destroy an encrypted filesystem or overwrite an unencrypted one with an encrypted one

[zdykstra]

See https://github.com/oetiker/znapzend/blob/master/lib/ZnapZend/ZFS.pm#L355

     -F      Force a rollback of the file system to the most recent snap-
             shot before performing the receive operation. If receiving an
             incremental replication stream (for example, one generated by
             "zfs send -R -Fi -iI"), destroy snapshots and file systems
             that do not exist on the sending side.

The -F argument to zfs recv is always used by znapzend, so it is not possible to back up encrypted datasets into non-encrypted destination, as can be seen from your log:

cannot receive new filesystem stream: zfs receive -F cannot be used to destroy an encrypted filesystem or overwrite an unencrypted one with an encrypted one

Yes, that's the crux of this problem - which is why opened the issue. If I manually create an encrypted filesystem on the remote side zfs create -o encryption=aes-256-gcm -o keylocation=prompt -o keyformat=passphrase zroot/backups/auri I still get the same error, even though I'm sending an encrypted snapshot to an encrypted filesystem.

Is there any way around this? If not, I don't see how I can use znapzend to send snapshots of my laptop to a remote host and still have them be encrypted at rest.

[hansoostendorp]

I think this is related to the issue I just submitted: #495 . When the destination dataset doesn't exist and you use autoCreation, ZnapZend will create a non encrypted destination dataset. But you can't send encrypted snapshots to a non encrypted dataset. The way around this, is to send the first encrypted snapshot yourself without the -F flag to a destination dataset that doesn't exist yet. After this, ZendZnap can send following snapshots without problems.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.