[Build] Use apt-get to predictably support dependency ordered configuration of lazy packages (#12164)

Why I did it
The current lazy installer relies on a filename sort for both unpack and configuration steps. When systemd services are configured [started] by multiple packages the order is by filename not by the declared package dependencies. This can cause the start order of services to differ between first-boot and subsequent boots. Declared systemd service dependencies further exacerbate the issue (e.g. blocking the first-boot script).

The current installer leaves packages un-configured if the package dependency order does not match the filename order.

This also fixes a trivial bug in [Build]: Support to use symbol links for lazy installation targets to reduce the image size #10923 where externally downloaded dependencies are duplicated across lazy package device directories.

How I did it
Changed the staging and first-boot scripts to use apt-get:

dpkg -i /host/image-$SONIC_VERSION/platform/$platform/*.deb

becomes

apt-get -y install /host/image-$SONIC_VERSION/platform/$platform/*.deb

when dependencies are detected during image staging.

How to verify it
Apt-get critical rules

Add a Depends= to the control information of a package. Grep the syslog for rc.local between images and observe the configuration order of packages change.

Author: lornelong

files/build_templates/sonic_debian_extension.j2

@@ -646,12 +646,19 @@ sudo mkdir -p $FILESYSTEM_ROOT/$PLATFORM_DIR/common
 sudo cp {{ deb }} $FILESYSTEM_ROOT/$PLATFORM_DIR/common/
 sudo ln -sf "../common/{{ debfilename }}" "$FILESYSTEM_ROOT/$PLATFORM_DIR/{{dev}}/{{ debfilename }}"
 for f in $(find $FILESYSTEM_ROOT/var/cache/apt/archives -name "*.deb"); do
-    sudo mv $f $FILESYSTEM_ROOT/$PLATFORM_DIR/{{dev}}/
+    sudo mv $f $FILESYSTEM_ROOT/$PLATFORM_DIR/common/
+    sudo ln -sf "../common/$(basename $f)" "$FILESYSTEM_ROOT/$PLATFORM_DIR/{{dev}}/$(basename $f)"
 done
 
 sudo dpkg --root=$FILESYSTEM_ROOT -P {{ debname }}
 
 {% endfor %}
+# create a trivial apt repo if any of the debs have dependencies, including between lazy debs
+if [ $(for f in $FILESYSTEM_ROOT/$PLATFORM_DIR/common/*.deb; do \
+           sudo dpkg -I $f | grep "Depends:\|Pre-Depends:"; done | wc -l) -gt 0 ]; then
+    (cd $FILESYSTEM_ROOT/$PLATFORM_DIR/common && sudo dpkg-scanpackages . | \
+         sudo gzip | sudo tee Packages.gz > /dev/null)
+fi
 {% endif %}
 
 # Remove sshd host keys, and will regenerate on first sshd start. This needs to be

files/image_config/platform/rc.local

@@ -284,7 +284,20 @@ if [ -f $FIRST_BOOT_FILE ]; then
     fi
 
     if [ -d /host/image-$SONIC_VERSION/platform/$platform ]; then
-        dpkg -i /host/image-$SONIC_VERSION/platform/$platform/*.deb
+        if [ -f /host/image-$SONIC_VERSION/platform/common/Packages.gz ]; then
+            # Use only the trivial repo and apt to support lazy package dependencies
+            mv /etc/apt/sources.list /etc/apt/sources.list.rc-local
+            echo "deb [trusted=yes] file:///host/image-$SONIC_VERSION/platform/common /" > /etc/apt/sources.list.d/sonic_debian_extension.list
+            LANG=C DEBIAN_FRONTEND=noninteractive apt-get update
+            LANG=C DEBIAN_FRONTEND=noninteractive apt-get -y install /host/image-$SONIC_VERSION/platform/$platform/*.deb
+            # Cleanup
+            rm -f /etc/apt/sources.list.d/sonic_debian_extension.list
+            rm -f /var/lib/apt/lists/_host_image-${SONIC_VERSION}_platform_common_Packages.lz4
+            # This is first boot so there should be no package lists or indices to restore, just the sources
+            mv /etc/apt/sources.list.rc-local /etc/apt/sources.list
+        else
+            dpkg -i /host/image-$SONIC_VERSION/platform/$platform/*.deb
+        fi
     fi
 
     sync