[xenial] Test upgrade path: Ubuntu 14.04.5 → 14.04.5 with all updates → 16.04

[eloquence]

Our plan of record is to upgrade from Ubuntu 14.04 to Ubuntu 16.04 by first ensuring that the system is updated to all latest packages available for Ubuntu 14.04 (not just security updates).

This is because the package version state of a SecureDrop server partially depends on whether or not the administrator has run the Ansible playbook, which causes a full package update.

To ensure that this strategy is viable, we should test the following upgrade path:

1. Install from a 14.04.5 base image.
2. Run apt-get update && apt-get upgrade
3. Run do-release-upgrade

Beyond looking for showstopper bugs, we will want to compare the final system state with an install from a 16.04 base image to ensure that this upgrade does not result in odd inconsistencies that could cause problems later. This research may generate more tickets.

Part of #3204.

[kushaldas]

Building dependency tree          
Reading state information... Done
Building data structures... Done 

Updating repository information

Third party sources disabled 

Some third party entries in your sources.list were disabled. You can 
re-enable them after the upgrade with the 'software-properties' tool 
or your package manager. 

To continue please press [ENTER]

While running do-release-upgrade we will see the above notice, this is a standard procedure from the Operating System vendors (think: Red Hat) to make sure that all third party vendor repos are disabled. That way, the chances of breakage due to missing packages(or dependencies) are low.

Even in the case of Fedora, we ask folks disable all third party repositories.

[kushaldas]

The xml error which we saw in #3968 (comment) does not occur if we execute just do-release-upgrade command. This is interesting.

[kushaldas]

These are various User INPUT I had to give during the upgrade process.

Updating repository information

Third party sources disabled 

Some third party entries in your sources.list were disabled. You can 
re-enable them after the upgrade with the 'software-properties' tool 
or your package manager. 

To continue please press [ENTER]


...

Do you want to start the upgrade? 


12 installed packages are no longer supported by Canonical. You can 
still get support from the community. 

19 packages are going to be removed. 141 new packages are going to be 
installed. 494 packages are going to be upgraded. 

You have to download a total of 248 M. This download will take about 
4 minutes with your connection. 

Installing the upgrade can take several hours. Once the download has 
finished, the process cannot be canceled. 

 Continue [yN]  Details [d]

Configuration file '/etc/modprobe.d/blacklist.conf'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** blacklist.conf (Y/I/N/O/D/Z) [default=N] ? 

Configuration file '/etc/ssh/moduli'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** moduli (Y/I/N/O/D/Z) [default=N] ? 

Configuration file '/etc/ssh/ssh_config'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** ssh_config (Y/I/N/O/D/Z) [default=N] ?
Setting up openssh-sftp-server (1:7.2p2-4ubuntu2.6) ...
Setting up openssh-server (1:7.2p2-4ubuntu2.6) ...
Installing new version of config file /etc/network/if-up.d/openssh-server ...

Configuration file '/etc/pam.d/sshd'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** sshd (Y/I/N/O/D/Z) [default=N] ? 




Processing triggers for resolvconf (1.78ubuntu6) ...
Errors were encountered while processing:
 initramfs-tools
Error in function: 


*** Send problem report to the developers?

After the problem report has been sent, please fill out the form in the
automatically opened web browser.

What would you like to do? Your options are:
  S: Send report (374.4 KB)
  V: View report
  K: Keep report file for sending later or copying to somewhere else
  I: Cancel and ignore future crashes of this program version
  C: Cancel
Please choose (S/V/K/I/C): K

[eloquence]

Just a note that part of the task here is to do a system state comparison (installed packages, state of relevant config files, etc.) of the upgraded system with a fresh 16.04 install -- we want to make sure that upgrading from 14.04.5 to 14.04.5 with all updates to 16.04 doesn't result in inconsistencies that could cause problems with SecureDrop.

[kushaldas]

Packages in the default xenial app (based on libvirt_staging_xenial_app_staging):

• acl Access control list utilities
• adwaita-icon-theme default icon theme of GNOME (small subset)
• at-spi2-core Assistive Technology Service Provider Interface (dbus core)
• bcache-tools bcache userspace tools
• btrfs-tools Checksumming Copy on Write Filesystem utilities
• cloud-guest-utils cloud guest utilities
• cloud-initramfs-copymods copy initramfs modules into root filesystem for later use
• cloud-initramfs-dyn-netconf write a network interface file in /run for BOOTIF
• colord system service to manage device colour profiles -- system daemon
• colord-data system service to manage device colour profiles -- data files
• dconf-gsettings-backend:amd64 simple configuration storage system - GSettings back-end
• dconf-service simple configuration storage system - D-Bus service
• dns-root-data DNS root data including root zone and DNSSEC key
• dnsmasq-base Small caching DNS proxy and DHCP/TFTP server
• firefox Safe and easy web browser from Mozilla
• gettext GNU Internationalization utilities
• git fast, scalable, distributed revision control system
• git-man fast, scalable, distributed revision control system (manual pages)
• glib-networking:amd64 network-related giomodules for GLib
• glib-networking-common network-related giomodules for GLib - data files
• glib-networking-services network-related giomodules for GLib - D-Bus services
• grub-legacy-ec2 Handles update-grub for ec2 instances
• gsettings-desktop-schemas GSettings desktop-wide schemas
• humanity-icon-theme Humanity Icon theme
• ifenslave configure network interfaces for parallel routing (bonding)
• libasound2:amd64 shared library for ALSA applications
• libasound2-data Configuration files and profiles for ALSA drivers
• libasprintf-dev:amd64 GNU Internationalization library development files
• libatk-bridge2.0-0:amd64 AT-SPI 2 toolkit bridge - shared library
• libatm1:amd64 shared library for ATM (Asynchronous Transfer Mode)
• libatspi2.0-0:amd64 Assistive Technology Service Provider Interface - shared library
• libboost-filesystem1.58.0:amd64 filesystem operations (portable paths, iteration over directories, etc) in C++
• libboost-system1.58.0:amd64 Operating system (e.g. diagnostics support) library
• libcairo-gobject2:amd64 Cairo 2D vector graphics library (GObject library)
• libcapnp-0.5.3:amd64 Cap'n Proto C++ library
• libcgi-fast-perl CGI subclass for work with FCGI
• libcgi-pm-perl module for Common Gateway Interface applications
• libcolord2:amd64 system service to manage device colour profiles -- runtime
• libcolorhug2:amd64 library to access the ColorHug colourimeter -- runtime
• libcroco3:amd64 Cascading Style Sheet (CSS) parsing and manipulation toolkit
• libcwidget3v5:amd64 high-level terminal interface library for C++ (runtime files)
• libdconf1:amd64 simple configuration storage system - runtime library
• libdumbnet1:amd64 dumb, portable networking library -- shared library
• libegl1-mesa:amd64 free implementation of the EGL API -- runtime
• libencode-locale-perl utility to determine the locale encoding
• libepoxy0:amd64 OpenGL function pointer management library
• liberror-perl Perl module for error/exception handling in an OO-ish way
• libexif12:amd64 library to parse EXIF files
• libfcgi-perl helper module for FastCGI
• libgbm1:amd64 generic buffer management API -- runtime
• libgd3:amd64 GD Graphics Library
• libgettextpo-dev:amd64 GNU Internationalization library development files
• libgettextpo0:amd64 GNU Internationalization library
• libgphoto2-6:amd64 gphoto2 digital camera library
• libgphoto2-l10n gphoto2 digital camera library - localized messages
• libgphoto2-port12:amd64 gphoto2 digital camera port library
• libgtk-3-0:amd64 GTK+ graphical user interface library
• libgtk-3-bin programs for the GTK+ graphical user interface library
• libgtk-3-common common files for the GTK+ graphical user interface library
• libgudev-1.0-0:amd64 GObject-based wrapper library for libudev
• libgusb2:amd64 GLib wrapper around libusb1
• libhtml-parser-perl collection of modules that parse HTML text documents
• libhtml-tagset-perl Data tables pertaining to HTML
• libhttp-date-perl module of date conversion routines
• libhttp-message-perl perl interface to HTTP style messages
• libieee1284-3:amd64 cross-platform library for parallel port access
• libio-html-perl open an HTML file with automatic charset detection
• libjson-glib-1.0-0:amd64 GLib JSON manipulation library
• libjson-glib-1.0-common GLib JSON manipulation library (common files)
• liblcms2-2:amd64 Little CMS 2 color management library
• libltdl7:amd64 System independent dlopen wrapper for GNU libtool
• liblwp-mediatypes-perl module to guess media type for a file or a URL
• liblxc1 Linux Containers userspace tools (library)
• liblzo2-2:amd64 data compression library
• libmirclient9:amd64 Display server for Ubuntu - client library
• libmircommon7:amd64 Display server for Ubuntu - shared library
• libmircore1:amd64 Display server for Ubuntu - shared library
• libmirprotobuf3:amd64 Display server for Ubuntu - RPC definitions
• libmspack0:amd64 library for Microsoft compression formats (shared library)
• libnetfilter-conntrack3:amd64 Netfilter netlink-conntrack library
• libprotobuf-lite9v5:amd64 protocol buffers C++ library (lite version)
• libproxy1v5:amd64 automatic proxy configuration management library (shared)
• librest-0.7-0:amd64 REST service access library
• librsvg2-2:amd64 SAX-based renderer library for SVG files (runtime)
• librsvg2-common:amd64 SAX-based renderer library for SVG files (extra runtime)
• libsane:amd64 API library for scanners
• libsane-common API library for scanners -- documentation and support files
• libsigc++-2.0-0v5:amd64 type-safe Signal Framework for C++ - runtime
• libsoup-gnome2.4-1:amd64 HTTP library implementation in C -- GNOME support library
• libsoup2.4-1:amd64 HTTP library implementation in C -- Shared library
• libstartup-notification0:amd64 library for program launch feedback (shared library)
• libunistring0:amd64 Unicode string library for C
• liburi-perl module to manipulate and access URI strings
• libvpx3:amd64 VP8 and VP9 video codec (shared library)
• libwayland-client0:amd64 wayland compositor infrastructure - client library
• libwayland-cursor0:amd64 wayland compositor infrastructure - cursor library
• libwayland-egl1-mesa:amd64 implementation of the Wayland EGL platform -- runtime
• libwayland-server0:amd64 wayland compositor infrastructure - server library
• libxcb-util1:amd64 utility libraries for X C Binding -- atom, aux and event
• libxcb-xfixes0:amd64 X C Binding, xfixes extension
• libxfont1:amd64 X11 font rasterisation library
• libxkbcommon0:amd64 library interface to the XKB compiler - shared library
• libxkbfile1:amd64 X11 keyboard file manipulation library
• libxmlsec1 XML security library
• libxmlsec1-openssl Openssl engine for the XML security library
• libxslt1.1:amd64 XSLT 1.0 processing library - runtime library
• linux-firmware-image-4.4.167-grsec Linux kernel firmware, version 4.4.167-grsec
• linux-image-4.4.0-131-generic Linux kernel image for version 4.4.0 on 64 bit x86 SMP
• linux-image-4.4.167-grsec Linux kernel, version 4.4.167-grsec
• linux-image-extra-4.4.0-131-generic Linux kernel extra modules for version 4.4.0 on 64 bit x86 SMP
• lxc-common Linux Containers userspace tools (common tools)
• lxcfs FUSE based filesystem for LXC
• lxd Container hypervisor based on LXC - daemon
• lxd-client Container hypervisor based on LXC - client
• mdadm tool to administer Linux MD arrays (software RAID)
• open-iscsi iSCSI initiator tools
• open-vm-tools Open VMware Tools for virtual machines hosted on VMware (CLI)
• overlayroot use an overlayfs on top of a read-only root filesystem
• pinentry-curses curses-based PIN or pass-phrase entry dialog for GnuPG
• python3-software-properties manage the repositories that you install software from
• snapd Daemon and tooling that enable snap packages
• software-properties-common manage the repositories that you install software from (common)
• sosreport Set of tools to gather troubleshooting data from a system
• squashfs-tools Tool to create and append to squashfs filesystems
• ubuntu-cloudimage-keyring GnuPG keys of the Ubuntu Cloud Image builder
• ubuntu-mono Ubuntu Mono Icon theme
• uidmap programs to help use subuids
• vlan user mode programs to enable VLANs on your ethernet devices
• x11-xkb-utils X11 XKB utilities
• xauth X authentication utility
• xfonts-base standard fonts for X
• xfonts-encodings Encodings for X.Org fonts
• xfonts-utils X Window System font utility programs
• xfsprogs Utilities for managing the XFS filesystem
• xserver-common common files used by various X servers
• xvfb Virtual Framebuffer 'fake' X server
• zerofree zero free blocks from ext2, ext3 and ext4 file-systems

Pakcages in upgraded xenial box from production app vm

• apt-xapian-index maintenance and search tools for a Xapian index of Debian packages
• bc GNU bc arbitrary precision calculator language
• biosdevname apply BIOS-given names to network devices
• cgmanager Central cgroup manager daemon
• cpp-4.8 GNU C preprocessor
• g++-4.8 GNU C++ compiler
• gcc-4.8 GNU C compiler
• gcc-4.8-base:amd64 GCC, the GNU Compiler Collection (base package)
• gcc-4.9-base:amd64 GCC, the GNU Compiler Collection (base package)
• heirloom-mailx feature-rich BSD mail(1) -- transitional package
• landscape-common The Landscape administration system client - Common files
• libapt-inst1.5:amd64 deb package format runtime library
• libapt-pkg4.12:amd64 package management runtime library
• libarchive-extract-perl generic archive extracting module
• libasan0:amd64 AddressSanitizer -- a fast memory error detector
• libbind9-90 BIND9 Shared Library used by BIND
• libboost-iostreams1.54.0:amd64 Boost.Iostreams Library
• libcgmanager0:amd64 Central cgroup manager daemon (client library)
• libck-connector0:amd64 ConsoleKit libraries
• libcloog-isl4:amd64 Chunky Loop Generator (runtime library)
• libcurl3:amd64 easy-to-use client-side URL transfer library (OpenSSL flavour)
• libcwidget3 high-level terminal interface library for C++ (runtime files)
• libdns100 DNS Shared Library used by BIND
• libept1.4.12:amd64 High-level library for managing Debian package information
• libgc1c2:amd64 conservative garbage collector for C and C++
• libgcc-4.8-dev:amd64 GCC support library (development files)
• libgck-1-0:amd64 Glib wrapper library for PKCS#11 - runtime
• libgcr-3-common Library for Crypto UI related tasks - common files
• libgcr-base-3-1:amd64 Library for Crypto related tasks
• libgcrypt11:amd64 LGPL Crypto library - runtime library
• libgnutls26:amd64 GNU TLS library - runtime library
• libgssglue1:amd64 mechanism-switch gssapi library
• libisc95 ISC Shared Library used by BIND
• libisccc90 Command Channel Library used by BIND
• libisccfg90 Config File Handling Library used by BIND
• libisl10:amd64 manipulating sets and relations of integer points bounded by linear constraints
• libiw30:amd64 Wireless tools - library
• libjasper1:amd64 JasPer JPEG-2000 runtime library
• libjson0:amd64 JSON manipulation library (transitional package)
• liblog-message-perl powerful and flexible message logging mechanism
• liblog-message-simple-perl simplified interface to Log::Message
• liblwres90 Lightweight Resolver Library used by BIND
• libmodule-pluggable-perl module for giving modules the ability to have plugins
• libmodule-runtime-perl Perl module for runtime module handling
• libnih-dbus1:amd64 NIH D-Bus Bindings Library
• libnl-genl-3-200:amd64 library for dealing with netlink sockets - generic netlink
• libpam-cap:amd64 POSIX 1003.1e capabilities (PAM module)
• libparams-classify-perl Perl module for argument type classification
• libparted0debian1:amd64 disk partition manipulator - shared library
• libpcsclite1:amd64 Middleware to access a smart card using PC/SC (library)
• libplymouth2:amd64 graphical boot animation and logger - shared libraries
• libpod-latex-perl module to convert Pod data to formatted LaTeX
• libprocps3:amd64 library for accessing process information from /proc
• libpth20:amd64 GNU Portable Threads
• libpython3.4-minimal:amd64 Minimal subset of the Python language (version 3.4)
• librtmp0:amd64 toolkit for RTMP streams (shared library)
• libsecret-1-0:amd64 Secret store
• libsecret-common Secret store (common files)
• libsigc++-2.0-0c2a:amd64 type-safe Signal Framework for C++ - runtime
• libstdc++-4.8-dev:amd64 GNU Standard C++ Library v3 (development files)
• libsystemd-daemon0:amd64 systemd utility library
• libsystemd-login0:amd64 systemd login utility library
• libterm-ui-perl Term::ReadLine UI made easy
• libtext-soundex-perl implementation of the soundex algorithm
• libxapian-1.3-5:amd64 Search engine library
• libxapian22 Search engine library
• libxtables10 netfilter xtables library
• linux-firmware-image-4.4.144-grsec Linux kernel firmware, version 4.4.144-grsec
• linux-image-4.4.0-31-generic Linux kernel image for version 4.4.0 on 64 bit x86 SMP
• linux-image-4.4.144-grsec Linux kernel, version 4.4.144-grsec
• linux-image-extra-4.4.0-31-generic Linux kernel extra modules for version 4.4.0 on 64 bit x86 SMP
• lockfile-progs Programs for locking and unlocking files and mailboxes
• memtest86
• module-init-tools transitional dummy package (module-init-tools to kmod)
• mountall filesystem mounting tool
• perl-modules Core Perl modules
• pinentry-gtk2 GTK+-2-based PIN or pass-phrase entry dialog for GnuPG
• python-attr Attributes without boilerplate (Python 2)
• python-cffi-backend Foreign Function Interface for Python calling C code - backend
• python-chardet universal character encoding detector for Python2
• python-colorama Cross-platform colored terminal text in Python - Python 2.x
• python-configobj simple but powerful config file reader and writer for Python 2
• python-cryptography Python library exposing cryptographic recipes and primitives (Python 2)
• python-debian Python modules to work with Debian-related data formats
• python-distlib low-level components of python distutils2/packaging
• python-enum34 backport of Python 3.4's enum package
• python-gdbm GNU dbm database support for Python
• python-html5lib HTML parser/tokenizer based on the WHATWG HTML5 specification (Python 2)
• python-idna Python IDNA2008 (RFC 5891) handling (Python 2)
• python-ipaddress Backport of Python 3 ipaddress module (Python 2)
• python-ndg-httpsclient enhanced HTTPS support for httplib and urllib2 using PyOpenSSL for Python2
• python-openssl Python 2 wrapper around the OpenSSL library
• python-pam Python interface to the PAM library
• python-pyasn1 ASN.1 library for Python (Python 2 module)
• python-pyasn1-modules Collection of protocols modules written in ASN.1 language
• python-requests elegant and simple HTTP library for Python2, built for human beings
• python-serial pyserial - module encapsulating access for the serial port
• python-service-identity Service identity verification for pyOpenSSL (Python 2 module)
• python-six Python 2 and 3 compatibility library (Python 2 interface)
• python-twisted-bin Event-based framework for internet applications
• python-twisted-core Event-based framework for internet applications
• python-urllib3 HTTP library with thread-safe connection pooling for Python
• python-xapian Xapian search engine interface for Python
• python-zope.interface Interfaces for Python
• python3-xapian1.3 Xapian search engine interface for Python3
• python3.4 Interactive high-level object-oriented language (version 3.4)
• python3.4-minimal Minimal subset of the Python language (version 3.4)
• systemd-services systemd runtime services
• systemd-shim shim for systemd
• update-motd compliments pam_motd in libpam-modules
• upstart event-based init daemon - essential binaries
• w3m WWW browsable pager with excellent tables/frames support
• watershed reduce superfluous executions of idempotent command
• wireless-tools Tools for manipulating Linux Wireless Extensions
• wpasupplicant client support for WPA and WPA2 (IEEE 802.11i)

Packages in the default xenial mon (based on libvirt_staging_xenial_mon_staging)

• acl Access control list utilities
• bcache-tools bcache userspace tools
• btrfs-tools Checksumming Copy on Write Filesystem utilities
• cloud-guest-utils cloud guest utilities
• cloud-initramfs-copymods copy initramfs modules into root filesystem for later use
• cloud-initramfs-dyn-netconf write a network interface file in /run for BOOTIF
• dns-root-data DNS root data including root zone and DNSSEC key
• dnsmasq-base Small caching DNS proxy and DHCP/TFTP server
• git fast, scalable, distributed revision control system
• git-man fast, scalable, distributed revision control system (manual pages)
• grub-legacy-ec2 Handles update-grub for ec2 instances
• ifenslave configure network interfaces for parallel routing (bonding)
• libatm1:amd64 shared library for ATM (Asynchronous Transfer Mode)
• libcgi-fast-perl CGI subclass for work with FCGI
• libcgi-pm-perl module for Common Gateway Interface applications
• libcwidget3v5:amd64 high-level terminal interface library for C++ (runtime files)
• libdpkg-perl Dpkg perl modules
• libdumbnet1:amd64 dumb, portable networking library -- shared library
• libencode-locale-perl utility to determine the locale encoding
• liberror-perl Perl module for error/exception handling in an OO-ish way
• libfcgi-perl helper module for FastCGI
• libfile-fcntllock-perl Perl module for file locking with fcntl(2)
• libhtml-parser-perl collection of modules that parse HTML text documents
• libhtml-tagset-perl Data tables pertaining to HTML
• libhttp-date-perl module of date conversion routines
• libhttp-message-perl perl interface to HTTP style messages
• libio-html-perl open an HTML file with automatic charset detection
• liblwp-mediatypes-perl module to guess media type for a file or a URL
• liblxc1 Linux Containers userspace tools (library)
• libmspack0:amd64 library for Microsoft compression formats (shared library)
• libnetfilter-conntrack3:amd64 Netfilter netlink-conntrack library
• libsigc++-2.0-0v5:amd64 type-safe Signal Framework for C++ - runtime
• liburi-perl module to manipulate and access URI strings
• libxmlsec1 XML security library
• libxmlsec1-openssl Openssl engine for the XML security library
• libxslt1.1:amd64 XSLT 1.0 processing library - runtime library
• linux-firmware-image-4.4.167-grsec Linux kernel firmware, version 4.4.167-grsec
• linux-image-4.4.0-131-generic Linux kernel image for version 4.4.0 on 64 bit x86 SMP
• linux-image-4.4.167-grsec Linux kernel, version 4.4.167-grsec
• linux-image-extra-4.4.0-131-generic Linux kernel extra modules for version 4.4.0 on 64 bit x86 SMP
• lxc-common Linux Containers userspace tools (common tools)
• lxcfs FUSE based filesystem for LXC
• lxd Container hypervisor based on LXC - daemon
• lxd-client Container hypervisor based on LXC - client
• mdadm tool to administer Linux MD arrays (software RAID)
• open-iscsi iSCSI initiator tools
• open-vm-tools Open VMware Tools for virtual machines hosted on VMware (CLI)
• overlayroot use an overlayfs on top of a read-only root filesystem
• python3-software-properties manage the repositories that you install software from
• snapd Daemon and tooling that enable snap packages
• software-properties-common manage the repositories that you install software from (common)
• sosreport Set of tools to gather troubleshooting data from a system
• squashfs-tools Tool to create and append to squashfs filesystems
• ubuntu-cloudimage-keyring GnuPG keys of the Ubuntu Cloud Image builder
• uidmap programs to help use subuids
• vlan user mode programs to enable VLANs on your ethernet devices
• xfsprogs Utilities for managing the XFS filesystem
• zerofree zero free blocks from ext2, ext3 and ext4 file-systems

Pakcages in upgraded xenial box from production mon vm

• apt-xapian-index maintenance and search tools for a Xapian index of Debian packages
• bc GNU bc arbitrary precision calculator language
• biosdevname apply BIOS-given names to network devices
• cgmanager Central cgroup manager daemon
• cpp-4.8 GNU C preprocessor
• gcc-4.8 GNU C compiler
• gcc-4.8-base:amd64 GCC, the GNU Compiler Collection (base package)
• gcc-4.9-base:amd64 GCC, the GNU Compiler Collection (base package)
• heirloom-mailx feature-rich BSD mail(1) -- transitional package
• landscape-common The Landscape administration system client - Common files
• libapt-inst1.5:amd64 deb package format runtime library
• libapt-pkg4.12:amd64 package management runtime library
• libarchive-extract-perl generic archive extracting module
• libasan0:amd64 AddressSanitizer -- a fast memory error detector
• libbind9-90 BIND9 Shared Library used by BIND
• libboost-iostreams1.54.0:amd64 Boost.Iostreams Library
• libcgmanager0:amd64 Central cgroup manager daemon (client library)
• libck-connector0:amd64 ConsoleKit libraries
• libcloog-isl4:amd64 Chunky Loop Generator (runtime library)
• libcurl3:amd64 easy-to-use client-side URL transfer library (OpenSSL flavour)
• libcwidget3 high-level terminal interface library for C++ (runtime files)
• libdns100 DNS Shared Library used by BIND
• libept1.4.12:amd64 High-level library for managing Debian package information
• libgcc-4.8-dev:amd64 GCC support library (development files)
• libgck-1-0:amd64 Glib wrapper library for PKCS#11 - runtime
• libgcr-3-common Library for Crypto UI related tasks - common files
• libgcr-base-3-1:amd64 Library for Crypto related tasks
• libgcrypt11:amd64 LGPL Crypto library - runtime library
• libgnutls26:amd64 GNU TLS library - runtime library
• libgssglue1:amd64 mechanism-switch gssapi library
• libisc95 ISC Shared Library used by BIND
• libisccc90 Command Channel Library used by BIND
• libisccfg90 Config File Handling Library used by BIND
• libisl10:amd64 manipulating sets and relations of integer points bounded by linear constraints
• libiw30:amd64 Wireless tools - library
• libjson0:amd64 JSON manipulation library (transitional package)
• libkyotocabinet16:amd64 Straightforward implementation of DBM - shared library
• liblog-message-perl powerful and flexible message logging mechanism
• liblog-message-simple-perl simplified interface to Log::Message
• liblwres90 Lightweight Resolver Library used by BIND
• libmodule-pluggable-perl module for giving modules the ability to have plugins
• libmodule-runtime-perl Perl module for runtime module handling
• libmysqlclient18:amd64 MySQL database client library
• libnih-dbus1:amd64 NIH D-Bus Bindings Library
• libnl-genl-3-200:amd64 library for dealing with netlink sockets - generic netlink
• libpam-cap:amd64 POSIX 1003.1e capabilities (PAM module)
• libparams-classify-perl Perl module for argument type classification
• libparted0debian1:amd64 disk partition manipulator - shared library
• libpcsclite1:amd64 Middleware to access a smart card using PC/SC (library)
• libplymouth2:amd64 graphical boot animation and logger - shared libraries
• libpod-latex-perl module to convert Pod data to formatted LaTeX
• libprocps3:amd64 library for accessing process information from /proc
• libpython3.4-minimal:amd64 Minimal subset of the Python language (version 3.4)
• librtmp0:amd64 toolkit for RTMP streams (shared library)
• libsigc++-2.0-0c2a:amd64 type-safe Signal Framework for C++ - runtime
• libsystemd-daemon0:amd64 systemd utility library
• libsystemd-login0:amd64 systemd login utility library
• libterm-ui-perl Term::ReadLine UI made easy
• libtext-soundex-perl implementation of the soundex algorithm
• libxapian-1.3-5:amd64 Search engine library
• libxapian22 Search engine library
• libxtables10 netfilter xtables library
• linux-firmware-image-4.4.144-grsec Linux kernel firmware, version 4.4.144-grsec
• linux-image-4.4.0-31-generic Linux kernel image for version 4.4.0 on 64 bit x86 SMP
• linux-image-4.4.144-grsec Linux kernel, version 4.4.144-grsec
• linux-image-extra-4.4.0-31-generic Linux kernel extra modules for version 4.4.0 on 64 bit x86 SMP
• lockfile-progs Programs for locking and unlocking files and mailboxes
• memtest86
• module-init-tools transitional dummy package (module-init-tools to kmod)
• mountall filesystem mounting tool
• perl-modules Core Perl modules
• python-attr Attributes without boilerplate (Python 2)
• python-cffi-backend Foreign Function Interface for Python calling C code - backend
• python-chardet universal character encoding detector for Python2
• python-configobj simple but powerful config file reader and writer for Python 2
• python-cryptography Python library exposing cryptographic recipes and primitives (Python 2)
• python-debian Python modules to work with Debian-related data formats
• python-enum34 backport of Python 3.4's enum package
• python-gdbm GNU dbm database support for Python
• python-idna Python IDNA2008 (RFC 5891) handling (Python 2)
• python-ipaddress Backport of Python 3 ipaddress module (Python 2)
• python-ndg-httpsclient enhanced HTTPS support for httplib and urllib2 using PyOpenSSL for Python2
• python-openssl Python 2 wrapper around the OpenSSL library
• python-pam Python interface to the PAM library
• python-pkg-resources Package Discovery and Resource Access using pkg_resources
• python-pyasn1 ASN.1 library for Python (Python 2 module)
• python-pyasn1-modules Collection of protocols modules written in ASN.1 language
• python-requests elegant and simple HTTP library for Python2, built for human beings
• python-serial pyserial - module encapsulating access for the serial port
• python-service-identity Service identity verification for pyOpenSSL (Python 2 module)
• python-six Python 2 and 3 compatibility library (Python 2 interface)
• python-twisted-bin Event-based framework for internet applications
• python-twisted-core Event-based framework for internet applications
• python-urllib3 HTTP library with thread-safe connection pooling for Python
• python-xapian Xapian search engine interface for Python
• python-zope.interface Interfaces for Python
• python3-xapian1.3 Xapian search engine interface for Python3
• python3.4 Interactive high-level object-oriented language (version 3.4)
• python3.4-minimal Minimal subset of the Python language (version 3.4)
• systemd-services systemd runtime services
• systemd-shim shim for systemd
• update-motd compliments pam_motd in libpam-modules
• upstart event-based init daemon - essential binaries
• w3m WWW browsable pager with excellent tables/frames support
• watershed reduce superfluous executions of idempotent command
• wireless-tools Tools for manipulating Linux Wireless Extensions
• wpasupplicant client support for WPA and WPA2 (IEEE 802.11i)

Many of the upgraded package names came up as they are of new version and the version name is part of the package name.

The default xenial image which we use in molecule also has a lot of extra packages, for example, lxd or cloud-* packages. Also wondering why Xenial has both systemd and upstart :)

The upgraded app vm gets gcc and g++ compilers,

[kushaldas]

While trying to upgrade the mon-server from the tails vm, I got the following. As you can see, many of those ncurses screens are totally messed up.

First, sudo vim /etc/update-manager/release-upgrades and changed the value of Prompt to lts.

Next, run sudo do-release-upgrade.

Then, it will ask to press Enter.

Reading package lists... Done    
Building dependency tree          
Reading state information... Done
Building data structures... Done 

Updating repository information

Third party sources disabled 

Some third party entries in your sources.list were disabled. You can 
re-enable them after the upgrade with the 'software-properties' tool 
or your package manager. 

To continue please press [ENTER]

And after some time, it will give the details about upgrade and tell to press y to continue.

12 installed packages are no longer supported by Canonical. You can 
still get support from the community. 

10 packages are going to be removed. 144 new packages are going to be 
installed. 434 packages are going to be upgraded. 

You have to download a total of 243 M. This download will take about 
52 seconds with your connection. 

Installing the upgrade can take several hours. Once the download has 
finished, the process cannot be canceled. 
  
 Continue [yN]  Details [d]

Next, it will ask about restarting the services, I chose yes by using the TAB.

Then, it will ask about language selector, I have selected <Ok> by using the TAB key and pressed Enter.
(You can see the messed up UI).

Next, it will give information about postfix and you have to press TAB to select <Ok> and pressed Enter.

Then, the actual postfix configuration screen, I have selected no configuration required (default value), and pressed TAB to select <Ok> and pressed Enter.

Next, the following input is required. I kept pressing Enter for the default input.

Configuration file '/etc/modprobe.d/blacklist.conf'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** blacklist.conf (Y/I/N/O/D/Z) [default=N] ? 



Configuration file '/etc/ssh/moduli'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** moduli (Y/I/N/O/D/Z) [default=N] ? 

Configuration file '/etc/ssh/ssh_config'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** ssh_config (Y/I/N/O/D/Z) [default=N] ? 

Configuration file '/etc/pam.d/sshd'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** sshd (Y/I/N/O/D/Z) [default=N] ? 

After this it will show a few grub related screens and we will have to verify what we see on the actual hardware based installations. I did select any and kept pressing Enter.

The next screen is about removing extra packages, where I pressed N and then pressed Enter.

Searching for obsolete software
Reading package lists... Done    
Building dependency tree          
Reading state information... Done
Building data structures... Done 
Building data structures... Done 

Remove obsolete packages? 


46 packages are going to be removed. 

 Continue [yN]  Details [d]N

Restart required 

To finish the upgrade, a restart is required. 
If you select 'y' the system will be restarted. 

Continue [yN] 

Press N and then it will ask you to press x to end the process. The Mon server should be restarted with Xenial.

[kushaldas]

A new error I saw today:

Calculating the changes

Calculating the changes

Could not calculate the upgrade 

An unresolvable problem occurred while calculating the upgrade. 

This can be caused by: 
* Upgrading to a pre-release version of Ubuntu 
* Running the current pre-release version of Ubuntu 
* Unofficial software packages not provided by Ubuntu 

If none of this applies, then please report this bug using the 
command 'ubuntu-bug ubuntu-release-upgrader-core' in a terminal. 


Restoring original system state

Aborting
Reading package lists... Done    
Building dependency tree          
Reading state information... Done
Building data structures... Done 
=== Command terminated with exit status 1 (Tue Feb  5 12:02:04 2019) ===

Seeing this again and again on my staging vm. @conorsch @eloquence @heartsucker @emkll @redshiftzero

[kushaldas]

Good part is the above error is only happening on staging, no clue on why.

[emkll]

12 installed packages are no longer supported by Canonical. You can
still get support from the community.

I suspect this means they moved the channel from main to universe. We can find out what these packages are in /var/log/dist-upgrade/main.log searching for demoted:

2019-02-05 17:02:02,252 DEBUG demoted: 'biosdevname gcc-4.8-base gcc-4.9-base libarchive-extract-perl liblog-message-simple-perl libmodule-pluggable-perl libpod-latex-perl libterm-ui-perl libtext-soundex-perl module-init-tools python-debian w3m'

[emkll]

I am still not sure why wifi-related packages are being installed in the upgrade scenario, per the list provided here: #3965 (comment)

[zenmonkeykstop]

Ran through do-release-upgrade via Tails admin against Monitor Server on physical hardware (7-series NUC). Prompts were as follows:

• do-release-upgrade:
  • ssh mon
  • edit /etc/update-manager/release-upgrades, change DEFAULT=never to DEFAULT=lts
  • press Enter after the "Some third party entries..." message
  • press y to continue after "Fetching and installing..." message (screen is messed up for me - resizing it helps a little (130x25 seems best) )
  • Choose OK on postfix info dialog
  • Choose No Configuration and OK on General type of mail configuration dialog
  • Choose Yes on Configuring libssl.0.0:amd64 dialog (connection breaks, presumably because sshd restarted)
  • ssh mon again after a few seconds
  • Choose No to overriding local changes, on PAM configuration dialog
  • Choose default action (keep local) on blacklist.conf changes
  • Choose Keep the local version currently installed on Configuring grub-efi-amd64 dialog
  • Choose default (keep current version) for /etc/ssh/moduli
  • Choose default (keep current version) for /etc/ssh/ssh_config
  • Choose default (keep current version) for /etc/pam.d/sshd
  • Choose keep local version currently installed and OK on Configuring unattended-upgrades dialog
  • Choose Y to remove obsolete packages
  • Choose Y to restart system and complete update

Update completes successfully, can't reconnect after initial reboot (might just be impatience on my part), but after rebooting again I can connect via ssh mon.