Skip to content

Commit 08337aa

Browse files
stepanblyschakstepanblyschak
authored
[sonic-package-manager] first phase implementation of sonic-package-manager (#1527)
What I did Implemented sonic-package-manager utility to manager SONiC Packages as per HLD sonic-net/SONiC#682. Implemented optional logic to migrate packages into new SONiC image in sonic-installer. How I did it Implemented as per HLD sonic-net/SONiC#682. How to verify it (Doc: sonic-net/SONiC#682) install package uninstall package upgrade package S2S upgrade THANK YOU, Stepan!
1 parent c166f66 commit 08337aa

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+5633
-24
lines changed

‎doc/Command-Reference.md

+317-1
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,7 @@
143143
* [Watermark Show commands](#watermark-show-commands)
144144
* [Watermark Config commands](#watermark-config-commands)
145145
* [Software Installation and Management](#software-installation-and-management)
146+
* [SONiC Package Manager](#sonic-package-manager)
146147
* [SONiC Installer](#sonic-installer)
147148
* [Troubleshooting Commands](#troubleshooting-commands)
148149
* [Routing Stack](#routing-stack)
@@ -7961,8 +7962,316 @@ Go Back To [Beginning of the document](#) or [Beginning of this section](#waterm
79617962
79627963
## Software Installation and Management
79637964
7964-
SONiC software can be installed in two methods, viz, "using sonic-installer tool", "ONIE Installer".
7965+
SONiC images can be installed in one of two methods:
7966+
1. From within a running SONiC image using the `sonic-installer` utility
7967+
2. From the vendor's bootloader (E.g., ONIE, Aboot, etc.)
79657968
7969+
SONiC packages are available as prebuilt Docker images and meant to be installed with the *sonic-package-manager* utility.
7970+
7971+
### SONiC Package Manager
7972+
7973+
The *sonic-package-manager* is a command line tool to manage (e.g. install, upgrade or uninstall) SONiC Packages.
7974+
7975+
**sonic-package-manager list**
7976+
7977+
This command lists all available SONiC packages, their desription, installed version and installation status.
7978+
SONiC package status can be *Installed*, *Not installed* or *Built-In*. "Built-In" status means that a feature is built-in to SONiC image and can't be upgraded or uninstalled.
7979+
7980+
- Usage:
7981+
```
7982+
sonic-package-manager list
7983+
```
7984+
7985+
- Example:
7986+
```
7987+
admin@sonic:~$ sonic-package-manager list
7988+
Name Repository Description Version Status
7989+
-------------- --------------------------- ---------------------------- --------- --------------
7990+
cpu-report azure/cpu-report CPU report package N/A Not Installed
7991+
database docker-database SONiC database package 1.0.0 Built-In
7992+
dhcp-relay azure/docker-dhcp-relay SONiC dhcp-relay package 1.0.0 Installed
7993+
fpm-frr docker-fpm-frr SONiC fpm-frr package 1.0.0 Built-In
7994+
lldp docker-lldp SONiC lldp package 1.0.0 Built-In
7995+
macsec docker-macsec SONiC macsec package 1.0.0 Built-In
7996+
mgmt-framework docker-sonic-mgmt-framework SONiC mgmt-framework package 1.0.0 Built-In
7997+
nat docker-nat SONiC nat package 1.0.0 Built-In
7998+
pmon docker-platform-monitor SONiC pmon package 1.0.0 Built-In
7999+
radv docker-router-advertiser SONiC radv package 1.0.0 Built-In
8000+
sflow docker-sflow SONiC sflow package 1.0.0 Built-In
8001+
snmp docker-snmp SONiC snmp package 1.0.0 Built-In
8002+
swss docker-orchagent SONiC swss package 1.0.0 Built-In
8003+
syncd docker-syncd-mlnx SONiC syncd package 1.0.0 Built-In
8004+
teamd docker-teamd SONiC teamd package 1.0.0 Built-In
8005+
telemetry docker-sonic-telemetry SONiC telemetry package 1.0.0 Built-In
8006+
```
8007+
8008+
**sonic-package-manager repository add**
8009+
8010+
This command will add a new repository as source for SONiC packages to the database. *NOTE*: requires elevated (root) privileges to run
8011+
8012+
- Usage:
8013+
```
8014+
Usage: sonic-package-manager repository add [OPTIONS] NAME REPOSITORY
8015+
8016+
Add a new repository to database.
8017+
8018+
NOTE: This command requires elevated (root) privileges to run.
8019+
8020+
Options:
8021+
--default-reference TEXT Default installation reference. Can be a tag or
8022+
sha256 digest in repository.
8023+
--description TEXT Optional package entry description.
8024+
--help Show this message and exit.
8025+
```
8026+
- Example:
8027+
```
8028+
admin@sonic:~$ sudo sonic-package-manager repository add \
8029+
cpu-report azure/sonic-cpu-report --default-reference 1.0.0
8030+
```
8031+
8032+
**sonic-package-manager repository remove**
8033+
8034+
This command will remove a repository as source for SONiC packages from the database . The package has to be *Not Installed* in order to be removed from package database. *NOTE*: requires elevated (root) privileges to run
8035+
8036+
- Usage:
8037+
```
8038+
Usage: sonic-package-manager repository remove [OPTIONS] NAME
8039+
8040+
Remove repository from database.
8041+
8042+
NOTE: This command requires elevated (root) privileges to run.
8043+
8044+
Options:
8045+
--help Show this message and exit.
8046+
```
8047+
- Example:
8048+
```
8049+
admin@sonic:~$ sudo sonic-package-manager repository remove cpu-report
8050+
```
8051+
8052+
**sonic-package-manager install**
8053+
8054+
This command pulls and installs a package on SONiC host. *NOTE*: this command requires elevated (root) privileges to run
8055+
8056+
- Usage:
8057+
```
8058+
Usage: sonic-package-manager install [OPTIONS] [PACKAGE_EXPR]
8059+
8060+
Install/Upgrade package using [PACKAGE_EXPR] in format
8061+
"<name>[=<version>|@<reference>]".
8062+
8063+
The repository to pull the package from is resolved by lookup in
8064+
package database, thus the package has to be added via "sonic-
8065+
package-manager repository add" command.
8066+
8067+
In case when [PACKAGE_EXPR] is a package name "<name>" this command
8068+
will install or upgrade to a version referenced by "default-
8069+
reference" in package database.
8070+
8071+
NOTE: This command requires elevated (root) privileges to run.
8072+
8073+
Options:
8074+
--enable Set the default state of the feature to enabled
8075+
and enable feature right after installation. NOTE:
8076+
user needs to execute "config save -y" to make
8077+
this setting persistent.
8078+
--set-owner [local|kube] Default owner configuration setting for a feature.
8079+
--from-repository TEXT Fetch package directly from image registry
8080+
repository. NOTE: This argument is mutually
8081+
exclusive with arguments: [package_expr,
8082+
from_tarball].
8083+
--from-tarball FILE Fetch package from saved image tarball. NOTE: This
8084+
argument is mutually exclusive with arguments:
8085+
[package_expr, from_repository].
8086+
-f, --force Force operation by ignoring package dependency
8087+
tree and package manifest validation failures.
8088+
-y, --yes Automatically answer yes on prompts.
8089+
-v, --verbosity LVL Either CRITICAL, ERROR, WARNING, INFO or DEBUG.
8090+
Default is INFO.
8091+
--skip-host-plugins Do not install host OS plugins provided by the
8092+
package (CLI, etc). NOTE: In case when package
8093+
host OS plugins are set as mandatory in package
8094+
manifest this option will fail the installation.
8095+
--allow-downgrade Allow package downgrade. By default an attempt to
8096+
downgrade the package will result in a failure
8097+
since downgrade might not be supported by the
8098+
package, thus requires explicit request from the
8099+
user.
8100+
--help Show this message and exit..
8101+
```
8102+
- Example:
8103+
```
8104+
admin@sonic:~$ sudo sonic-package-manager install dhcp-relay=1.0.2
8105+
```
8106+
```
8107+
admin@sonic:~$ sudo sonic-package-manager install dhcp-relay@latest
8108+
```
8109+
```
8110+
admin@sonic:~$ sudo sonic-package-manager install dhcp-relay@sha256:9780f6d83e45878749497a6297ed9906c19ee0cc48cc88dc63827564bb8768fd
8111+
```
8112+
```
8113+
admin@sonic:~$ sudo sonic-package-manager install --from-repository azure/sonic-cpu-report:latest
8114+
```
8115+
```
8116+
admin@sonic:~$ sudo sonic-package-manager install --from-tarball sonic-docker-image.gz
8117+
```
8118+
8119+
**sonic-package-manager uninstall**
8120+
8121+
This command uninstalls package from SONiC host. User needs to stop the feature prior to uninstalling it.
8122+
*NOTE*: this command requires elevated (root) privileges to run.
8123+
8124+
- Usage:
8125+
```
8126+
Usage: sonic-package-manager uninstall [OPTIONS] NAME
8127+
8128+
Uninstall package.
8129+
8130+
NOTE: This command requires elevated (root) privileges to run.
8131+
8132+
Options:
8133+
-f, --force Force operation by ignoring package dependency tree and
8134+
package manifest validation failures.
8135+
-y, --yes Automatically answer yes on prompts.
8136+
-v, --verbosity LVL Either CRITICAL, ERROR, WARNING, INFO or DEBUG. Default
8137+
is INFO.
8138+
--help Show this message and exit.
8139+
```
8140+
- Example:
8141+
```
8142+
admin@sonic:~$ sudo sonic-package-manager uninstall dhcp-relay
8143+
```
8144+
8145+
**sonic-package-manager reset**
8146+
8147+
This comamnd resets the package by reinstalling it to its default version. *NOTE*: this command requires elevated (root) privileges to run.
8148+
8149+
- Usage:
8150+
```
8151+
Usage: sonic-package-manager reset [OPTIONS] NAME
8152+
8153+
Reset package to the default version.
8154+
8155+
NOTE: This command requires elevated (root) privileges to run.
8156+
8157+
Options:
8158+
-f, --force Force operation by ignoring package dependency tree and
8159+
package manifest validation failures.
8160+
-y, --yes Automatically answer yes on prompts.
8161+
-v, --verbosity LVL Either CRITICAL, ERROR, WARNING, INFO or DEBUG. Default
8162+
is INFO.
8163+
--skip-host-plugins Do not install host OS plugins provided by the package
8164+
(CLI, etc). NOTE: In case when package host OS plugins
8165+
are set as mandatory in package manifest this option
8166+
will fail the installation.
8167+
--help Show this message and exit.
8168+
```
8169+
- Example:
8170+
```
8171+
admin@sonic:~$ sudo sonic-package-manager reset dhcp-relay
8172+
```
8173+
8174+
**sonic-package-manager show package versions**
8175+
8176+
This command will retrieve a list of all available versions for the given package from the configured upstream repository
8177+
8178+
- Usage:
8179+
```
8180+
Usage: sonic-package-manager show package versions [OPTIONS] NAME
8181+
8182+
Show available versions.
8183+
8184+
Options:
8185+
--all Show all available tags in repository.
8186+
--plain Plain output.
8187+
--help Show this message and exit.
8188+
```
8189+
- Example:
8190+
```
8191+
admin@sonic:~$ sonic-package-manager show package versions dhcp-relay
8192+
• 1.0.0
8193+
• 1.0.2
8194+
• 2.0.0
8195+
```
8196+
```
8197+
admin@sonic:~$ sonic-package-manager show package versions dhcp-relay --plain
8198+
1.0.0
8199+
1.0.2
8200+
2.0.0
8201+
```
8202+
```
8203+
admin@sonic:~$ sonic-package-manager show package versions dhcp-relay --all
8204+
• 1.0.0
8205+
• 1.0.2
8206+
• 2.0.0
8207+
• latest
8208+
```
8209+
8210+
**sonic-package-manager show package changelog**
8211+
8212+
This command fetches the changelog from the package manifest and displays it. *NOTE*: package changelog can be retrieved from registry or read from image tarball without installing it.
8213+
8214+
- Usage:
8215+
```
8216+
Usage: sonic-package-manager show package changelog [OPTIONS] [PACKAGE_EXPR]
8217+
8218+
Show package changelog.
8219+
8220+
Options:
8221+
--from-repository TEXT Fetch package directly from image registry
8222+
repository NOTE: This argument is mutually exclusive
8223+
with arguments: [from_tarball, package_expr].
8224+
--from-tarball FILE Fetch package from saved image tarball NOTE: This
8225+
argument is mutually exclusive with arguments:
8226+
[package_expr, from_repository].
8227+
--help Show this message and exit.
8228+
```
8229+
- Example:
8230+
```
8231+
admin@sonic:~$ sonic-package-manager show package changelog dhcp-relay
8232+
1.0.0:
8233+
8234+
• Initial release
8235+
8236+
Author ([email protected]) Mon, 25 May 2020 12:25:00 +0300
8237+
```
8238+
8239+
**sonic-package-manager show package manifest**
8240+
8241+
This command fetches the package manifest and displays it. *NOTE*: package manifest can be retrieved from registry or read from image tarball without installing it.
8242+
8243+
- Usage:
8244+
```
8245+
Usage: sonic-package-manager show package manifest [OPTIONS] [PACKAGE_EXPR]
8246+
8247+
Show package manifest.
8248+
8249+
Options:
8250+
--from-repository TEXT Fetch package directly from image registry
8251+
repository NOTE: This argument is mutually exclusive
8252+
with arguments: [package_expr, from_tarball].
8253+
--from-tarball FILE Fetch package from saved image tarball NOTE: This
8254+
argument is mutually exclusive with arguments:
8255+
[from_repository, package_expr].
8256+
-v, --verbosity LVL Either CRITICAL, ERROR, WARNING, INFO or DEBUG
8257+
--help Show this message and exit.
8258+
```
8259+
- Example:
8260+
```
8261+
admin@sonic:~$ sonic-package-manager show package manifest dhcp-relay=2.0.0
8262+
{
8263+
"version": "1.0.0",
8264+
"package": {
8265+
"version": "2.0.0",
8266+
"depends": [
8267+
"database>=1.0.0,<2.0.0"
8268+
]
8269+
},
8270+
"service": {
8271+
"name": "dhcp_relay"
8272+
}
8273+
}
8274+
```
79668275
79678276
### SONiC Installer
79688277
This is a command line tool available as part of the SONiC software; If the device is already running the SONiC software, this tool can be used to install an alternate image in the partition.
@@ -8033,6 +8342,13 @@ This command is used to install a new image on the alternate image partition. T
80338342
Done
80348343
```
80358344
8345+
Installing a new image using the sonic-installer will keep using the packages installed on the currently running SONiC image and automatically migrate those. In order to perform clean SONiC installation use the *--skip-package-migration* option:
8346+
8347+
- Example:
8348+
```
8349+
admin@sonic:~$ sudo sonic-installer install https://sonic-jenkins.westus.cloudapp.azure.com/job/xxxx/job/buildimage-xxxx-all/xxx/artifact/target/sonic-xxxx.bin --skip-package-migration
8350+
```
8351+
80368352
**sonic-installer set_default**
80378353
80388354
This command is be used to change the image which can be loaded by default in all the subsequent reboots.

‎setup.py

+14
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,8 @@
4848
'show.plugins',
4949
'sonic_installer',
5050
'sonic_installer.bootloader',
51+
'sonic_package_manager',
52+
'sonic_package_manager.service_creator',
5153
'tests',
5254
'undebug',
5355
'utilities_common',
@@ -151,20 +153,30 @@
151153
'sonic-clear = clear.main:cli',
152154
'sonic-installer = sonic_installer.main:sonic_installer',
153155
'sonic_installer = sonic_installer.main:sonic_installer', # Deprecated
156+
'sonic-package-manager = sonic_package_manager.main:cli',
157+
'spm = sonic_package_manager.main:cli',
154158
'undebug = undebug.main:cli',
155159
'watchdogutil = watchdogutil.main:watchdogutil',
156160
]
157161
},
158162
install_requires=[
159163
'click==7.0',
164+
'click-log==0.3.2',
165+
'docker==4.4.4',
166+
'docker-image-py==0.1.10',
167+
'filelock==3.0.12',
168+
'enlighten==1.8.0',
160169
'ipaddress==1.0.23',
170+
'jinja2==2.11.3',
161171
'jsondiff==1.2.0',
162172
'jsonpatch==1.32.0',
163173
'm2crypto==0.31.0',
164174
'natsort==6.2.1', # 6.2.1 is the last version which supports Python 2. Can update once we no longer support Python 2
165175
'netaddr==0.8.0',
166176
'netifaces==0.10.7',
167177
'pexpect==4.8.0',
178+
'poetry-semver==0.1.0',
179+
'prettyprinter==0.18.0',
168180
'pyroute2==0.5.14',
169181
'requests==2.25.0',
170182
'sonic-config-engine',
@@ -173,13 +185,15 @@
173185
'sonic-yang-mgmt',
174186
'swsssdk>=2.0.1',
175187
'tabulate==0.8.2',
188+
'www-authenticate==0.9.2',
176189
'xmltodict==0.12.0',
177190
],
178191
setup_requires= [
179192
'pytest-runner',
180193
'wheel'
181194
],
182195
tests_require = [
196+
'pyfakefs',
183197
'pytest',
184198
'mockredispy>=2.9.3',
185199
'deepdiff==5.2.3'

0 commit comments

Comments
 (0)