Support shared headroom pool on top of dynamic buffer calculation (sonic-net#1348)

**- What I did**
Support shared headroom pool on top of dynamic buffer calculation.

Signed-off-by: Stephen Sun <[email protected]>

**- How I did it**

1. Introduce new commands for shared headroom pool configuration.
   - config buffer shared-headroom-pool over-subscribe-ratio
   - config buffer shared-headroom-pool size
   Shared headroom pool can be enabled by configuring either over-subscribe-ratio or size.
   In case both configured, the size will take effect.
   The following existing commands has been updated accordingly:
   - config buffer profile
2. Update db_migrator:
   - Port version 1.0.5 from 201911 branch
   - Port the main logic of Mellanox buffer migrator from 201911
   - Add unit test infra for Mellanox buffer migrator
3. Command "config qos clear/reload" has been updated with tables for dynamic buffer calculation cleared as well

**- How to verify it**
Run unit test and regression test.

Author: stephenxs

config/main.py

@@ -658,7 +658,9 @@ def _clear_qos():
             'BUFFER_POOL',
             'BUFFER_PROFILE',
             'BUFFER_PG',
-            'BUFFER_QUEUE']
+            'BUFFER_QUEUE',
+            'DEFAULT_LOSSLESS_BUFFER_PARAMETER',
+            'LOSSLESS_TRAFFIC_PATTERN']
 
     namespace_list = [DEFAULT_NAMESPACE]
     if multi_asic.get_num_asics() > 1:
@@ -3679,11 +3681,28 @@ def set_profile(db, profile, xon, xoff, size, dynamic_th, pool):
     update_profile(ctx, config_db, profile, xon, xoff, size, dynamic_th, pool, profile_entry)
 
 
+def _is_shared_headroom_pool_enabled(ctx, config_db):
+    ingress_lossless_pool = config_db.get_entry('BUFFER_POOL', 'ingress_lossless_pool')
+    if 'xoff' in ingress_lossless_pool:
+        return True
+
+    default_lossless_param_table = config_db.get_table('DEFAULT_LOSSLESS_BUFFER_PARAMETER')
+    if not default_lossless_param_table:
+        ctx.fail("Dynamic buffer calculation is enabled while no entry found in DEFAULT_LOSSLESS_BUFFER_PARAMETER table")
+    default_lossless_param = list(default_lossless_param_table.values())[0]
+    over_subscribe_ratio = default_lossless_param.get('over_subscribe_ratio')
+    if over_subscribe_ratio and over_subscribe_ratio != '0':
+        return True
+
+    return False
+
+
 def update_profile(ctx, config_db, profile_name, xon, xoff, size, dynamic_th, pool, profile_entry = None):
     params = {}
     if profile_entry:
         params = profile_entry
-    dynamic_calculate = True
+
+    shp_enabled = _is_shared_headroom_pool_enabled(ctx, config_db)
 
     if not pool:
         pool = 'ingress_lossless_pool'
@@ -3693,48 +3712,62 @@ def update_profile(ctx, config_db, profile_name, xon, xoff, size, dynamic_th, po
 
     if xon:
         params['xon'] = xon
-        dynamic_calculate = False
     else:
         xon = params.get('xon')
 
     if xoff:
         params['xoff'] = xoff
-        dynamic_calculate = False
     else:
         xoff = params.get('xoff')
 
     if size:
         params['size'] = size
-        dynamic_calculate = False
-        if xon and not xoff:
-            xoff = int(size) - int (xon)
-            params['xoff'] = xoff
-    elif not dynamic_calculate:
-        if xon and xoff:
-            size = int(xon) + int(xoff)
-            params['size'] = size
-        else:
-            ctx.fail("Either both xon and xoff or size should be provided")
+    else:
+        size = params.get('size')
+
+    dynamic_calculate = False if (xon or xoff or size) else True
 
     if dynamic_calculate:
         params['headroom_type'] = 'dynamic'
         if not dynamic_th:
             ctx.fail("Either size information (xon, xoff, size) or dynamic_th needs to be provided")
-
-    if dynamic_th:
         params['dynamic_th'] = dynamic_th
     else:
-        # Fetch all the keys of default_lossless_buffer_parameter table
-        # and then get the default_dynamic_th from that entry (should be only one)
-        keys = config_db.get_keys('DEFAULT_LOSSLESS_BUFFER_PARAMETER')
-        if len(keys) > 1 or len(keys) == 0:
-            ctx.fail("Multiple or no entry in DEFAULT_LOSSLESS_BUFFER_PARAMETER found while no dynamic_th specified")
-
-        default_lossless_param = config_db.get_entry('DEFAULT_LOSSLESS_BUFFER_PARAMETER', keys[0])
-        if 'default_dynamic_th' in default_lossless_param.keys():
-            params['dynamic_th'] = default_lossless_param['default_dynamic_th']
-        else:
-            ctx.fail("No dynamic_th defined in DEFAULT_LOSSLESS_BUFFER_PARAMETER")
+        if not xon:
+            ctx.fail("Xon is mandatory for non-dynamic profile")
+
+        if not xoff:
+            if shp_enabled:
+                ctx.fail("Shared headroom pool is enabled, xoff is mandatory for non-dynamic profile")
+            elif not size:
+                ctx.fail("Neither xoff nor size is provided")
+            else:
+                xoff_number = int(size) - int(xon)
+                if xoff_number <= 0:
+                    ctx.fail("The xoff must be greater than 0 while we got {} (calculated by: size {} - xon {})".format(xoff_number, size, xon))
+                params['xoff'] = str(xoff_number)
+
+        if not size:
+            if shp_enabled:
+                size = int(xon)
+            else:
+                size = int(xon) + int(xoff)
+            params['size'] = size
+
+        if dynamic_th:
+            params['dynamic_th'] = dynamic_th
+        elif not params.get('dynamic_th'):
+            # Fetch all the keys of default_lossless_buffer_parameter table
+            # and then get the default_dynamic_th from that entry (should be only one)
+            keys = config_db.get_keys('DEFAULT_LOSSLESS_BUFFER_PARAMETER')
+            if len(keys) != 1:
+                ctx.fail("Multiple entries are found in DEFAULT_LOSSLESS_BUFFER_PARAMETER while no dynamic_th specified")
+
+            default_lossless_param = config_db.get_entry('DEFAULT_LOSSLESS_BUFFER_PARAMETER', keys[0])
+            if 'default_dynamic_th' in default_lossless_param:
+                params['dynamic_th'] = default_lossless_param['default_dynamic_th']
+            else:
+                ctx.fail("No dynamic_th defined in DEFAULT_LOSSLESS_BUFFER_PARAMETER")
 
     config_db.set_entry("BUFFER_PROFILE", (profile_name), params)
 
@@ -3760,6 +3793,68 @@ def remove_profile(db, profile):
     else:
         ctx.fail("Profile {} doesn't exist".format(profile))
 
+@buffer.group(cls=clicommon.AbbreviationGroup)
+@click.pass_context
+def shared_headroom_pool(ctx):
+    """Configure buffer shared headroom pool"""
+    pass
+
+
+@shared_headroom_pool.command()
+@click.argument('ratio', metavar='<ratio>', type=int, required=True)
+@clicommon.pass_db
+def over_subscribe_ratio(db, ratio):
+    """Configure over subscribe ratio"""
+    config_db = db.cfgdb
+    ctx = click.get_current_context()
+
+    port_number = len(config_db.get_table('PORT'))
+    if ratio < 0 or ratio > port_number:
+        ctx.fail("Invalid over-subscribe-ratio value {}. It should be in range [0, {}]".format(ratio, port_number))
+
+    default_lossless_param = config_db.get_table("DEFAULT_LOSSLESS_BUFFER_PARAMETER")
+    first_item = True
+    for k, v in default_lossless_param.items():
+        if not first_item:
+            ctx.fail("More than one item in DEFAULT_LOSSLESS_BUFFER_PARAMETER table. Only the first one is updated")
+        first_item = False
+
+        if ratio == 0:
+            if "over_subscribe_ratio" in v.keys():
+                v.pop("over_subscribe_ratio")
+        else:
+            v["over_subscribe_ratio"] = ratio
+
+        config_db.set_entry("DEFAULT_LOSSLESS_BUFFER_PARAMETER", k, v)
+
+
+@shared_headroom_pool.command()
+@click.argument('size', metavar='<size>', type=int, required=True)
+@clicommon.pass_db
+def size(db, size):
+    """Configure shared headroom pool size"""
+    config_db = db.cfgdb
+    state_db = db.db
+    ctx = click.get_current_context()
+
+    _hash = 'BUFFER_MAX_PARAM_TABLE|global'
+    buffer_max_params = state_db.get_all(state_db.STATE_DB, _hash)
+    if buffer_max_params:
+        mmu_size = buffer_max_params.get('mmu_size')
+        if mmu_size and int(mmu_size) < size:
+            ctx.fail("Shared headroom pool must be less than mmu size ({})".format(mmu_size))
+
+    ingress_lossless_pool = config_db.get_entry("BUFFER_POOL", "ingress_lossless_pool")
+
+    if size == 0:
+        if "xoff" in ingress_lossless_pool:
+            ingress_lossless_pool.pop("xoff")
+    else:
+        ingress_lossless_pool["xoff"] = size
+
+    config_db.set_entry("BUFFER_POOL", "ingress_lossless_pool", ingress_lossless_pool)
+
+
 #
 # 'platform' group ('config platform ...')
 #

doc/Command-Reference.md

@@ -2363,25 +2363,53 @@ Dynamic buffer management is responsible for calculating buffer size according t
 
 ### Configuration commands
 
+**configure shared headroom pool**
+
+This command is used to configure the shared headroom pool. The shared headroom pool can be enabled in the following ways:
+
+- Configure the over subscribe ratio. In this case, the size of shared headroom pool is calculated as the accumulative xoff of all of the lossless PG divided by the over subscribe ratio.
+- Configure the size.
+
+In case both of the above parameters have been configured, the `size` will take effect. To disable shared headroom pool, configure both parameters to zero.
+
+- Usage:
+
+  ```
+  config buffer shared-headroom-pool over-subscribe-ratio <over-subscribe-ratio>
+  config buffer shared-headroom-pool size <size>
+  ```
+
+  The range of over-subscribe-ratio is from 1 to number of ports inclusive.
+
+- Example:
+
+  ```
+  admin@sonic:~$ sudo config shared-headroom-pool over-subscribe-ratio 2
+  admin@sonic:~$ sudo config shared-headroom-pool size 1024000
+  ```
+
 **configure a lossless buffer profile**
 
 This command is used to configure a lossless buffer profile.
 
 - Usage:
 
   ```
-  config buffer_profile add <profile_name> -xon <xon_threshold> -xoff <xoff_threshold> [-size <size>] [-dynamic_th <dynamic_th_value>] [-pool <ingress_lossless_pool_name>]
-  config buffer_profile set <profile_name> -xon <xon_threshold> -xoff <xoff_threshold> [-size <size>] [-dynamic_th <dynamic_th_value>] [-pool <ingress_lossless_pool_name>]
-  config buffer_profile remove <profile_name>
+  config buffer profile add <profile_name> --xon <xon_threshold> --xoff <xoff_threshold> [-size <size>] [-dynamic_th <dynamic_th_value>] [-pool <ingress_lossless_pool_name>]
+  config buffer profile set <profile_name> --xon <xon_threshold> --xoff <xoff_threshold> [-size <size>] [-dynamic_th <dynamic_th_value>] [-pool <ingress_lossless_pool_name>]
+  config buffer profile remove <profile_name>
   ```
 
   All the parameters are devided to two groups, one for headroom and one for dynamic_th. For any command at lease one group of parameters should be provided.
   For headroom parameters:
 
-  - At lease one of `xoff` and `size` should be provided and the other will be optional and conducted via the formula `xon + xoff = size`.
-  All other parameters are optional.
   - `xon` is madantory.
-  - `xon` + `xoff` <= `size`; For Mellanox platform xon + xoff == size
+  - If shared headroom pool is disabled:
+    - At lease one of `xoff` and `size` should be provided and the other will be optional and conducted via the formula `xon + xoff = size`.
+    - `xon` + `xoff` <= `size`; For Mellanox platform xon + xoff == size
+  - If shared headroom pool is enabled:
+    - `xoff` should be provided.
+    - `size` = `xoff` if it is not provided.
 
   If only headroom parameters are provided, the `dynamic_th` will be taken from `CONFIG_DB.DEFAULT_LOSSLESS_BUFFER_PARAMETER.default_dynamic_th`.
 
@@ -2397,8 +2425,8 @@ This command is used to configure a lossless buffer profile.
 - Example:
 
   ```
-  admin@sonic:~$ sudo config buffer_profile add profile1 -xon 18432 -xoff 18432
-  admin@sonic:~$ sudo config buffer_profile remove profile1
+  admin@sonic:~$ sudo config buffer profile add profile1 --xon 18432 --xoff 18432
+  admin@sonic:~$ sudo config buffer profile remove profile1
   ```
 
 **config interface cable_length**
@@ -2597,6 +2625,12 @@ This command is used to display the status of buffer pools and profiles currentl
 
   ```
   admin@sonic:~$ show buffer configuration
+  Lossless traffic pattern:
+  --------------------  -
+  default_dynamic_th    0
+  over_subscribe_ratio  0
+  --------------------  -
+
   Pool: ingress_lossless_pool
   ----  --------
   type  ingress