Skip to content

Commit bc7b409

Browse files
authored
Merge pull request #924 from ds952811/autoneg
[autoneg] add support for remote speed advertisement and clarify the expected autoneg behaviors
2 parents c5fd06a + 1cfa504 commit bc7b409

File tree

2 files changed

+60
-27
lines changed

2 files changed

+60
-27
lines changed

doc/port_auto_neg/flow_rmt_adv.png

23.8 KB
Loading

doc/port_auto_neg/port-auto-negotiation-design.md

+60-27
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@
2626
- [SWSS Enhancements](#swss-enhancements)
2727
- [portsyncd and portmgrd Consideration](#portsyncd-and-portmgrd-consideration)
2828
- [Port Breakout Consideration](#port-breakout-consideration)
29-
- [xcvrd Consideration](#xcvrd-consideration)
29+
- [PMON xcvrd Consideration](#pmon-xcvrd-consideration)
3030
- [Warmboot and Fastboot Design Impact](#warmboot-and-fastboot-design-impact)
3131
- [Restrictions/Limitations](#restrictions/limitations)
3232
- [Testing Requirements/Design](#testing-requirements/design)
@@ -36,10 +36,11 @@
3636

3737
### Revision
3838

39-
| Rev | Date | Author | Change Description |
40-
|:---:|:-----------:|:------------------:|-----------------------------------|
41-
| 0.1 | | Junchao Chen | Initial version |
42-
| 0.2 | | Junchao Chen | Fix review comment |
39+
| Rev | Date | Author | Change Description |
40+
|:---:|:-----------:|:-------------------:|--------------------------------------------|
41+
| 0.1 | | Junchao Chen | Initial version |
42+
| 0.2 | | Junchao Chen | Fix review comment |
43+
| 0.3 | | Dante (Kuo-Jung) Su | Add support for SFPs and operational states|
4344

4445
### Scope
4546
This document is the design document for port auto negotiation feature on SONiC. This includes the requirements, CLI change, DB schema change, DB migrator change, yang model change and swss change.
@@ -51,7 +52,12 @@ N/A
5152

5253
The IEEE 802.3 standard defines a set of Ethernet protocols that are comprised of speed rate and interface type. It allows for configuring multiple values at the same time for port provisioning and advertising to the remote side. However, on SONiC, user can configure the speed of port, and user can configure auto negotiation mode via config DB. Port attributes such as interface type, advertised speeds, advertised interface types are not supported.
5354

54-
The feature in this document is to address the above issues.
55+
The feature in this document is to address the above issues, and support the following medium typs
56+
57+
- Twisted-pair cables (Non-Gearbox design)
58+
- SFP/QSFP/QSFPDD CR/KR transceivers (Non-Gearbox design)
59+
60+
The complexities behind vendor-specific SAI implementation is outside the scope of this document.
5561

5662
### Requirements
5763

@@ -86,6 +92,11 @@ Currently, SAI already defines a few port attributes to support port auto negoti
8692
3. If autoneg is enabled and adv_speeds is not configured or empty, SAI must advertise it with all supported speeds.
8793
4. If autoneg is enabled and adv_interface_types is not configured or empty, SAI must advertise it with all supported interface types.
8894
5. If autoneg is disabled and interface_type is not configured, SAI must use SAI_PORT_INTERFACE_TYPE_NONE.
95+
6. If autoneg is enabled, the administrative port speed updates should not disable the autoneg. The configured speed should be cached in swss#orchagent and gets replayed when autoneg is transitioned from enabled to disabled.
96+
7. If autoneg is enabled, while the administrative interface type updates via CONFIG_DB should be blocked, the dynamic interface type updates from pmon#xcvrd via APPL_DB should be delivered to the SAI to update the autoneg advertisement. Please refer to [PMON xcvrd Consideration](#pmon-xcvrd-consideration) for details.
97+
8. If autoneg is enabled on a SFP/QSFP/QSFPDD port, SAI should also activate the link-training to dynamically tune the TX FIR.
98+
9. If autoneg is enabled on a SFP/QSFP/QSFPDD port, when it's transitioned to disabled, the port speed, FEC, interface type and TX FIR should be restored with the corresponding values in the APPL_DB. If the individual configuration is not present in the APPL_DB, the SAI driver defaults should be restored. Precisely speaking, default interface type is SAI_PORT_INTERFACE_TYPE_NONE, default fec mode is SAI_PORT_FEC_MODE_NONE and TX FIR is vendor-specific, as it's NOT a set of constants applicable to all switch ASIC.
99+
10. If autoneg is enabled on a SFP/QSFP/QSFPDD port, the "fec" field of APPL_DB should override the result of autoneg.
89100

90101
The related port attributes are listed below:
91102
```cpp
@@ -234,26 +245,29 @@ This command always replace the advertised interface types instead of append. Fo
234245

235246
##### Show interfaces auto negotiation status
236247

237-
As command `show interfaces status` already has 11 columns, a new CLI command will be added to display the port auto negotiation status. All data of this command are fetched from **APPL_DB**.
248+
As command `show interfaces status` already has 11 columns, a new CLI command will be added to display the port auto negotiation status. All data of this command are fetched from **APPL_DB** and **STATE_DB**.
238249

239250
```
240251
Format:
241-
show interfaces autoneg-status <interface_name>
252+
show interfaces autoneg status <interface_name>
242253
243254
Arguments:
244255
interface_name: optional. Name of the interface to be shown. e.g: Ethernet0. If interface_name is not given, this command shows auto negotiation status for all interfaces.
245256
246257
Example:
247-
show interfaces autoneg-status
248-
show interfaces autoneg-status Ethernet0
258+
show interfaces autoneg status
259+
show interfaces autoneg status Ethernet0
249260
250261
Return:
251262
error message if interface_name is invalid otherwise:
252263
253-
Interface Auto-Neg Mode Speed Adv Speeds Type Adv Types Oper Admin
254-
----------- --------------- ------- ------------ ------ ----------- ------ -------
255-
Ethernet0 disabled 25G N/A N/A N/A down up
256-
Ethernet32 enabled 40G 40G,100G KR4 KR4,CR4 up up
264+
Interface Auto-Neg Mode Speed Adv Speeds Rmt Adv Speeds Type Adv Types Oper Admin
265+
----------- --------------- ------- ------------ ---------------- ------ ----------- ------ -------
266+
Ethernet0 enabled 400G N/A 400G CR4 CR4 up up
267+
Ethernet8 enabled 400G N/A N/A KR4 KR4 down up
268+
Ethernet16 enabled 400G N/A 400G CR4 CR4 up up
269+
Ethernet24 N/A 400G N/A N/A N/A N/A down up
270+
Ethernet32 disabled 400G N/A N/A N/A N/A down up
257271
```
258272

259273
##### CLI validation
@@ -379,7 +393,7 @@ The change in APP_DB is similar to CONFIG_DB. 3 new fields **adv_speeds**, **int
379393
Valid value of the new fields are the same as **PORT** table in CONFIG_DB.
380394

381395
Here is the table to map the fields and SAI attributes:
382-
| **Parameter** | **sai_port_attr_t**
396+
| **Parameter** | **sai_port_attr_t** |
383397
|---------------------|------------------------------------------------|
384398
| adv_interface_types | SAI_PORT_ATTR_ADVERTISED_INTERFACE_TYPE |
385399
| adv_speeds | SAI_PORT_ATTR_ADVERTISED_SPEED |
@@ -390,15 +404,19 @@ Here is the table to map the fields and SAI attributes:
390404
#### State DB Enhancements
391405

392406
To support validate interface speed on CLI side, a new field **supported_speeds** will be added to **PORT_TABLE**.
407+
To support checking remote advertised speeds, a new field **rmt_adv_speeds** will be added to **PORT_TABLE**.
393408

394409
; Defines information for port state
395410
key = PORT_TABLE:port_name ; state of the port
396411
; field = value
397412
...
398413
supported_speeds = STRING ; supported speed list
399-
speed = STRING ; operational speed
414+
speed = STRING ; operational speed
415+
rmt_adv_speeds = STRING ; advertised speed list of the remote
400416

401-
An example value of supported_speeds could be "10000,25000,40000,100000".
417+
- supported_speeds: A list of supported speeds separated by commas, for example, "10000,25000,40000,100000".
418+
- speed: Operational speed, for example, "100000".
419+
- rmt_adv_speeds: A list of supported speeds advertised by the connected remote, for example, "10000,25000,40000,100000".
402420

403421
Before this feature, port speed in APP DB indicates both the configured speed and the operational speed. It is OK without this feature because port operational speed must be configured speed or port operational status is down. However, this is not true with this feature. Consider following flow:
404422

@@ -413,14 +431,21 @@ To overcome this issue, following changes are required:
413431
1. Put port operational speed to STATE DB PORT_TABLE
414432
2. intfutil, portstat, voqutil shall be change to get port operational speed from STATE DB first. For backward compatible, intfutil, portstat, voqutil shall still get port operational speed from APP DB if port speed is not available in STATE DB or port operational state is down.
415433

434+
Here is the table to map the fields and SAI attributes:
435+
| **Parameter** | **sai_port_attr_t** |
436+
|---------------------|------------------------------------------------|
437+
| rmt_adv_speeds | SAI_PORT_ATTR_REMOTE_ADVERTISED_SPEED |
438+
| speed | SAI_PORT_ATTR_SPEED |
439+
| supported_speeds | SAI_PORT_ATTR_SUPPORTED_SPEED |
440+
416441
#### YANG Model Enhancements
417442

418443
The port yang model needs to update according to DB schema change. The yang model changes of new fields are described below:
419444

420445
```
421446
leaf autoneg {
422447
type string {
423-
pattern "0|1";
448+
pattern "off"|"on";
424449
}
425450
}
426451
@@ -470,6 +495,8 @@ Will be migrated to:
470495

471496
#### SWSS Enhancements
472497

498+
##### Setting Auto Negotiation
499+
473500
The current SONiC speed setting flow in PortsOrch can be described in following pseudo code:
474501

475502
```
@@ -511,6 +538,14 @@ else if autoneg == false:
511538
setInterfaceType(port, interface_type)
512539
```
513540

541+
##### Getting Remote Advertisement
542+
543+
A new periodic timer task will be introduced into PortsOrch, it periodically loops through physical ports and update the per-port remote advertisement if autoneg is enabled and the link is down.
544+
545+
![](flow_rmt_adv.png)
546+
547+
##### Backward Compatibility Considerations
548+
514549
SONiC usually does not call SAI interface when there is no related configuration in APPL_DB. In order to keep backward compatible, this feature also apply this rule.
515550

516551
swss will do validation for auto negotiation related fields, although it still CANNOT guarantee that all parameters passed to SAI will be accepted by SAI. swss validation for these field are described below:
@@ -520,6 +555,7 @@ swss will do validation for auto negotiation related fields, although it still C
520555
3. interface_type value from APPL_DB must be able to transfer to a valid interface type value. For invalid value, swss must catch the exception, log the error value and skip the rest configuration of this port.
521556
4. adv_interface_types value from APPL_DB must be able to transfer to a list of valid interface type values. For invalid value, swss must catch the exception, log the error value and skip the rest configuration of this port.
522557

558+
523559
#### portsyncd and portmgrd Consideration
524560

525561
No changes for portsyncd and portmgrd.
@@ -617,18 +653,15 @@ I choose this solution because:
617653

618654
Dynamic port breakout feature also introduces a hwsku.json file to describe the port capability. It defines the default dynamic breakout mode for now. As we won't automatically set auto negotiation attributes for a port after port breakout, it is not necessary to change the hwsku.json in this feature.
619655

620-
#### xcvrd Consideration
621-
622-
It is recommended to use CLI/CONFIG_DB to setting the port auto negotiation attributes. But there is still other way.
623-
624-
There is a media_setting.json which is used for setting the default value for some port attributes. This media_setting.json is handled by xcvrd for setting pre-emphasis value for ports. xcvrd process the file this way:
656+
#### PMON xcvrd Consideration
625657

626-
1. When xcvrd start, it reads the media_setting.json and set pre-emphasis values to APPL_DB for each port
627-
2. When a new cable is inserted, xcvrd uses the value in media_setting.json to set pre-emphasis value to APPL_DB for this port.
658+
While it is possible to use CLI/CONFIG_DB for setting the port auto negotiation attributes, this feature is also available via [media_settings.json](https://github.com/Azure/SONiC/blob/master/doc/media-settings/Media-based-Port-settings.md)
628659

629-
xcvrd just reads the configuration from media_settings.json and set value to APPL_DB. xcvrd does not care what the configuration is. So user could use media_settings.json to specify the default value for autoneg, adv_speeds and other port attributes. **Nothing needs to be changed in xcvrd**.
660+
- **Interface Type**
661+
Unfortunately, it's not straightforward for users to identify which interface type is most appropriate to the attached transceivers, and the link will not get up unless the connected devices are both advertising the same interface type. This requires domain knowledge, correct EERPOM information and the individual hardware datasheet review process. Hence it's recommended to use [media_settings.json](https://github.com/Azure/SONiC/blob/master/doc/media-settings/Media-based-Port-settings.md) to automate the while process. A interface type update request from [media_settings.json](https://github.com/Azure/SONiC/blob/master/doc/media-settings/Media-based-Port-settings.md) is triggered by the transceiver detection of pmon#xcvrd, which leverages **APPL_DB** instead of **CONFIG_DB**, hence the requests will not be blocked by **portsyncd**. If the interface type update request arrives when autoneg is enabled, it should alter the advertisement and restart the autoneg. That said, if the user has interface type configured in both CONFIG_DB and media_settings.json, it wouldn't be possible to predict which would take precedence as there is no logic giving one priority over other. Hence please be sure to use either CLI/CONFIG_DB or media_settings.json at a time, never have both of them activated.
630662

631-
However, it is worthy mentioning that: if user have port attributes configured both in CONFIG_DB and media_settings.json, the value in media_settings.json will override the value in CONFIG_DB after rebooting, restarting pmon or re-insert cables. Base on that, if user choose to use media_settings.json, they probably should not use CLI or CONFIG_DB to avoid configuration loss after rebooting, restarting pmon or re-insert cables.
663+
- **Pre-Emphasis**
664+
Typically, prior to some process, such as transmission over cable, or recording to phonograph record or tape, the input frequency range most susceptible to noise is boosted. This is referred to as "pre-emphasis" - before the process the signal will undergo. While this is rarely necessay to the native RJ45 ports, it's important to SFP/QSFP/QSFPDD ports. For optical arrangements (e.g. SR/LR/DR transceivers), the loss is miniscule. The loss budget allocated for the host PCB board is small, which implies the pre-emphasis calibrated is supposed to work regardless of the fiber cable attached. On the other hand, on passive media, there is a significant amount of frequency dependent loss and the channel loss can range up to the CR/KR loss spec. It is therefore important and useful to activate Link-Training to "train" the TX FIR in both directions to help equalize that loss. IEEE 802.3 Clause 72, Clause 73 and Clause 93 define the auto-negotiation and link-training support for CR/KR transceivers, while the support of opticals (e.g. SR/LR/DR) are never in the scope. On the other hand, when autoneg is enabled, the link-training will also be activated, and the link-training handshake will only get started after negotiation if autoneg is enabled. Furtherly, link-training is to dynamically tune hardware signals at runtime, as a result, the static pre-emphasis parameters in media_setting.json is not necessary and will not be taken into autoneg process.
632665

633666
### Warmboot and Fastboot Design Impact
634667

0 commit comments

Comments
 (0)