Skip to content

[pmon] xcvrd failed to start on Arista-7260 #6341

Closed
sonic-net/sonic-platform-common
#157
@bingwang-ms

Description

@bingwang-ms

Description
xcvrd failed to start on Arista-7260, which results in pmon service failed to start.

Jan  4 02:29:34.281759 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd Traceback (most recent call last):
Jan  4 02:29:34.282026 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd   File "/usr/local/bin/xcvrd", line 8, in <module>
Jan  4 02:29:34.282078 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd     sys.exit(main())
Jan  4 02:29:34.282078 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd   File "/usr/local/lib/python3.7/dist-packages/xcvrd/xcvrd.py", line 1376, in main
Jan  4 02:29:34.282091 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd     xcvrd.run()
Jan  4 02:29:34.282091 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd   File "/usr/local/lib/python3.7/dist-packages/xcvrd/xcvrd.py", line 1324, in run
Jan  4 02:29:34.282104 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd     self.init()
Jan  4 02:29:34.282157 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd   File "/usr/local/lib/python3.7/dist-packages/xcvrd/xcvrd.py", line 1289, in init
Jan  4 02:29:34.282157 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd     post_port_sfp_dom_info_to_db(is_warm_start, self.stop_event)
Jan  4 02:29:34.282196 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd   File "/usr/local/lib/python3.7/dist-packages/xcvrd/xcvrd.py", line 486, in post_port_sfp_dom_info_to_db
Jan  4 02:29:34.282196 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd     post_port_sfp_info_to_db(logical_port_name, int_tbl[asic_index], transceiver_dict, stop_event)
Jan  4 02:29:34.282196 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd   File "/usr/local/lib/python3.7/dist-packages/xcvrd/xcvrd.py", line 280, in post_port_sfp_info_to_db
Jan  4 02:29:34.282217 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd     port_info_dict = _wrapper_get_transceiver_info(physical_port)
Jan  4 02:29:34.282275 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd   File "/usr/local/lib/python3.7/dist-packages/xcvrd/xcvrd.py", line 164, in _wrapper_get_transceiver_info
Jan  4 02:29:34.282289 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd     return platform_chassis.get_sfp(physical_port).get_transceiver_info()
Jan  4 02:29:34.282363 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd   File "/usr/lib/python3/dist-packages/arista/utils/sonic_platform/sfp.py", line 110, in get_transceiver_info
Jan  4 02:29:34.282363 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd     return self._get_sfputil().get_transceiver_info_dict(self._index)
Jan  4 02:29:34.282424 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd   File "/usr/local/lib/python3.7/dist-packages/sonic_sfp/sfputilbase.py", line 912, in get_transceiver_info_dict
Jan  4 02:29:34.282424 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd     byte0 = self._read_eeprom_specific_bytes(sysfsfile_eeprom, 0, 1)[0]
Jan  4 02:29:34.282485 str-7260cx3-acs-2 INFO pmon#/supervisord: xcvrd TypeError: 'NoneType' object is not subscriptable
Jan  4 02:29:35.327279 str-7260cx3-acs-2 INFO pmon#/supervisor-proc-exit-listener: Process xcvrd exited unxepectedly. Terminating supervisor...

Steps to reproduce the issue:
1.
2.
3.

Describe the results you received:
pmon service is down because xcvrd failed to start.

Describe the results you expected:
All service should be healthy.

Additional information you deem important (e.g. issue happens only occasionally):

**Output of `show version`:**
SONiC Software Version: SONiC.HEAD.203-ba92a081
Distribution: Debian 10.7
Kernel: 4.19.0-9-2-amd64
Build commit: ba92a081
Build date: Thu Dec 31 02:13:42 UTC 2020
Built by: johnar@jenkins-worker-22

Platform: x86_64-arista_7260cx3_64
HwSKU: Arista-7260CX3-D108C8
ASIC: broadcom
ASIC Count: 1
Serial Number: SSJ17432414
Uptime: 02:37:04 up 14:27,  2 users,  load average: 4.34, 3.13, 2.63

Docker images:
REPOSITORY                    TAG                 IMAGE ID            SIZE
docker-platform-monitor       HEAD.203-ba92a081   9122a354b3f7        588MB
docker-platform-monitor       latest              9122a354b3f7        588MB
docker-syncd-brcm             HEAD.203-ba92a081   5c2940ba09c6        637MB
docker-syncd-brcm             latest              5c2940ba09c6        637MB
docker-snmp                   HEAD.203-ba92a081   58ef68401d0a        431MB
docker-snmp                   latest              58ef68401d0a        431MB
docker-teamd                  HEAD.203-ba92a081   ff8c4d7914d5        449MB
docker-teamd                  latest              ff8c4d7914d5        449MB
docker-sonic-mgmt-framework   HEAD.203-ba92a081   3c7fae0581dd        604MB
docker-sonic-mgmt-framework   latest              3c7fae0581dd        604MB
docker-router-advertiser      HEAD.203-ba92a081   83e30d5fae91        390MB
docker-router-advertiser      latest              83e30d5fae91        390MB
docker-lldp                   HEAD.203-ba92a081   c5346ab419e0        430MB
docker-lldp                   latest              c5346ab419e0        430MB
docker-dhcp-relay             HEAD.203-ba92a081   8d526a0511ab        397MB
docker-dhcp-relay             latest              8d526a0511ab        397MB
docker-database               HEAD.203-ba92a081   e39c71a1a2ac        390MB
docker-database               latest              e39c71a1a2ac        390MB
docker-orchagent              HEAD.203-ba92a081   3a2f6b7c84c7        468MB
docker-orchagent              latest              3a2f6b7c84c7        468MB
docker-nat                    HEAD.203-ba92a081   2b8555260901        452MB
docker-nat                    latest              2b8555260901        452MB
docker-sonic-telemetry        HEAD.203-ba92a081   29211dac9791        464MB
docker-sonic-telemetry        latest              29211dac9791        464MB
docker-sflow                  HEAD.203-ba92a081   4ccfc8890955        450MB
docker-sflow                  latest              4ccfc8890955        450MB
docker-fpm-frr                HEAD.203-ba92a081   cf06956fb441        467MB
docker-fpm-frr                latest              cf06956fb441        467MB
**Attach debug file `sudo generate_dump`:**

```
(paste your output here)
```

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions