Command "show arp" returns error: "int() argument must be a string, a bytes-like object or a number, not 'NoneType'"

[ppikh]

Command "show arp" returns error: "int() argument must be a string, a bytes-like object or a number, not 'NoneType'"

Setup
SONiC switch with Linux host connected to port Ethernet68
Linux host with interface enp5s0f1 connected to SONiC port Ethernet68
SONiC ----> Ethernet68 ----> enp5s0f1 ----> Linux Host

Steps to reproduce the issue:

1. Do configuration on DUT
   sudo config portchannel add PortChannel0002
   sudo config portchannel member add PortChannel0002 Ethernet68
   sudo config vlan add 40
   sudo config vlan member add 40 PortChannel0002
   sudo config interface ip add Vlan40 40.0.0.1/24
2. Do configuration on Linux host
   sudo ip link add bond0 type bond
   sudo ip link set dev bond0 type bond mode 4
   sudo ip link set enp5s0f1 down
   sudo ip link set enp5s0f1 master bond0
   sudo ip link set enp5s0f1 up
   sudo ip link set bond0 up
   sudo ip link add link bond0 name bond0.40 type vlan id 40
   sudo ip link set bond0.40 up
   sudo ip addr add 40.0.0.3/24 dev bond0.40
3. Do ping from linux host to DUT IP 40.0.0.1
4. Do command "show arp" on DUT

Describe the results you received:
Command "show arp" returned error.

root@r-boxer-sw01:/home/admin# show arp
int() argument must be a string, a bytes-like object or a number, not 'NoneType'

Describe the results you expected:
Command "show arp" returned correct output

Additional information you deem important (e.g. issue happens only occasionally):
Issue reproduce 100%

**Output of `show version`:**

SONiC Software Version: SONiC.SONIC.master.50-b2c147f_Internal
Distribution: Debian 10.7
Kernel: 4.19.0-9-2-amd64
Build commit: b2c147fe
Build date: Tue Jan  5 16:36:57 UTC 2021
Built by: sw-r2d2-bot@r-build-sonic-ci02

Platform: x86_64-mlnx_msn2010-r0
HwSKU: ACS-MSN2010
ASIC: mellanox
ASIC Count: 1
Serial Number: MT1749X10061
Uptime: 09:56:20 up 25 min,  1 user,  load average: 5.63, 4.98, 3.77

Docker images:
REPOSITORY                    TAG                                IMAGE ID            SIZE
docker-syncd-mlnx             SONIC.master.50-b2c147f_Internal   1746ce01a44b        535MB
docker-syncd-mlnx             latest                             1746ce01a44b        535MB
docker-snmp                   SONIC.master.50-b2c147f_Internal   d31077be9ee9        431MB
docker-snmp                   latest                             d31077be9ee9        431MB
docker-sonic-mgmt-framework   SONIC.master.50-b2c147f_Internal   f05285cba6ab        604MB
docker-sonic-mgmt-framework   latest                             f05285cba6ab        604MB
docker-nat                    SONIC.master.50-b2c147f_Internal   129e73e1c2b0        453MB
docker-nat                    latest                             129e73e1c2b0        453MB
docker-router-advertiser      SONIC.master.50-b2c147f_Internal   4440f0d178d7        390MB
docker-router-advertiser      latest                             4440f0d178d7        390MB
docker-platform-monitor       SONIC.master.50-b2c147f_Internal   51934d4206b9        671MB
docker-platform-monitor       latest                             51934d4206b9        671MB
docker-lldp                   SONIC.master.50-b2c147f_Internal   7c05e0e4de67        430MB
docker-lldp                   latest                             7c05e0e4de67        430MB
docker-database               SONIC.master.50-b2c147f_Internal   2a3c0268c847        390MB
docker-database               latest                             2a3c0268c847        390MB
docker-orchagent              SONIC.master.50-b2c147f_Internal   0be64466ff32        468MB
docker-orchagent              latest                             0be64466ff32        468MB
docker-dhcp-relay             SONIC.master.50-b2c147f_Internal   b7818d179c3d        397MB
docker-dhcp-relay             latest                             b7818d179c3d        397MB
docker-sonic-telemetry        SONIC.master.50-b2c147f_Internal   c7dbee664703        465MB
docker-sonic-telemetry        latest                             c7dbee664703        465MB
docker-teamd                  SONIC.master.50-b2c147f_Internal   9ca15386759c        450MB
docker-teamd                  latest                             9ca15386759c        450MB
docker-fpm-frr                SONIC.master.50-b2c147f_Internal   fb9df1dca12c        468MB
docker-fpm-frr                latest                             fb9df1dca12c        468MB
docker-sflow                  SONIC.master.50-b2c147f_Internal   6e20ee32bc5a        451MB
docker-sflow                  latest                             6e20ee32bc5a        451MB

Tech support dump here:
sonic_dump_r-boxer-sw01_20210106_095423.tar.gz

[ghost]

Dear @ppikh,

I have analyzed the attached dump and found a root cause of the issue.
ASIC_DB.json inside the dump contains the next FDB entry record:

  "ASIC_STATE:SAI_OBJECT_TYPE_FDB_ENTRY:{\"bvid\":\"oid:0x26000000000013\",\"mac\":\"0C:42:A1:B4:D7:E9\",\"switch_id\":\"oid:0x21000000000000\"}": {
    "expireat": 1609926906.1291392, 
    "ttl": -0.001, 
    "type": "hash", 
    "value": {
      "SAI_FDB_ENTRY_ATTR_BRIDGE_PORT_ID": "oid:0x3a0000000005dc", 
      "SAI_FDB_ENTRY_ATTR_PACKET_ACTION": "SAI_PACKET_ACTION_FORWARD", 
      "SAI_FDB_ENTRY_ATTR_TYPE": "SAI_FDB_ENTRY_TYPE_DYNAMIC"
    }
  }, 

But, if my understanding is right, bvid "oid:0x26000000000013" belongs to default VLAN 1, which does not present in the system. The next record has not "SAI_VLAN_ATTR_VLAN_ID":

  "ASIC_STATE:SAI_OBJECT_TYPE_VLAN:oid:0x26000000000013": {
    "expireat": 1609926906.088011, 
    "ttl": -0.001, 
    "type": "hash", 
    "value": {
      "NULL": "NULL"
    }
  }, 

This causes an exception, output of which you could see in CLI.

I have made PR, which contains a probable solution. Could I ask you to review it?
sonic-net/sonic-utilities#1357

In overview of the PR you could see the way, in which the issue has reproduced for me. If I use just yours steps, command "show arp" works fine, because the additional FDB entry with default VLAN is not being created.
I think some mistakes inside system configuration may lead to the issue too. Could you check your system configuration in "config_db.json" or somewhere else?

In addition, I have found a similar issue, but with "show mac":
sonic-net/sonic-utilities#894

If PR 1357 will be accepted, I think the issue 894 could be fixed in the same way.

[ghost]

Dear @ppikh,

I have investigated the system behavior more deeper and realized that the system looks like works as designed. The FDB entry with default VLAN 1 is a normal reaction on untagged traffic, so, it is ok that such entry presents in the system.
As result, I think sonic-net/sonic-utilities#1357 is a solution for the issue.