Enhancement: Need a proper solution to media_settings tuning values being lost after swss restart

[arista-nwolfe]

Is it platform specific

generic

Importance or Severity

High

Description of the enhancement

Don't lose media_settings tunings on swss restart without having to restart the PMON container.

Current Behavior

XCVRD currently stores the tuning values extracted from the media_settings.json in PORT_TABLE:EthernetX in APPL_DB but on SWSS restart APPL_DB is flushed causing the tuning values to be lost and XCVRD doesn't repopulate these values.

More details in original issue: #21902

We temporarily worked around this issue by restarting PMON (container holding XCVRD) whenever SWSS is restarted on specific SKUs. Restarting PMON causes the media_settings.json to be re-parsed and APPL_DB re-populated. Temporary workaround: #22240

Proposed Behavior

But we need a proper fix to this problem.
Arvind suggested a couple possible solutions:

1. Move the tuning values from APPL_DB to STATE_DB
2. Have XCVRD subscribe to PORT_TABLE in APPL_DB and re-parse media_settings.json whenever a new entry is added.

[judyjoseph]

@arista-nwolfe to Look into how to prevent pmon/xcvrd from being restarted ( as we plan to do today via #22240)