Frr kernel design utp doc

[praveen-li]

Frr kernel design and utp document. Kindly Review.

[praveen-li]

@lguohan @zhenggen-xu @jipanyang for review.

[jipanyang]

Overall the kernel route reconciliation design is awesome, we should incorporate it into sonic once it goes into FRR upstream.

As to integration with sonic, yes, there are some questions which need further clarification:

<1> Currently -r option is not enabled in either bgpd or zebra, pkill -9 is used to bring them down. Are you recommending to enable -r option?

<2> The new -K will not clear kernel route, I assume it doesn't clear route to fpmsyncd either. Is there any behavior change as to route handling toward fpmsyncd? Without -K or -k zebra option, zebra first clear route (kernel/fpm)upon startup?

<3> synchronization between kernel route reconciliation and fpmsyncd reconciliation.
New route will be available to kernel immediately, but will be held in fpmsyncd until bgp warm_restart timer expires, so ASIC dataplane won't have the route either until then. Any problem with the discrepancy?
Depending on how the kernel_reconcile timer and bgp warm_restart timer are configured, they might affect the stale route in kernel and ASIC differently.

[praveen-li]

Overall the kernel route reconciliation design is awesome, we should incorporate it into sonic once it goes into FRR upstream.

As to integration with sonic, yes, there are some questions which need further clarification:

<1> Currently -r option is not enabled in either bgpd or zebra, pkill -9 is used to bring them down. Are you recommending to enable -r option?

<2> The new -K will not clear kernel route, I assume it doesn't clear route to fpmsyncd either. Is there any behavior change as to route handling toward fpmsyncd? Without -K or -k zebra option, zebra first clear route (kernel/fpm)upon startup?

<3> synchronization between kernel route reconciliation and fpmsyncd reconciliation.
New route will be available to kernel immediately, but will be held in fpmsyncd until bgp warm_restart timer expires, so ASIC dataplane won't have the route either until then. Any problem with the discrepancy?
Depending on how the kernel_reconcile timer and bgp warm_restart timer are configured, they might affect the stale route in kernel and ASIC differently.

Hi Jipan

Thanks for reviewing this.

<1> -r is not needed in case of "pkill -9". But I think, -r is good to have in config of zebra to reflect that kernel routes will not be flushed when FRR stops, with or without pkill.

<2> With -k: Since current fpmsyncd DB reconciliation code combines Route [delete->add] as add operation. We have no problem and it seems to be working fine. This is without zebra<->kernel reconciliation code.
With -K: since zebra will send delete to fpmsyncd only if route is not learned via BGP after FRR restart, we are good here. ADD\Change operations are always sent to both kernel and FPM.
It means, we do not need FpmSyncd DB reconciliation code with -K option except Sonic system restart case.

<3> Two different timers will results in different routes in kernel and DB for some duration. BTW, I feel this problem always exists in Sonic.
I feel, similar to Neightable, If we can feed RouteTable to kernel in Sonic system restart case, then we can avoid FpmSyncd DB reconciliation code altogether, Which avoids many problems. We need little more analysis on this. Kindly let me know if you see any problem with deprecation of FpmSyncd DB reconciliation code.

Thanks again.

[jipanyang]

Replacing fpmsyncd route reconciliation logic with the one in zebra is a interesting idea, but I doubt we may be keep linux kernel route exactly the same as those in appDB due to policy config on kernel and any possible special processing in fpmsyncd for route to appDB.

[praveen-li]

Replacing fpmsyncd route reconciliation logic with the one in zebra is a interesting idea, but I doubt we may be keep linux kernel route exactly the same as those in appDB due to policy config on kernel and any possible special processing in fpmsyncd for route to appDB.

Yeah, if kernel routes are not in sync with APP_DB, it may create stale entries in APP_DB after FRR restart. In that case, It is good to keep Fpmsyncd DB reconciliation code.

[praveen-li]

@lguohan
@yxieca
Hi guys,
I request you to approve and merge this PR, since there is not pending review comment. Thanks a lot.

[praveen-li]

@pavel-shirshov
@lguohan

Hi Pavel,
I request you to approve and merge this PR, since there is not pending review comment.
Also, the code for this PR is already merged in sonic-frr :). Thanks.

[praveen-li]

closing since we are not planning to merge this HLD in sonic.