Skip to content

[chassis] Configure midplane network on supervisor #11725

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 16, 2022

Conversation

mlorrillere
Copy link
Contributor

Why I did it

Multi-asic Docker instances are created behind Docker's default bridge
which doesn't allow talking to other Docker instances that are in the
host network (like database-chassis).

On linecards, we configure midplane interfaces to let per-asic docker
containers talk to CHASSIS_DB on the supervisor through internal chassis
network.

On the supervisor we don't need to use chassis internal network, but we
still need a similar setup in order to allow fabric containers to talk
to database-chassis

How I did it

Reusing existing code used to setup the midplane network on a linecard,
except that we only need a pair of veth interfaces on the supervisor to allow
per-asic docker instances access to the midplane subnet.

@mlorrillere mlorrillere requested a review from lguohan as a code owner August 12, 2022 18:13
@mlorrillere
Copy link
Contributor Author

@abdosi
Copy link
Contributor

abdosi commented Aug 16, 2022

wondering this change can be be handle in platform code ? Not all vendors/platform have br1 define on supervisor.

Similar to creation of eth1-midplane is platform/vendor specific.

@mlorrillere
Copy link
Contributor Author

wondering this change can be be handle in platform code ? Not all vendors/platform have br1 define on supervisor.

Similar to creation of eth1-midplane is platform/vendor specific.

I don't think we can easily move that change in platform code because all namespaces are created and destroyed here, that's the reason the entire logic was moved from interface-config to database.sh in #10500

Here we assume br1 exists (same as eth1-midplane). If there is no consensus on the name of the bridge we can add that in chassisdb.conf like we do for midplane_subnet.

@mlorrillere mlorrillere force-pushed the pr-configure-midplane-sup branch from bfb2bc0 to be4e77e Compare August 22, 2022 21:13
Multi-asic Docker instances are created behind Docker's default bridge
which doesn't allow talking to other Docker instances that are in the
host network (like database-chassis).

On linecards, we configure midplane interfaces to let per-asic docker
containers talk to CHASSIS_DB on the supervisor through internal chassis
network.

On the supervisor we don't need to use chassis internal network, but we
still need a similar setup in order to allow fabric containers to talk
to database-chassis
@mlorrillere mlorrillere force-pushed the pr-configure-midplane-sup branch from be4e77e to 38a82d7 Compare August 23, 2022 16:32
@arlakshm
Copy link
Contributor

wondering this change can be be handle in platform code ? Not all vendors/platform have br1 define on supervisor.
Similar to creation of eth1-midplane is platform/vendor specific.

I don't think we can easily move that change in platform code because all namespaces are created and destroyed here, that's the reason the entire logic was moved from interface-config to database.sh in #10500

Here we assume br1 exists (same as eth1-midplane). If there is no consensus on the name of the bridge we can add that in chassisdb.conf like we do for midplane_subnet.

Hi @mlorrillere, not all platform need a software bridge to communicate between LC and Sup. So they dont need to create br1. On platforms which don't create the bridge we create an eth1-midplane interface. Wondering on the platfrom which need a software bridge we care create a bridge with name eth1-midplane ?

@mlorrillere
Copy link
Contributor Author

wondering this change can be be handle in platform code ? Not all vendors/platform have br1 define on supervisor.
Similar to creation of eth1-midplane is platform/vendor specific.

I don't think we can easily move that change in platform code because all namespaces are created and destroyed here, that's the reason the entire logic was moved from interface-config to database.sh in #10500
Here we assume br1 exists (same as eth1-midplane). If there is no consensus on the name of the bridge we can add that in chassisdb.conf like we do for midplane_subnet.

Hi @mlorrillere, not all platform need a software bridge to communicate between LC and Sup. So they dont need to create br1. On platforms which don't create the bridge we create an eth1-midplane interface. Wondering on the platfrom which need a software bridge we care create a bridge with name eth1-midplane ?

eth1-midplane is the name of the interface on the linecard side. On the sup side the endpoint of the eth1-midplane of all linecards is added to br1.

In practice this PR is already a noop on platforms that don't have br1 because this code is conditional on midplane_subnet being defined in chassisdb.conf of the sup (which is usually not defined, hence why it is added to device/arista/x86_64-arista_7800_sup/chassisdb.conf). Since this is not obvious, an alternative could be to add a midplane_bridge=br1 to chassisdb.conf, that way we could clearly see which part of the code is run on the sup/linecard.

@mlorrillere
Copy link
Contributor Author

As discussed offline, we will keep br1 in this PR as changing the name of the bridge requires updating Arista's platform modules. We will update Arista platform modules to create the bridge as eth1-midplane and update docker_image_ctl.j2 when we bump-up sonic-platform-modules-arista.

@mlorrillere
Copy link
Contributor Author

Hi @arlakshm ,
I believe we will need to port this PR to 202205 branch.

@arlakshm arlakshm merged commit 0a7dd50 into sonic-net:master Sep 16, 2022
yxieca pushed a commit that referenced this pull request Sep 21, 2022
Multi-asic Docker instances are created behind Docker's default bridge
which doesn't allow talking to other Docker instances that are in the
host network (like database-chassis).

On linecards, we configure midplane interfaces to let per-asic docker
containers talk to CHASSIS_DB on the supervisor through internal chassis
network.

On the supervisor we don't need to use chassis internal network, but we
still need a similar setup in order to allow fabric containers to talk
to database-chassis
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants