Add switchdev-online service to hold off on network init #56
Conversation
The addresses a race condition between the switchdev driver, udev and ifupdown2. In some cases, ifupdown2 attempts to configure switchdev interfaces (1) before they are ready, or (2) before any udev rules have had a chance to rename them per udev configuration. When this happens, not all interfaces are configured correctly. The solution is to 1. create a dummy systemd service for each switchdev interface (platform specific) that depends on the well-known systemd device path for that netdev (currently 'sys-subsystem-net-devices-FOO.device'. 2. Make the system-wide network-pre.service depend on *all* switchdev devices coming online 3. make ifupdown2 (via networking.service) depend on network-pre.service See https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/ Signed-off-by: Carl Roth <[email protected]>
Signed-off-by: Carl Roth <[email protected]>
- based on fixes for non-DN systems - please check my work, I don't have physical access to these systems to verify port count Signed-off-by: Carl Roth <[email protected]>
Signed-off-by: Carl Roth <[email protected]>
|
Two typos:
I wonder why no other ONL instances had these problems. Does this delay the startup somehow noticeably ( I guess it’s time to update to Debian 11 (bullseye) and switch to systemd-networkd soon. |
Not sure why no one noticed this yet. The general issue is that (1) the switchdev driver initializes front panel ports much more slowly that e.g. the management ethernet on the SoC and (2) ifupdown2 does not explicitly wait for devices specified in the ENI file.
I can't tell from looking at the online discussions how good the support is for mstpd in systemd-networkd. |
| # See e.g. | ||
| # https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/ | ||
| Wants=network-pre.target | ||
| Before=network-pre.target |
There was a problem hiding this comment.
I am wondering about this line, as the drop-in above has
After=network-pre.target
Isn’t that a contradiction?
/cc @tjjh89017
There was a problem hiding this comment.
@paulmenzel
Suppose you mean After=network-pre.target in networking.service.d/switchdev-online.conf conflicts with Before=network-pre.target in [email protected] (these are two different service description file)
In Rothcar's patch, he want to guarantee that networking.service will always be executed after ONLP insert ASIC driver.
he made [email protected] will always be before network-pre.target.
and networking.service (ifupdown2) will always be after network-pre.target
So you can see this order below in Rothcar's patch:
- execute [email protected] (load driver, let Linux can see those port)
- reach network-pre.target
- execute networking.service (load config. if ports isn't existing, it will fail)
and in my PR #65
try to guarantee this order below (from No. 3 to 5)
- execute [email protected] (load driver, let Linux can see those port)
- reach network-pre.target
- execute networking.service (load config. if ports and config are mismatch, it will fail)
- reach network.target
- execute other service which need network (e.g. isc-dhcp-server. if ports are not configured correctly, it will fail)
There was a problem hiding this comment.
Sorry, I totally screwed up, and in networking.service.d/switchdev-online.conf only read switchdev-online and associated it with the switchdev-online unit. But it belongs to to the unit networking.service. Sorry about that.
This is probably _not_ something we want to keep, it seems to be a workaround that may be needed only on Debian 10 (see the original PR for dentOS): Files from: https://github.com/dentproject/dentOS/tree/main/builds/any/rootfs/stretch/common/overlay/etc/systemd/system Enable swp (switch port) services based on information from: https://github.com/dentproject/dentOS/blob/main/packages/platforms/delta/arm64/tn48m-dn/tn48m-dn/platform-config/r0/src/python/arm64_delta_tn48m_dn_r0/__init__.py Original pull request in dentOS: Add switchdev-online service to hold off on network init dentproject/dentOS#56 Signed-off-by: Roger Luethi <[email protected]>
Add switchdev-online service to hold off on network init
This addresses a race condition between the switchdev driver, udev
and ifupdown2. In some cases, ifupdown2 attempts to configure
switchdev interfaces (1) before they are ready, or (2) before any
udev rules have had a chance to rename them per udev configuration.
When this happens, not all interfaces are configured correctly.
The solution is to
(platform specific) that depends on the well-known systemd
device path for that netdev (currently
'sys-subsystem-net-devices-FOO.device').
switchdev devices coming online
See
https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/