host-device: copy host interface IP addresses and routes into container by SchSeba · Pull Request #1257 · containernetworking/plugins

SchSeba · 2026-05-04T14:29:25Z

Add a new configuration option useInterfaceNetwork that instructs the host-device plugin to capture the interface's IP addresses and routes from the host before moving the device into the container namespace, and then apply them inside the container.

This is critical for virtual environments (AWS, IBM Cloud, GPC) where the cloud provider configures IP addresses and routes directly on the network device. In these environments, there is no traditional IPAM source; the ground truth for L3 configuration lives on the host interface itself.

When useInterfaceNetwork is enabled, the plugin:

Captures all global-scope addresses and non-local routes from the host device before moving it into the container namespace.
Applies the captured addresses and routes to the interface inside the container.
Reports the addresses and routes in the CNI result (merged with any IPAM result if an IPAM plugin is also configured).

NOTE: The interface configuration on the host node must be persistent. When the device is moved back to the host (via DEL) and renamed to its original name, the system's network management service (e.g. NetworkManager, systemd-networkd, cloud-init, or cloud-specific agents) is expected to detect the device and re-apply the IP addresses and routes. This plugin does NOT re-configure the host interface on DEL; it relies on the node's network configuration being declarative and reconciled by the platform's networking stack.

Also implements the STATUS command to verify the host device exists, replacing the previous TODO stub.

SchSeba · 2026-05-04T18:47:35Z

Hi @s1061123 @squeed @LionelJouin if you have time please take a look on the PR.
This is critical for us to support virtual clusters running on clouds where the VFs are pass into the cluster VMs nodes with network configuration.

karampok · 2026-05-05T08:15:33Z

+	localRouteTable = 255
+)
+
+// HostNetworkStateFile holds the captured host-side L3 configuration


personal preference no comment unless exported

Done - removed doc comments from all unexported symbols.

karampok · 2026-05-05T08:15:33Z

+
+// HostNetworkStateFile holds the captured host-side L3 configuration
+// (addresses, routes, and rules) that should be applied to the container interface.
+type HostNetworkStateFile struct {


prefix ^File$ not very nice not really a file. May InterfaceInfo, InterfaceConfig, or just Interface

Good call - renamed HostNetworkStateFile to HostNetworkState throughout.

karampok · 2026-05-05T08:15:33Z

+type HostNetworkStateFile struct {
+	HostIfName    string       `json:"hostIfName"`
+	HostLinkWasUp bool         `json:"hostLinkWasUp"`
+	Addresses     []string     `json:"addresses,omitempty"`


can we use netlink.Addr, netlink.routes, rule

The string-based representation is intentional here - netlink.Addr, netlink.Route and netlink.Rule contain net.IP / *net.IPNet fields that don't round-trip cleanly through JSON (net.IP marshals as a base64 byte array, *net.IPNet isn't directly marshalable). Using strings gives us portable, human-readable JSON and avoids coupling the serialization format to the netlink library's internal types.

We do convert back to netlink types when applying (applyOnLink), so the actual netlink interaction is the same.

karampok · 2026-05-05T08:15:33Z

+
+// applyNetworkStateToPod applies captured state to the moved interface inside the pod namespace.
+func applyNetworkStateToPod(containerNs ns.NetNS, contDev netlink.Link, state *HostNetworkStateFile) error {
+	if state == nil {


having that check, (imo) indicates that this function maybe should be method to the struct.

Agreed - converted applyNetworkStateToPod and applyNetworkStateOnLink to methods on *HostNetworkState (applyToPod and applyOnLink). The nil-receiver check now reads more naturally.

karampok · 2026-05-05T08:15:33Z

+			},
+		},
+	}
+	mergeNetworkStateIntoResult(result, state)


imo if user decides to keep network config from the host then we should ignore IPAM or block the combination witt IPAM in the config. I think is either IPAM, or host network config (no ip is also valid config)

I see the point, but there are valid use cases for the combination: the host interface provides the base L3 config (addresses/routes from the cloud provider), and IPAM adds additional addresses on top (e.g. secondary IPs, service IPs). Blocking the combination would reduce flexibility for users who need both.

The current merge behavior is additive - IPAM addresses are appended alongside host-captured ones. If you feel strongly, we could add a validation warning instead of an error, but I'd prefer to keep the flexibility. WDYT?

What about conflicting default routes? IPs on the same prefix?

Good question. Both cases are handled gracefully:

Conflicting IPs on the same prefix: applyOnLink wraps every AddrAdd / RouteAdd call with isAlreadyExistsErr - if the host-captured address or route already exists (because IPAM configured the same one), the duplicate is silently ignored.

Conflicting default routes: Linux allows multiple default routes with different metrics (ECMP). If both the host config and IPAM supply a default route with the same gateway+metric, the second RouteAdd returns EEXIST which we handle. If they differ in metric, both coexist as expected.

The order of operations is: host addresses/routes are applied first (in applyToPod), then IPAM runs and calls ConfigureIface. So IPAM is always the "last writer" for identical entries, and duplicates are harmlessly skipped.

In practice, the combination is intended for cases like: host provides the primary IP/routes from the cloud provider, and IPAM adds secondary IPs or service-specific routes on top - they typically won't conflict.

karampok · 2026-05-05T08:15:34Z

+)
+
+// TestUseInterfaceNetwork verifies useInterfaceNetwork boolean behavior.
+func TestUseInterfaceNetwork(t *testing.T) {


what exactly this test is doing?

Removed this test - it was only validating the trivial boolean guard function which is already covered implicitly by the integration tests.

karampok · 2026-05-05T08:15:34Z

+}
+
+// TestStateJSONHasNoNeighbors verifies state serialization excludes neighbors.
+func TestStateJSONHasNoNeighbors(t *testing.T) {


what exactly this test is doing? Not sure the tests in the file really add value

Removed this test as well. Kept the TestMergeNetworkState* and TestLoadConf* tests since those exercise actual logic (result merging, config parsing, DPDK rejection).

s1061123

This PR introduces new parameter, 'useInterfaceNetwork', so could you please create another PR in https://github.com/containernetworking/cni.dev/pulls to modify host-device CNI document as well?

s1061123 · 2026-05-11T13:42:02Z

 	RuntimeConfig struct {
 		DeviceID string `json:"deviceID,omitempty"`
 	} `json:"runtimeConfig,omitempty"`
+	UseInterfaceNetwork bool `json:"useInterfaceNetwork,omitempty"`


Recommend to add comment to quickly mention what is 'UseInterfaceNetwork' because the option name is not intuitive (what 'useInterfaceNetwork' is used?)

Done - added an inline comment: // When true, copy the host interface's IP addresses and routes into the container before IPAM runs.

I also open the PR for the documentation site containernetworking/cni.dev#156

karampok · 2026-05-12T12:42:26Z

I am a bit unsure if we capture everything that need to be re-applied or if capture something we should not.
For example, when IPv6, when SLAAC, routes from RA, nothing should be copied imo. (IPv6 test are missing)

Some AI generated list which seems possible

1. Neighbours not captured — GCP assigns /32 to NICs; all egress needs ARP entry for gateway; LinkSetNsFd wipes neighbour table; traffic blackholes.
  proof
  2. rp_filter not captured — AWS multi-ENI requires loose (2); container gets netns default strict (1); asymmetric return traffic silently dropped.
  proof
  3. RTPROT_RA routes copied — RA routes have expiry (expires Nsec); re-applied without lifetime; no RA daemon in container to refresh; stale routes
  persist forever. proof
  4. SLAAC addresses copied without IFA_F_PERMANENT filter — dynamic addrs (ip addr show → dynamic) become permanent in container; no renewal; never
  expire. [proof](/usr/include/linux/if_addr.h:54 — IFA_F_PERMANENT 0x80]

SchSeba · 2026-06-24T14:11:56Z

Thanks for the thorough review @karampok - addressing your IPv6/SLAAC concerns:

Addressed in the latest push:

RTPROT_RA routes — now filtered out in captureHostNetworkState. RA-learned routes carry kernel-managed lifetimes and there's no RA daemon inside the container to refresh them, so copying them would create stale permanent routes.
SLAAC addresses — now filtered via IFA_F_PERMANENT flag check. Only permanently configured addresses are captured; dynamic (SLAAC/DHCPv6) addresses are skipped since they won't be renewed inside the container.

Out of scope for this PR (can be follow-ups):

Neighbours — On GCP with /32 NICs, the gateway ARP entry is indeed wiped when the device moves namespaces. However, once traffic flows in the container, the kernel will re-do ARP resolution (the gateway is reachable via the copied routes). If specific cloud setups require pre-populated neighbour entries, that can be added as a follow-up with a new captureNeighbours option. Adding it here would increase scope and complexity significantly.
rp_filter — This is a per-interface sysctl, not part of the netlink L3 configuration. Managing sysctls is the responsibility of the tuning CNI plugin, which is the standard approach in CNI. Users on AWS multi-ENI setups can chain tuning after host-device to set rp_filter=2.

Also added IPv6-related filtering (link-local unicast routes were already skipped).

Add a new configuration option `useInterfaceNetwork` that instructs the host-device plugin to capture the interface's IP addresses and routes from the host before moving the device into the container namespace, and then apply them inside the container. This is critical for virtual environments (AWS, IBM Cloud, GPC) where the cloud provider configures IP addresses and routes directly on the network device. In these environments, there is no traditional IPAM source; the ground truth for L3 configuration lives on the host interface itself. When `useInterfaceNetwork` is enabled, the plugin: - Captures all global-scope addresses and non-local routes from the host device before moving it into the container namespace. - Applies the captured addresses and routes to the interface inside the container. - Reports the addresses and routes in the CNI result (merged with any IPAM result if an IPAM plugin is also configured). NOTE: The interface configuration on the host node must be persistent. When the device is moved back to the host (via DEL) and renamed to its original name, the system's network management service (e.g. NetworkManager, systemd-networkd, cloud-init, or cloud-specific agents) is expected to detect the device and re-apply the IP addresses and routes. This plugin does NOT re-configure the host interface on DEL; it relies on the node's network configuration being declarative and reconciled by the platform's networking stack. Also implements the STATUS command to verify the host device exists, replacing the previous TODO stub. Signed-off-by: Sebastian Sch <sebassch@gmail.com>

SchSeba force-pushed the host-device-l3-info branch 2 times, most recently from 0feea32 to df398aa Compare May 4, 2026 17:27

karampok reviewed May 5, 2026

View reviewed changes

SchSeba force-pushed the host-device-l3-info branch from df398aa to 20dc60e Compare May 11, 2026 13:04

s1061123 requested changes May 11, 2026

View reviewed changes

SchSeba force-pushed the host-device-l3-info branch 3 times, most recently from 72eec2d to 10936b3 Compare May 11, 2026 13:47

squeed requested a review from mlguerrero12 June 22, 2026 14:24

SchSeba force-pushed the host-device-l3-info branch from 10936b3 to a2cbe4c Compare June 24, 2026 17:10

SchSeba force-pushed the host-device-l3-info branch from a2cbe4c to c7b50c6 Compare June 25, 2026 06:10

SchSeba mentioned this pull request Jun 25, 2026

docs: document host-device useInterfaceNetwork option containernetworking/cni.dev#156

Open

Uh oh!

Conversation

SchSeba commented May 4, 2026

Uh oh!

SchSeba commented May 4, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

s1061123 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

karampok commented May 12, 2026

Uh oh!

SchSeba commented Jun 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants