Multi-VRFs documentation#2665
Open
matthewdupre wants to merge 18 commits into
Open
Conversation
✅ Deploy Preview for calico-docs-preview-next ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
❌ Deploy Preview for tigera failed. Why did it fail? →Built without sensitive environment variables
|
matthewdupre
changed the title
matthewdupre
changed the title
ctauchen
reviewed
May 21, 2026
Collaborator
ctauchen
left a comment
ctauchen
left a comment
There was a problem hiding this comment.
@matthewdupre Thanks for this! A few comments, mostly small suggestions.
One general thing we could improve (perhaps before GA), would be to make the full procedure more specific. Much of the procedural content is pretty high-level, leaving users to fill in the details themselves.
I'm happy enough to get this in now for testing and the release. Might be worth chatting about it and revising before GA.
LGTM, with suggestions.
| ``` | ||
|
|
||
| :::tip | ||
| On Ubuntu <= 25.04, `apt install linux-generic`. |
Collaborator
There was a problem hiding this comment.
This < character is killing the build. Suggest you rewrite as follows:
On Ubuntu 25.04 or earlier,
On Ubuntu 25.10 or later,
This does leave me wondering about the .05-.09 releases, for what it's worth.
Member
Author
There was a problem hiding this comment.
Ubuntu use a year.month format with 2 releases a year (04 and 10). That said I(/Claude) will just remove these.
|
|
||
| ## Big picture | ||
|
|
||
| Attach pods to one or more Virtual Routing and Forwarding (VRF) domains |
Collaborator
There was a problem hiding this comment.
Suggested change
| Attach pods to one or more Virtual Routing and Forwarding (VRF) domains | |
| Attach pods to one or more virtual routing and forwarding (VRF) domains |
| - **Reach and be reached from overlapping external IPs.** The same external IP address can be reused | ||
| in multiple tenant networks outside the cluster. Pods on a VRF can reach the | ||
| copy of that IP in their own tenant network, and responses come back in the | ||
| correct VRF. Services backed by VRF attached pods can also be exported to |
Collaborator
There was a problem hiding this comment.
Suggested change
| correct VRF. Services backed by VRF attached pods can also be exported to | |
| correct VRF. Services backed by VRF attached pods can also be exported to |
worth removing all double spaces
|
|
||
| Typically the easiest way to use VRFs will be to attach pods to the flat network | ||
| (with the primary interface), and then whichever VRF networks are required using | ||
| Multus `NetworkAttachmentDefinition`s. `multus-service` can then be used to |
Collaborator
There was a problem hiding this comment.
Best to describe generically or use the code as a modifier. Helpful for people not already familiar with what you're referring to.
using Multus network attachment definitions.
using Multus NetworkAttachmentDefinition resources.
Same goes for multus-service and similar cases elsewhere.
| **Required** | ||
|
|
||
| - $[prodname] installed with the **nftables dataplane** (`linuxDataplane: Nftables` in the [Installation](../../reference/installation/api.mdx) resource). | ||
| The iptables and eBPF dataplanes are not supported in the tech preview, eBPF will be added before GA but iptables support is not planned. |
Collaborator
There was a problem hiding this comment.
Remove. The first bit is covered in your previous line, and we don't want to be announcing future work. We'll add that when it happens.
Comment on lines
+120
to
+128
| ```bash | ||
| sudo modprobe vrf && lsmod | grep '^vrf ' | ||
| ``` | ||
|
|
||
| :::tip | ||
| On Ubuntu <= 25.04, `apt install linux-generic`. | ||
| On Ubuntu >= 25.10, the module is built in. | ||
| On RHEL, the module is built in. | ||
| ::: |
Collaborator
There was a problem hiding this comment.
I'm inclined to say we can expect people to figure out whether they have this module and to install if necessary. For economy's sake, I'd scrap these lines.
| ::: | ||
|
|
||
| - BGP peering configured between each node and the upstream router on each VRF (see [Create BGPPeers and BGPFilters](#create-bgppeers-and-bgpfilters)). | ||
| - [Multus](./multiple-networks.mdx) installed if you want to attach pods to a VRF using a secondary interface (the most common topology). |
Collaborator
There was a problem hiding this comment.
Suggested change
| - [Multus](./multiple-networks.mdx) installed if you want to attach pods to a VRF using a secondary interface (the most common topology). | |
| - If you want to attach pods to a VRF using a secondary interface (the most common topology), you need to have [Multus](./multiple-networks.mdx) installed on your cluster. |
Always best to lead with conditionals.
Comment on lines
+133
to
+135
| **Recommended** | ||
|
|
||
| - Pin `nodeAddressAutodetection` in the [Installation](../../reference/installation/api.mdx) to a specific interface (for example `eth0`) or to `kubernetes: NodeInternalIP`. The default "first found" autodetection can chase a VRF-attached interface when additional interfaces are brought up, breaking the cluster. |
Collaborator
There was a problem hiding this comment.
This sounds important. Can we just make it a requirement?
Comment on lines
+153
to
+154
| - **Dataplane**: only the [nftables dataplane](../../operations/nftables.mdx) is supported. iptables and eBPF are not supported. | ||
| - **kube-proxy**: must be in `nftables` mode. `ipvs` mode is **never** supported (this is a permanent restriction, not a tech-preview limitation). |
Collaborator
There was a problem hiding this comment.
covered already in the requirements section
Comment on lines
+166
to
+171
| ## Requirements | ||
|
|
||
| - Linux kernel **5.6 or later** (for the `meta sdifname` nftables match used by VRF policy dispatch). | ||
| - The `vrf` kernel module must be loaded on every node. On Ubuntu this is part of `linux-modules-extra-$(uname -r)`. Confirm with `sudo modprobe vrf && lsmod | grep '^vrf '`. | ||
| - $[prodname] must be installed with `linuxDataplane: Nftables` and kube-proxy must also be in nftables mode. | ||
| - The cluster's [Installation](../installation/api.mdx) should pin `nodeAddressAutodetection` to a specific interface or to `kubernetes: NodeInternalIP`. The default "first found" autodetection can chase a VRF-attached interface and break the cluster when extra interfaces are added. |
Collaborator
There was a problem hiding this comment.
this is already covered above
Address ctauchen review comment on multi-vrf.mdx:15: > ```suggestion > Attach pods to one or more virtual routing and forwarding (VRF) domains > ``` Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Address ctauchen review comment on multi-vrf.mdx:31:
> ```suggestion
> correct VRF. Services backed by VRF attached pods can also be exported to
> ```
> worth removing all double spaces
Applied the suggested fix on line 31 and also collapsed the double-space
on line 59 ("`NetworkAttachmentDefinition`s. `multus-service`").
Resolves tigera#2665 (comment)
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Address ctauchen review comment on multi-vrf.mdx:59: > Best to describe generically or use the code as a modifier. Helpful for > people not already familiar with what you're referring to. > > using Multus network attachment definitions. > using Multus `NetworkAttachmentDefinition` resources. > > Same goes for multus-service and similar cases elsewhere. `multus-service` is already code-formatted in this file, so the only edit needed is the NetworkAttachmentDefinition phrasing on line 59. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Address ctauchen review comment on multi-vrf.mdx:115: > Remove. The first bit is covered in your previous line, and we don't > want to be announcing future work. We'll add that when it happens. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Address ctauchen review comment on multi-vrf.mdx:116: > Isn't this covered in the first item? Can Calico be in nftables while > kube-proxy is something else? Calico's dataplane and kube-proxy's mode are technically independent knobs, but for multi-VRF both must be nftables. Merging the two bullets keeps both requirements visible without making them look like separate prerequisites. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…ments Address ctauchen review comment on multi-vrf.mdx:117: > This is covered already by the general system requirements: > https://docs.tigera.io/calico-enterprise/latest/getting-started/install-on-clusters/requirements#node-requirements Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…uild) Address ctauchen review comment on multi-vrf.mdx:125: > This `<` character is killing the build. Suggest you rewrite as follows: > > On Ubuntu 25.04 or earlier, > > On Ubuntu 25.10 or later, > > This does leave me wondering about the .05-.09 releases, for what it's worth. Ubuntu does not ship .05-.09 point releases, so the wording covers all Ubuntu versions cleanly. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Address ctauchen review comment on multi-vrf.mdx:128: > I'm inclined to say we can expect people to figure out whether they > have this module and to install if necessary. For economy's sake, > I'd scrap these lines. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Address ctauchen review comment on multi-vrf.mdx:131: > ```suggestion > - If you want to attach pods to a VRF using a secondary interface (the most common topology), you need to have [Multus](./multiple-networks.mdx) installed on your cluster. > ``` > Always best to lead with conditionals. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Address ctauchen review comment on multi-vrf.mdx:135: > This sounds important. Can we just make it a requirement? Moved the `nodeAddressAutodetection` bullet from the **Recommended** section to **Required**, and dropped the now-empty **Recommended** heading. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Address ctauchen review comment on network.mdx:154: > covered already in the requirements section Removed the kube-proxy bullet from Limitations. The Requirements section already states that kube-proxy must be in nftables mode; the "ipvs is never supported" sub-point (a permanent restriction, not a tech-preview limitation) is folded into that Requirements bullet so it stays documented. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Address ctauchen review comment on network.mdx:171: > this is already covered above The `nodeAddressAutodetection` recommendation is deployment guidance that belongs in the how-to (Configure multi-VRF networking), not in this API reference page. The how-to lists it as a Required item. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…all instructions) Follow-up to the earlier "drop `vrf` kernel module bullet" commit. The full bullet (with the modprobe one-liner and the per-distro tip) was scrapped per ctauchen's suggestion at multi-vrf.mdx:128: > I'm inclined to say we can expect people to figure out whether they > have this module and to install if necessary. For economy's sake, > I'd scrap these lines. Per author follow-up: keep the bare statement that the module must be available — just drop the detection / install instructions. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Follow-up to the earlier "drop duplicated kube-proxy limitation" commit on network.mdx, which folded an `ipvs is never supported` parenthetical into the Requirements bullet. Per author follow-up: drop it. `ipvs` is only minimally supported elsewhere in the product, and requiring nftables mode already implies "not ipvs", so the parenthetical adds noise without information. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Member
Author
|
@ctauchen thanks! Markups made. I've left them as separate commits (I like reviewing Claude better that way), but we should definitely squash before merging! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
<!---PR title format: [GH#][DOCS#]: --->Product Version(s):
Issue:
Link to docs preview:
SME review:
DOCS review:
Additional information:
Merge checklist: