diff --git a/http-routing.html.md.erb b/http-routing.html.md.erb index e42eea93..c87aa7f3 100644 --- a/http-routing.html.md.erb +++ b/http-routing.html.md.erb @@ -228,6 +228,10 @@ data path must trust that the back-end component has not allowed the header to b If you configure the load balancer to terminate TLS and set the XFCC header from the received client certificate, you must also configure the load balancer to strip this header if it is present in client requests. This configuration is required to prevent spoofing of the client certificate. +

+To require and validate client certificates on a per-domain basis and enforce identity-based route policies, see Identity-aware routing. +

+ The following sections describe supported deployment configurations. <% if vars.platform_code == 'CF' %> diff --git a/identity-aware-routing.html.md.erb b/identity-aware-routing.html.md.erb new file mode 100644 index 00000000..a1a13514 --- /dev/null +++ b/identity-aware-routing.html.md.erb @@ -0,0 +1,308 @@ +--- +title: Identity-aware routing +owner: CF for VMs Networking +--- + +This topic provides you with an overview of how identity-aware routing works in <%= vars.app_runtime_first %>. + +Identity-aware routing lets the Gorouter require and validate mutual TLS (mTLS) on a per-domain basis and route requests based on the caller's verified identity. It enables app-to-app traffic to flow _through the Gorouter_ — load-balanced and platform-enforced — while the platform authorizes each request from the caller's certificate rather than relying on the app to authenticate callers itself. + +Identity-aware routing has two tiers: + +* **Identity-aware domain (the headline use case).** An operator creates an mTLS domain with route-policy enforcement enabled. The Gorouter validates the caller's Diego _instance identity_ certificate, extracts the caller's app, space, and org, and enforces route policies with a default-deny model. This is the recommended way to do authenticated <%= vars.app_runtime_abbr %> app-to-app communication over the Gorouter. +* **mTLS domain (a variant).** An operator creates an mTLS domain that requires and validates any client certificate signed by a configured certificate authority (CA), then forwards it to the app. Route policies are not enforced, so the _backend app does its own authorization_. This tier suits external, non-<%= vars.app_runtime_abbr %> clients. + +The `*.apps.identity` domain is to identity-aware routing what `*.apps.internal` is to container-to-container (C2C) networking: a conventional wildcard domain for app-to-app traffic. The key difference is the data path. Identity-aware traffic goes through the Gorouter, where it is load-balanced and centrally authorized, whereas C2C traffic flows directly between containers over an overlay network. + +

+For direct, low-latency app-to-app traffic over an overlay network instead of through the Gorouter, see Container-to-container networking. +

+ + +## Architecture + +Identity-aware routing combines four pieces: per-domain mTLS in the Gorouter, route policies stored by the Cloud Controller, the Diego instance identity that gives each app instance a verifiable certificate, and BOSH DNS for the `*.apps.identity` wildcard alias. + +To understand the components and how they work together, see the following diagram and table. + +![Identity-aware routing request flow: App A presents its instance identity certificate to the Gorouter, which validates it and enforces route policy before forwarding to App B](./images/identity-aware-routing-arch.svg) + + + + + + + + + + + + + + + + + + + + + + + + + + +
PartFunction
Gorouter (per-domain mTLS)Terminates mTLS for domains configured under router.domains. For each such domain it: +
    +
  • Requires a client certificate and validates it against the domain's configured CA (ca_certs), independent of the platform-wide client-certificate validation setting.
    • +
  • Extracts the caller's <%= vars.app_runtime_abbr %> identity from the certificate Subject.
    • +
  • Enforces the route's policies, denying the request if no policy allows the caller.
    • +
  • Forwards the request to the backend with the X-Forwarded-Client-Cert (XFCC) header.
    • +
+
Cloud ControllerStores route policies and the per-domain enforcement setting. It flattens policies into route options that are synced to Diego so the Gorouter can enforce them.
Diego instance identityIssues every app instance a short-lived identity certificate. The Subject carries the caller's identity as organizational units, for example CN=<instance-id>, OU=app:<app-guid>, OU=space:<space-guid>, OU=organization:<org-guid>. The certificate and key are available to the app as CF_INSTANCE_CERT and CF_INSTANCE_KEY.
BOSH DNSResolves the *.apps.identity wildcard alias to the Gorouter so that callers reach the platform's mTLS listener.
+ + +## How app-to-app identity-aware routing works + +When one app calls another over an identity-aware domain, the request flows through the following steps. The numbers correspond to the architecture diagram above. + +1. The calling app (for example, `frontend-app`) makes an HTTPS request to the destination route, such as `https://backend.apps.identity`, presenting its Diego instance identity certificate as the client certificate. The certificate and key are mounted in the container as `CF_INSTANCE_CERT` and `CF_INSTANCE_KEY`. +1. The Gorouter terminates mTLS for the domain and validates the presented certificate against the domain's configured CA (the instance identity CA). +1. The Gorouter extracts the caller's <%= vars.app_runtime_abbr %> identity from the certificate Subject organizational units: `OU=app:`, `OU=space:`, and `OU=organization:`. +1. The Gorouter checks the destination route's policies. If no policy allows this caller, the Gorouter denies the request with an HTTP `403 Forbidden` response. This is the default-deny model. +1. If a policy allows the caller, the Gorouter forwards the request to the backend app and sets the `X-Forwarded-Client-Cert` (XFCC) header so the backend can also see the verified caller identity. For more information, see [The client certificate header and identity](#xfcc). + +This model is _destination-controlled_: the policies that decide who may reach a route live on the destination route, not on the caller. + + +## Route policies and the default-deny model + +Route policies determine which callers are allowed to reach a route on an identity-aware domain. They are _destination-controlled_: only a Space Developer in the route's own space manages them. + +![Default-deny route policies on *.apps.identity: App A is allowed to reach App B and App C by explicit policy, while App B to App C is denied by default](./images/identity-aware-routing-policy.svg) + +### Enabling enforcement + +Route-policy enforcement is turned on when the _domain_ is created, and it is immutable for the life of the domain. An operator or org manager creates the domain with `--enforce-route-policies`: + +
+$ cf create-shared-domain apps.identity --enforce-route-policies
+$ cf create-private-domain my-org apps.identity --enforce-route-policies
+
+ +You can optionally bound the scope that policy sources may target with `--scope`, which accepts `any`, `org`, or `space`. The `--scope` flag is only valid together with `--enforce-route-policies`: + +
+$ cf create-shared-domain apps.identity --enforce-route-policies --scope org
+
+ +When enforcement is on, every route on the domain denies all callers until a policy explicitly allows them. + +### Policy sources + +A policy allows one source to reach a route. A source identifies the caller by its verified <%= vars.app_runtime_abbr %> identity and takes one of the following forms: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
SourceFriendly flagRaw value
A specific app--source-app APPcf:app:<app-guid>
All apps in a space--source-space SPACEcf:space:<space-guid>
All apps in an org--source-org ORGcf:org:<org-guid>
Any authenticated <%= vars.app_runtime_abbr %> caller--source-anycf:any
+ +### Managing policies + +Use the cf CLI to add, list, and remove route policies. For example, to allow `frontend-app` to reach the `backend` route, list the policies on the domain, and then remove the policy: + +
+$ cf add-route-policy apps.identity --hostname backend --source-app frontend-app
+$ cf route-policies --domain apps.identity
+$ cf remove-route-policy apps.identity --hostname backend --source-app frontend-app
+
+ +You can express the same source with the raw `--source` form, for example `--source cf:app:`. To match a specific path, add `--path`. + + +## The client certificate header and identity + +After the Gorouter validates the caller's certificate, it passes the certificate — or a digest of it — to the backend app in the `X-Forwarded-Client-Cert` (XFCC) header. This is the same mechanism <%= vars.app_runtime_abbr %> uses for forwarding client certificates in general. For more information, see Forwarding client certificate to apps. + +A `router.domains` entry sets how the certificate is forwarded with `xfcc_format`: + + + + + + + + + + + + + + + + + + + + + +
FormatHeader contentsApproximate size
raw (default)The full client certificate, base64-encoded PEM.~1.5 KB
envoyA compact representation, Hash=<sha256>;Subject="<DN>".~300 B
+ +The backend app reads the certificate Subject organizational units (`OU=app:`, `OU=space:`, `OU=organization:`) to learn the caller's app, space, and org. On an identity-aware domain the platform has already validated the certificate and authorized the request, so the XFCC header conveys an identity the app can trust for auditing or finer-grained, app-level decisions. + +

+Consuming the envoy (hashed) XFCC value in Java apps relies on the java-buildpack-client-certificate-mapper (cloudfoundry/java-buildpack-client-certificate-mapper#11). If that support is not yet released in your buildpack, prefer the raw format for Java backends. +

+ + +## Identity-aware routing, C2C networking, and ASGs + +Identity-aware routing, container-to-container (C2C) networking, and application security groups (ASGs) all control app connectivity, but they operate at different points and with different identity models. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Identity-aware routingC2C networkingASGs
Data pathThrough the Gorouter (load-balanced)Direct, over the overlay networkEgress firewall (no app-to-app path of its own)
Identity sourceVerified client certificate (Diego instance identity; app, space, org OUs)Source app GUID, tagged in the VXLAN GBP headerSource space
GranularityCaller app, space, or org → a routeSource app → destination appSpace → destination IP range and ports
Enforcement pointGorouter (per-domain mTLS)VXLAN policy agent on the Diego CellDiego Cell egress
Default modelDeny until a policy allowsDeny until a policy allowsDeny until an ASG allows
Load balancing and access logsYes (Gorouter access logs)No (direct connection)Not applicable
+ +In short: use identity-aware routing for north-south app-to-app traffic over the Gorouter with a verified caller identity; use C2C networking for direct, low-latency east-west traffic; and use ASGs for coarse-grained egress control. + + +## External client certificates + +Not every caller is a <%= vars.app_runtime_abbr %> app. Partner systems, IoT devices, and other external clients present their own certificates, which do not carry a Diego instance identity. For these callers an operator configures a plain _mTLS domain_. + +In this configuration the operator adds a `router.domains` entry whose CA (`ca_certs`) is the external CA that issues the client certificates. The Gorouter requires and validates the client certificate against that CA and forwards it to the backend in the XFCC header. Because there is no <%= vars.app_runtime_abbr %> identity to evaluate, route policies are not used, and the _backend app authorizes the request_ from the certificate it receives. + +

+Do not pass --enforce-route-policies for a domain that serves non-<%= vars.app_runtime_abbr %> certificates. Route policies key off the <%= vars.app_runtime_abbr %> identity organizational units (app, space, org) in a Diego instance identity certificate, which external certificates do not have. Enforcement on such a domain would deny every caller. +

+ + +## Observability + +The Gorouter records the outcome of mTLS validation and route-policy enforcement in its access logs. When the following fields are enabled as extra fields in the access-log configuration, each router (RTR) log line can include the verified caller identity and the policy decision: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldMeaning
caller_cf_appThe caller's app GUID, from the validated client certificate.
caller_cf_spaceThe caller's space GUID.
caller_cf_orgThe caller's org GUID.
route_policyThe route-policy rule that matched the request, for example cf:app:<app-guid>. It is - when no rule matched or enforcement is disabled.
tls_sniThe TLS Server Name Indication (SNI) value the caller requested.
+ +Each field is `-` when the request carried no verified identity or the field does not apply. For example, an allowed request to `backend.apps.identity` records the caller and the matching rule: + +
+... 200 ... tls_sni:"backend.apps.identity" caller_cf_app:"app-guid-123" caller_cf_space:"space-guid-456" caller_cf_org:"org-guid-789" route_policy:"cf:app:app-guid-123"
+
+ +A request that is denied by the default-deny model records the verified caller but no matching rule, with an HTTP `403` status: + +
+... 403 ... tls_sni:"backend.apps.identity" caller_cf_app:"app-guid-123" caller_cf_space:"space-guid-456" caller_cf_org:"org-guid-789" route_policy:"-"
+
+ +These fields let operators audit who reached a route and which policy decisions allowed or denied traffic. + + +## Related reading + +* [RFC-0055: Identity-Aware Routing for Gorouter](https://github.com/cloudfoundry/community/blob/main/toc/rfc/rfc-0055-identity-aware-routing-for-gorouter.md) +* Container-to-container networking +* HTTP routing: Forwarding client certificate to apps +* [Configuring identity-aware routing](../devguide/deploy-apps/identity-aware-routing.html) (developer how-to) +* [Enabling identity-aware routing](../deploying/cf-deployment/enable-identity-aware-routing.html) (operator setup) diff --git a/images/identity-aware-routing-arch.mmd b/images/identity-aware-routing-arch.mmd new file mode 100644 index 00000000..95705309 --- /dev/null +++ b/images/identity-aware-routing-arch.mmd @@ -0,0 +1,14 @@ +flowchart LR + A["App A
presents client
identity cert"] + GR["GoRouter
per-domain mTLS
on *.apps.identity"] + V{"Validate and
authorize"} + B["App B
route: my-api.apps.identity"] + D[("Diego
instance identity CA")] + CC[("Cloud Controller
route policies")] + + A -->|"1 - HTTPS plus client identity cert"| GR + GR --> V + D -.->|"trust: instance identity CA"| V + CC -.->|"sync: route policies, default-deny"| V + V -->|"2 - cert valid, OU app/space/org allowed; forward plus XFCC"| B + V -.->|"denied: 403"| A diff --git a/images/identity-aware-routing-arch.svg b/images/identity-aware-routing-arch.svg new file mode 100644 index 00000000..bb2e8bd0 --- /dev/null +++ b/images/identity-aware-routing-arch.svg @@ -0,0 +1 @@ +

1 - HTTPS plus client identity cert

trust: instance identity CA

sync: route policies, default-deny

2 - cert valid, OU app/space/org allowed; forward plus XFCC

denied: 403

App A
presents client
identity cert

GoRouter
per-domain mTLS
on *.apps.identity

Validate and
authorize

App B
route: my-api.apps.identity

Diego
instance identity CA

Cloud Controller
route policies

 \ No newline at end of file diff --git a/images/identity-aware-routing-policy.mmd b/images/identity-aware-routing-policy.mmd new file mode 100644 index 00000000..14327885 --- /dev/null +++ b/images/identity-aware-routing-policy.mmd @@ -0,0 +1,8 @@ +flowchart LR + A["App A"] + B["App B
billing.apps.identity"] + C["App C
reports.apps.identity"] + + A -->|"allowed: policy A to B"| B + A -->|"allowed: policy A to C"| C + B -.->|"denied by default"| C diff --git a/images/identity-aware-routing-policy.svg b/images/identity-aware-routing-policy.svg new file mode 100644 index 00000000..74ee3ba9 --- /dev/null +++ b/images/identity-aware-routing-policy.svg @@ -0,0 +1 @@ +

allowed: policy A to B

allowed: policy A to C

denied by default

App A

App B
billing.apps.identity

App C
reports.apps.identity

 \ No newline at end of file diff --git a/security-index.html.md.erb b/security-index.html.md.erb index 57659773..03a0f262 100644 --- a/security-index.html.md.erb +++ b/security-index.html.md.erb @@ -8,6 +8,7 @@ The following topics provide information about security and networking: * [<%= vars.app_runtime_abbr %> Security](security.html). * [Container Security](container-security.html). * [Container-to-Container Networking](understand-cf-networking.html). +* [Identity-Aware Routing](identity-aware-routing.html). * [Orgs, Spaces, Roles, and Permissions](roles.html). * [App Security Groups](asg.html). * [App SSH Components and Processes](./diego/ssh-conceptual.html). \ No newline at end of file diff --git a/understand-cf-networking.html.md.erb b/understand-cf-networking.html.md.erb index f76d377e..2a2ce9e2 100644 --- a/understand-cf-networking.html.md.erb +++ b/understand-cf-networking.html.md.erb @@ -10,6 +10,10 @@ Container-to-container networking is not available for apps hosted on Microsoft The container-to-container networking feature enables app instances to communicate with each other directly. <%= vars.cf_networking %> +

+To route app-to-app traffic through the Gorouter with platform-enforced, identity-based authorization (rather than a direct overlay network), see Identity-aware routing. +

+ ## Architecture