Replace grand-central Ingress with HTTPRoute and Traefik Middlewares

[tomach]

Summary of changes

Extends the existing exposure field support to grand-central. When spec.cluster.exposure: traefik, grand-central is now exposed through the Gateway API (HTTPRoute) and three Traefik Middlewares instead of an nginx Ingress. The default loadbalancer path is unchanged.

• grand-central.py adds builders for HTTPRoute, compress-js, buffering, and ip-allowlist Middlewares; adds create_grand_central_exposure (routing resources only, no deployment/service) and delete helpers for both paths
• exposure.py - ChangeExposureSubHandler now also switches grand-central resources when the exposure field changes
• operations.py - suspend_or_start_grand_central deletes routing resources on suspend and recreates them on resume, respecting the active exposure mode
• handle_update_allowed_cidrs.py patches the ip-allowlist Middleware instead of the Ingress annotation when exposure=traefik
• RBAC - adds permissions for gateway.networking.k8s.io/httproutes and traefik.io/middlewares

Checklist

• Link to issue this PR refers to: https://github.com/crate/cloud/issues/2905
• Relevant changes are reflected in CHANGES.rst
• Added or changed code is covered by tests
• Documentation has been updated if necessary
• Changed code does not contain any breaking changes (or this is a major version change)

[goat-ssh]

Caught some errors on dev on /auth and /health endpoints:

The 'Access-Control-Allow-Origin' header contains multiple values 
'https://console.cratedb-dev.cloud,http://localhost:8000', but only one is allowed.

According to the W3C and MDN web specs, the Access-Control-Allow-Origin header can only contain a single origin, the wildcard *, or null. It cannot accept a comma-separated list of multiple origins. When you pass a list, browsers reject it as an invalid value, causing the CORS block.

[juanpardo]

Added a few comments. It looks good

[juanpardo]

I think @WalBeh was challenging using this namespace in the meeting. I also think it would be nice if we could use the project namespace (but not sure if it's possible) CC @goat-ssh

[juanpardo]

This would still not delete the DNS entry, right? I hope not because that way we can avoid the DNS propagation time when resuming. CC @goat-ssh

[goat-ssh]

Record	Resource	How DNS is created
*.aks1.eastus.azure.cratedb-dev.net → CNAME	IngressRouteTCP (CrateDB cluster)	CNAME to the regional Traefik LB hostname
*.gc.aks1.eastus.azure.cratedb-dev.net → A	HTTPRoute (grand-central)	external-dns creates an A record directly from the Gateway's resolved IP

Note: Both records resolve to the same Traefik load balancer IP: 51.8.42.241

so, yes the tenant grand central DNS is deleted on suspend.

[tomach]

I think it's better to keep the current behaviour (delete routing resources on suspend) for consistency with how the nginx Ingress is handled on suspend already - both are deleted and recreated on resume. In practice our clusters are up and healthy in under 2 minutes on resume, so the DNS wait is not a significant concern.

[goat-ssh]

@tomach another angle about proper ordering of the Traefik Middlewares:

filters:
  # 1. GUARD: Drop unauthorized traffic immediately to save resources
  - extensionRef:
      group: traefik.io
      kind: Middleware
      name: grand-central-ip-allowlist
    type: ExtensionRef

  # 2. SANITIZE/TRANSFORM: Apply security headers and buffering
  - responseHeaderModifier:
      set:
      - name: X-Frame-Options
        value: DENY
      # ... (rest of your headers)
    type: ResponseHeaderModifier

  - extensionRef:
      group: traefik.io
      kind: Middleware
      name: grand-central-cors
    type: ExtensionRef

  - extensionRef:
      group: traefik.io
      kind: Middleware
      name: grand-central-buffering
    type: ExtensionRef

  # 3. OPTIMIZE: Compression should be last to act on the finalized response payload
  - extensionRef:
      group: traefik.io
      kind: Middleware
      name: grand-central-compress-js
    type: ExtensionRef

[tomach]

@tomach another angle about proper ordering of the Traefik Middlewares:

filters:
  # 1. GUARD: Drop unauthorized traffic immediately to save resources
  - extensionRef:
      group: traefik.io
      kind: Middleware
      name: grand-central-ip-allowlist
    type: ExtensionRef

  # 2. SANITIZE/TRANSFORM: Apply security headers and buffering
  - responseHeaderModifier:
      set:
      - name: X-Frame-Options
        value: DENY
      # ... (rest of your headers)
    type: ResponseHeaderModifier

  - extensionRef:
      group: traefik.io
      kind: Middleware
      name: grand-central-cors
    type: ExtensionRef

  - extensionRef:
      group: traefik.io
      kind: Middleware
      name: grand-central-buffering
    type: ExtensionRef

  # 3. OPTIMIZE: Compression should be last to act on the finalized response payload
  - extensionRef:
      group: traefik.io
      kind: Middleware
      name: grand-central-compress-js
    type: ExtensionRef

@goat-ssh Good point, reordered the filters so the IP allowlist runs first as a guard and compression runs last - fc61577

[goat-ssh]

@tomach all looking good on dev! let's ship it