feat: stronger peer identity guarantees

[meling]

Issue: Stronger Peer Identity Guarantees

Status: Open — current implementation uses weak identity; stronger guarantees deferred (this proposal). This was written more than one month ago and some of the file and type names are possibly outdated after various other refactorings.

---

Current Implementations

Two approaches are available in inbound_manager.go:

1. identifyPeer(ctx) — client sends its listening address in gorums-addr metadata. The server extracts the peer's source IP via peer.FromContext and checks it against the IP in the claimed address. A match triggers an address→NodeID lookup in InboundManager.knownNodes. NOTE: This was deleted from inbound_manager.go

2. nodeID(ctx) — client sends its NodeID directly in gorums-node-id metadata. No address lookup table is needed; the server trusts the claimed integer value, subject to validation against the known set.

---

Related Note

We currently use WithNodes() to provide ID to address mapping. We rely on NodeAddress interface to get the Addr() of a node. We may consider an additional interface to provide cryptographic identity information (e.g. certificate subject) for stronger peer identity guarantees, which would be used in the future identifyPeer implementation.

---

Security Limitations

IP Matching Does Not Prevent Spoofing

The identifyPeer check compares the source IP seen by gRPC (peer.Peer.Addr) against the IP inside the claimed address. This provides a weak guard against accidental misconfiguration (e.g., a node claiming the wrong peer's address), but is not a cryptographic guarantee:

• An attacker with access to the network segment can send packets with a forged source IP.
• In containerised or virtualised environments (Kubernetes, Docker bridge networks, VMs), the source IP may be rewritten by NAT or overlay networking, making the check unreliable even for legitimate peers.
• TCP itself does prevent blind IP spoofing in most configurations (three-way handshake), but this relies on routing and OS-level protections, not application logic.

Plain NodeID in Metadata Is Trivially Forgeable

nodeID(ctx) places even less trust in the network: any client can claim any integer NodeID. It is safe only when the network is already trusted (e.g., a single-machine test cluster) or when the returned ID is used solely as a lookup hint that is then confirmed by another mechanism.

---

Stronger Guarantee: Mutual TLS Authentication

For production deployments, gorums replicas are expected to use TLS (grpc.WithTransportCredentials). When mTLS (mutual TLS) is in use, the peer.Peer.AuthInfo field is populated with a credentials.TLSInfo value, which contains the verified TLS state including the peer's X.509 certificate chain.

import "google.golang.org/grpc/credentials"

if tlsInfo, ok := p.AuthInfo.(credentials.TLSInfo); ok {
    // tlsInfo.State.PeerCertificates is the verified chain
    cert := tlsInfo.State.PeerCertificates[0]
    // cert.Subject, cert.DNSNames, etc. are CA-verified
}

This means the server can bind a NodeID to a certificate subject or SAN (Subject Alternative Name), and the mapping is enforced by the PKI, not by per-packet networking.

Possible Design: NodeID Encoded in the Certificate

Each replica is issued a certificate whose Subject.CommonName or a custom extension encodes its NodeID. identifyPeer reads the NodeID from the verified certificate rather than from metadata. This requires:

• A CA (or self-signed cert infrastructure) that issues per-replica certificates.
• The server's gRPC listener to be configured with RequireAndVerifyClientCert.
• identifyPeer to inspect p.AuthInfo.(credentials.TLSInfo) after verifying p.AuthInfo != nil.

Possible Design: Encrypted NodeID in Metadata

An alternative that does not require changes to certificate issuance: the client encrypts its NodeID with its private key (a signature, essentially) and includes the result in gorums-node-id metadata. The server verifies the signature using the client's public key from the mTLS certificate.

Concretely:

1. Client signs sha256(nodeID || nonce) with its private key and sends (nodeID, nonce, signature) in metadata.
2. Server retrieves the client's public key from TLSInfo.State.PeerCertificates[0].
3. Server verifies the signature before accepting the claimed NodeID.

This binds the claimed NodeID to the certificate identity without requiring NodeID to be embedded in the certificate itself.

---

Recommended Path Forward

1. Short term: accept that nodeID is suitable for trusted-network and development use.
2. Medium term: add a WithTLSPeerIdentity option that, when TLS is in use, reads NodeID from the certificate CN/SAN instead of metadata, delegating trust fully to the PKI.
3. Long term: explore the signed-metadata approach for deployments that cannot modify certificate issuance pipelines.

The check p.AuthInfo != nil can be used to detect whether the connection is protected; the identity function could log a warning (or refuse to register the peer) if the connection is unencrypted and the deployment is configured to require TLS.

---

Appendix: Archived identifyPeer Implementation

The identifyPeer approach was prototyped but removed from client_identity.go in favour of the simpler nodeID approach. It is preserved here as a reference for the IP-validation design and its tests, which may be useful when a stronger identity mechanism is implemented.

identifyPeer (removed from client_identity.go)

const gorumsAddrKey = "gorums-addr"

// identifyPeer extracts the claimed server address from the stream context.
// It returns the validated address, or "" for external clients.
// The peer's source IP is checked against the IP in the claimed address.
func identifyPeer(ctx context.Context) (string, error) {
    p, ok := peer.FromContext(ctx)
    if !ok {
        return "", nil
    }
    peerHost, _, _ := net.SplitHostPort(p.Addr.String())

    md, ok := metadata.FromIncomingContext(ctx)
    if !ok {
        return "", nil
    }
    addrs := md.Get(gorumsAddrKey)
    if len(addrs) == 0 {
        return "", nil
    }
    claimedAddr := addrs[0]
    claimedHost, _, _ := net.SplitHostPort(claimedAddr)

    if peerHost != claimedHost {
        return "", fmt.Errorf("peer address mismatch: peer=%s, claimed=%s", peerHost, claimedHost)
    }
    return claimedAddr, nil
}

Tests (removed from client_identity_test.go)

func peerCtx(addr string) context.Context {
    tcpAddr, _ := net.ResolveTCPAddr("tcp", addr)
    return peer.NewContext(context.Background(), &peer.Peer{Addr: tcpAddr})
}

func peerCtxWithMeta(peerAddr, claimedAddr string) context.Context {
    ctx := peerCtx(peerAddr)
    md := metadata.Pairs(gorumsAddrKey, claimedAddr)
    return metadata.NewIncomingContext(ctx, md)
}

func TestIdentifyPeer(t *testing.T) {
    tests := []struct {
        name        string
        ctx         context.Context
        wantAddr    string
        wantErrLike string
    }{
        {name: "KnownReplicaMatchingIP",   ctx: peerCtxWithMeta("192.168.1.10:54321", "192.168.1.10:8080"), wantAddr: "192.168.1.10:8080"},
        {name: "ExternalClientNoMetadata", ctx: peerCtx("192.168.1.10:54321"),                             wantAddr: ""},
        {name: "NoPeerInContext",          ctx: context.Background(),                                       wantAddr: ""},
        {name: "NoMetadataInContext",      ctx: peerCtx("10.0.0.1:9999"),                                   wantAddr: ""},
        {name: "IPMismatchSpoofAttempt",   ctx: peerCtxWithMeta("10.0.0.1:54321", "192.168.1.10:8080"),    wantAddr: "", wantErrLike: "peer address mismatch"},
        {name: "LoopbackReplica",          ctx: peerCtxWithMeta("127.0.0.1:55000", "127.0.0.1:8080"),      wantAddr: "127.0.0.1:8080"},
    }
    for _, tc := range tests {
        t.Run(tc.name, func(t *testing.T) {
            got, err := identifyPeer(tc.ctx)
            if tc.wantErrLike != "" {
                if err == nil || !strings.Contains(err.Error(), tc.wantErrLike) {
                    t.Fatalf("identifyPeer() error = %v; want %q", err, tc.wantErrLike)
                }
                return
            }
            if err != nil { t.Fatal(err) }
            if got != tc.wantAddr { t.Errorf("got %q; want %q", got, tc.wantAddr) }
        })
    }
}