Bug Report
1. Minimal reproduce step (Required)
On a NextGen TiDB build where Bazel shards share the same PD/etcd cluster:
- Bootstrap a user-keyspace
Domain. Its eager SYSTEM-keyspace runtime registers a virtual /tidb/server/info/<UUID> record in the SYSTEM etcd namespace.
- Close or idle-GC that cross-keyspace runtime through
crossks.SessionManager.close.
- Before the 90-second server-info lease expires, query the exact virtual server-info key or execute an all-TiDB cluster table such as
information_schema.cluster_slow_query.
The issue is also visible in pull_integration_realcluster_test_next_gen: when TestCrossKSRuntimeGCLoopStartedBySystemDomain runs before TestClusterTableSendError, the latter receives two synthesized TiDB timeout warnings instead of one.
CI evidence:
2. What did you expect to see? (Required)
Closing or GC-evicting a cross-keyspace runtime should synchronously remove that runtime's virtual server-info record. Later cluster-table fanout should enumerate only currently live TiDB entries.
3. What did you see instead (Required)
crossks.SessionManager.close stops its goroutines and closes the etcd client without calling RemoveServerInfo or revoking the server-info lease. The virtual record remains visible for up to the 90-second TTL.
An all-TiDB cluster-table query can therefore create one task for the live SYSTEM Domain and another for the stale assumed-SYSTEM entry. In the failpoint test path, each task creates a timeout warning, making TestClusterTableSendError fail deterministically with an observed warning count of 2 instead of 1 whenever the cross-keyspace shard runs first.
4. What is your TiDB version? (Required)
Confirmed on master at 5e1901b920b and on CI base ebeea41e5667465bed38b6812be75a139ec804ea.
Analysis
The normal Domain.Close lifecycle removes its server-info key before closing etcd access. The cross-keyspace SessionManager.close lifecycle omits that step. The fix should remove the runtime-owned exact key after keeper goroutines exit and before the namespaced etcd client is closed; read-side AssumedKeyspace filtering would only mask the stale state and change cluster-table semantics.
Bug Report
1. Minimal reproduce step (Required)
On a NextGen TiDB build where Bazel shards share the same PD/etcd cluster:
Domain. Its eager SYSTEM-keyspace runtime registers a virtual/tidb/server/info/<UUID>record in the SYSTEM etcd namespace.crossks.SessionManager.close.information_schema.cluster_slow_query.The issue is also visible in
pull_integration_realcluster_test_next_gen: whenTestCrossKSRuntimeGCLoopStartedBySystemDomainruns beforeTestClusterTableSendError, the latter receives two synthesized TiDB timeout warnings instead of one.CI evidence:
2. What did you expect to see? (Required)
Closing or GC-evicting a cross-keyspace runtime should synchronously remove that runtime's virtual server-info record. Later cluster-table fanout should enumerate only currently live TiDB entries.
3. What did you see instead (Required)
crossks.SessionManager.closestops its goroutines and closes the etcd client without callingRemoveServerInfoor revoking the server-info lease. The virtual record remains visible for up to the 90-second TTL.An all-TiDB cluster-table query can therefore create one task for the live SYSTEM Domain and another for the stale assumed-SYSTEM entry. In the failpoint test path, each task creates a timeout warning, making
TestClusterTableSendErrorfail deterministically with an observed warning count of 2 instead of 1 whenever the cross-keyspace shard runs first.4. What is your TiDB version? (Required)
Confirmed on
masterat5e1901b920band on CI baseebeea41e5667465bed38b6812be75a139ec804ea.Analysis
The normal
Domain.Closelifecycle removes its server-info key before closing etcd access. The cross-keyspaceSessionManager.closelifecycle omits that step. The fix should remove the runtime-owned exact key after keeper goroutines exit and before the namespaced etcd client is closed; read-sideAssumedKeyspacefiltering would only mask the stale state and change cluster-table semantics.