[RFC]: Fixed-Lifetime Soft-Priority Retention for Groups

[g310v]

Changes proposed

Summary

This RFC proposes porting the group-retention proof of concept from #2835 onto the fixed-lifetime soft-pin lifecycle introduced in #2909.

The original proof of concept added a RetainGroups API that allowed callers to declare retention intent for a group before all of its objects existed. It implemented retention by granting ordinary read leases to current and future group members.

This proposal preserves the group-level API and lifecycle while giving group retention its own source of truth. Where possible, the implementation reuses the fixed-deadline, expiry, and eviction-priority machinery introduced in #2909 without storing group policy in each member’s object-level soft-pin state.

Each retained group receives an independent fixed deadline. While that deadline is active, group membership contributes a soft retention signal during eviction. Reads, existence checks, write retries, and replica completion do not extend the group deadline.

This RFC is intended as a first-stage design for discussion. The exact priority model and fallback eviction behavior remain open.

Motivation and Background

Mooncake already supports organizing related objects through group_ids. Existing group membership is used for behavior such as grouped eviction.

This RFC does not introduce a new grouping or session abstraction. It proposes adding a retention policy that operates on existing groups.

Mooncake currently has two relevant per-object lifetime mechanisms:

• Read leases, which are refreshed by read activity and temporarily prevent recently accessed objects from being reclaimed.
• Object-level soft pins, which use a fixed deadline committed when a write becomes readable and are not renewed by reads.

Neither mechanism fully addresses pre-declared group retention.

A serving system may know at request start that it is about to produce a family of related KV-cache objects. Those objects may be written incrementally. Without pre-declared retention, early members may become eviction candidates before later members have been published or before the completed group is consumed.

The caller therefore needs to express:

I am about to produce objects belonging to this group. Apply a retention policy to its current and future members for a bounded amount of time.

The proof of concept reused ordinary leases because the eviction system already understood them. However, this couples explicit retention policy to normal read behavior.

If a group member is read, its lease may be refreshed. Group retention implemented through leases can therefore be extended by access patterns rather than by an explicit retention request. The caller can no longer reason precisely about when the group’s retention will end.

Group retention should instead have an independent fixed lifetime.

Why not soft-pin every object?

Object-level soft pins and group retention represent different intent.

An object soft pin applies to a particular write and becomes active when that object is published. Group retention may be declared before the first object in the group exists.

Applying soft pins individually would also require the caller to coordinate every current and future member and would duplicate group-level policy in object metadata.

Group retention provides:

• pre-declaration before members exist;
• one lifetime for the logical group;
• automatic coverage for future members;
• separation between group organization and object-level state.

Goals

• Allow retention to be declared before a group has any members.
• Apply active group retention to both current and future members.
• Use a fixed deadline that is not renewed by reads or ordinary lease activity.
• Allow explicit retention requests to extend, but never shorten, an existing deadline.
• Keep group retention separate from read leases and object-level soft pins.
• Bound the number of retained groups and the maximum requested TTL.
• Keep retention scoped by tenant and group ID.
• Remove the group’s retention contribution automatically after expiry.
• Preserve existing behavior for clients that do not use RetainGroups.

Non-Goals

This RFC does not propose:

• changing how group_ids or group membership are represented;
• changing the meaning of session or group identifiers;
• replacing object-level soft pins;
• changing ordinary read-lease semantics;
• treating group retention as hard pinning;
• redesigning placement or replication policy;
• introducing an indefinite retention mechanism;
• changing PUT or GET correctness semantics.

Proposed Semantics

RetainGroups API

The initial API follows the proof-of-concept shape:

RetainGroups(
    group_ids,
    ttl_ms,
    tenant_id
) -> per-group results

A successful result means that retention intent has been recorded for the tenant/group pair.

The group does not need to have members at the time of the request. Supporting an empty or not-yet-created group is required for the pre-declaration use case.

The initial behavior should be:

• reject empty group IDs;
• reject a zero TTL;
• reject TTLs above a configured maximum;
• enforce a configured active-retained-group limit;
• report admission independently for each group in the batch;
• allow a repeated call to extend an active lifetime;
• never allow a repeated call to shorten an active lifetime.

An explicit release operation is not required in the first stage.

Fixed-Lifetime Deadline

Each admitted tenant/group pair is associated with a retention deadline. This group-level deadline is the authoritative source of truth for retention. It is not copied into each member's soft_pin_timeout.

A repeated request may extend the deadline but cannot reduce the group’s remaining retention lifetime.

The deadline may only change because of another explicit group-retention request. It must not be extended by:

• Get;
• ExistKey;
• read-lease refresh;
• repeated PutEnd;
• completion of additional replicas;
• object-level soft-pin operations.

After the deadline expires, the group contributes no special retention priority.

Future Members

Retention intent may exist before any group member is written.

When an object later becomes a member of a group with an active retention deadline, the object becomes subject to the group’s remaining retention lifetime.

The member does not receive a new full TTL. All members observe the same group deadline.

This association does not modify the object’s ordinary lease or soft-pin state.

A retained group may temporarily have no members. Its retention entry should remain valid until expiry because new members may still arrive.

High-Level Lifecycle

flowchart TD
    A[Caller invokes RetainGroups] --> B[Validate TTL and admission limits]
    B -->|Rejected| C[Return per-group rejection]
    B -->|Admitted| D[Record or extend tenant/group deadline]
    D --> E{Current members exist?}
    E -->|No| F[Preserve intent for future members]
    E -->|Yes| G[Members observe active group retention]
    F --> H[Objects are later written into the group]
    H --> G
    G --> I[Eviction evaluates group retention independently]
    I --> J{Deadline active?}
    J -->|Yes| K[Apply group soft-priority policy]
    J -->|No| L[Use ordinary eviction behavior]

Loading

Interaction With Read Leases

Group retention and read leases remain independent.

A read may refresh an object’s ordinary lease according to existing behavior. It must not affect the group deadline.

Similarly, calling RetainGroups must not modify the read-lease timestamps of existing or future members.

When group retention expires, an object may still be temporarily ineligible for eviction because it has an active ordinary lease. That is existing lease behavior rather than residual group retention.

Interaction With Object Soft Pins

Group retention and object-level soft pins are also independent.

An object may have:

• neither mechanism;
• only an object soft pin;
• only active group retention;
• both.

Neither mechanism renews, clears, or rewrites the other.

A copy or additional replica of an existing object remains part of the same logical object and group. Replica completion must not restart or extend the group deadline.

Although the policy state remains separate, the implementation should reuse the fixed-lifetime deadline evaluation, expiry normalization, cleanup patterns, metrics, and eviction-priority hooks from #2909 where practical. Reuse of this machinery must not make an object’s soft_pin_timeout the source of truth for group retention.

Eviction Integration

The key change from the proof of concept is that active group retention is represented by a separate group-level source of truth rather than by ordinary read leases or copied per-object soft-pin state. The eviction path may reuse #2909’s existing soft-priority evaluation and ranking machinery to consume this signal.

The intended behavior is:

• lower-priority evictable objects are reclaimed first;
• active group retention increases the effective retention priority of the relevant group or members;
• hard pins, active read leases, replica references, and other non-evictable constraints remain authoritative;
• the eviction path revalidates group retention before reclaiming an object;
• expiry immediately removes the group’s priority contribution;
• group retention does not modify group membership or object metadata used by the existing soft-pin lifecycle.

The exact representation of group priority remains open.

Two likely models are:

Fixed retained-group tier

All actively retained groups receive the same priority tier above ordinary objects.

This preserves the current RetainGroups(group_ids, ttl_ms, tenant_id) API and provides a small first-stage implementation.

Explicit numeric group priority

RetainGroups also accepts a priority value. The eviction system combines object-level and group-level priorities through a defined rule.

This provides more flexibility but expands the API and requires agreement on priority ranges, normalization, and composition.

A fixed tier is the narrower first-stage option, while numeric group priority can remain future work.

Compatibility

This proposal is additive.

• Existing clients that do not call RetainGroups behave as before.
• Existing group_ids behavior remains unchanged.
• Existing group membership and grouped-eviction organization remain unchanged.
• Existing read-lease semantics remain unchanged.
• Existing object-level soft-pin semantics remain unchanged.
• Group retention state is stored and evaluated independently from both mechanisms.
• Existing PUT and GET correctness behavior is unaffected.
• Older clients do not need to provide new fields.

Testing Plan

The implementation should add tests covering:

• retaining a group before any members exist;
• writing future members into an actively retained group;
• future members observing only the remaining group lifetime;
• repeated requests extending but not shortening the deadline;
• retention expiry removing the group’s priority contribution;
• reads not extending group retention;
• existence checks not extending group retention;
• repeated PutEnd not extending group retention;
• additional replica completion not extending group retention;
• coexistence with object-level soft pins;
• coexistence with ordinary read leases;
• admission-limit enforcement;
• concurrent admission;
• tenant isolation;
• active groups with no current members;
• behavior under the selected fallback eviction model;
• snapshot recovery behavior, depending on the persistence decision.

Existing behavior without group retention should remain covered by current regression tests.

Rollout Plan

Stage 1: Group deadline state and core API

• Add bounded group-retention configuration.
• Add tenant-scoped group-retention deadlines.
• Port RetainGroups.
• Preserve retention for groups without current members.
• Add deadline expiry cleanup.
• Remove all lease mutations from the proof-of-concept behavior.

Stage 2: Membership and eviction integration

• Connect existing group registration to active retention state.
• Ensure future members observe the existing group deadline.
• Add group retention as an independent soft-priority signal.
• Revalidate retention during final eviction.
• Add interaction tests for leases, object soft pins, and grouped eviction.

Stage 3: Interfaces and observability

• Port RPC and client plumbing.
• Port Python bindings.
• Add retained-group metrics and logging.
• Document recovery behavior.
• Add snapshot persistence later if agreed.

Open Questions

1. Priority representation

Should the first stage use:

• one fixed priority tier for all retained groups; or
• an explicit numeric priority supplied by the caller?

A fixed tier keeps the API narrow. Numeric priority is more flexible but requires broader policy decisions.

2. Composition with object priority

When an object has both object-level and group-level soft priority, should its effective priority be:

• the maximum of the two;
• an ordered combination of independent signals; or
• another rule?

The chosen rule should remain easy to reason about and must not cause either lifetime to renew the other.

3. Fallback eviction

Under severe memory pressure, may actively retained groups eventually be reclaimed?

Possible models include:

• group retention is respected until expiry;
• retained groups are considered only in a fallback eviction pass;
• group retention acts purely as a ranking preference.

This determines whether RetainGroups provides a temporary guarantee or a best-effort preference.

4. Snapshot persistence

Should active group deadlines survive master snapshot recovery?

The proof of concept does not persist them. Keeping them transient is simpler but requires clients to reissue retention after recovery.

Persisting deadlines improves continuity but requires serialization and timestamp semantics.

5. Explicit release

Should a later version support early cancellation through:

• no release mechanism;
• a separate ReleaseGroups API; or
• a cancellation form of RetainGroups?

If several callers may retain the same group independently, release semantics may also require ownership or reference counting.

6. Admission-limit scope

Should the retained-group limit be:

• global;
• per tenant; or
• both?

A global limit is simpler and matches the proof of concept. Per-tenant limits provide stronger isolation.

Proposed First PR Scope

For the first PR, keep the scope narrow:

• retain the existing RetainGroups(group_ids, ttl_ms, tenant_id) API shape;
• store one fixed deadline per tenant/group pair;
• support retention before members exist;
• apply active retention to future members without modifying their object metadata;
• allow explicit calls to extend, but never shorten, the deadline;
• never mutate ordinary read leases;
• keep the group deadline and retention policy as the authoritative source of truth;
• never copy group retention into each member's soft_pin_timeout;
• reuse [Store] Add fixed soft pin lifecycle #2909's fixed-deadline expiry and eviction-priority machinery where practical;
• introduce a fixed retained-group priority tier and integrate it into eviction ranking;
• add expiry cleanup;
• port the relevant RPC, client, and Python interfaces;
• add lifecycle, concurrency, and interaction tests;
• do not add explicit release initially;
• do not persist retention across snapshot recovery initially.

The first PR should keep group/session organization separate from retention policy and leave room for later numeric priorities, release semantics, per-tenant quotas, and persistence.

Before submitting a new issue...

• Make sure you already searched for relevant issues and read the documentation

[github-actions]

Thanks for opening this issue, @g310v!

Field	Value
Issue	#2948
GitHub user ID	88919321
Reporter	@g310v

A maintainer will triage this when possible. To help us respond faster, please include:

• Mooncake version or commit SHA
• Environment (OS, CUDA/driver, RDMA stack if relevant)
• Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.