Skip to content

Generation state does not track complypack digests #583

@jpower432

Description

@jpower432

Problem

GenerationState tracks PolicyDigest to detect when policy artifacts need regeneration, but does not track complypack digests. If a user pulls a new complypack via complyctl get without changing the policy, checkGenerationFreshness sees a fresh policy digest and skips regeneration. Providers continue using artifacts generated from the old complypack.

This also affects step identity in evaluation logs: state.json reflects the new complypack's repository@digest, but the scan results are based on content generated from the old complypack.

Steps to reproduce

  1. complyctl get — pulls policy + complypack, generates artifacts
  2. Update the complypack in the registry (new content, same evaluator-id)
  3. complyctl get — pulls new complypack, updates state.json with new digest
  4. complyctl scan — generation skipped (policy unchanged), providers use stale artifacts

Proposed fix

Add complypack digests to GenerationState and check them in needsRegeneration. When any complypack digest changes, trigger regeneration.

type GenerationState struct {
    PolicyID          string            `json:"policy_id"`
    PolicyDigest      string            `json:"policy_digest"`
    ComplypackDigests map[string]string `json:"complypack_digests,omitempty"` // evaluatorID → digest
    GeneratedAt       string            `json:"generated_at"`
    EvaluatorIDs      []string          `json:"evaluator_ids"`
}

Related

  • PR feat: populate assessment plan and step identity in evaluation logs #579 — step identity uses state.json for complypack repository@digest, which can drift from what was actually used during generation
  • A separate enhancement (co-locating OCI provenance in the cache directory) would improve data locality but does not fix this issue on its own — generation invalidation is necessary regardless

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No fields configured for Bug.

    Projects

    Status
    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions