Skip to content

CLID-614: Test for Operator incremental mirroring#1413

Open
nidangavali wants to merge 1 commit into
openshift:mainfrom
nidangavali:CLID-614
Open

CLID-614: Test for Operator incremental mirroring#1413
nidangavali wants to merge 1 commit into
openshift:mainfrom
nidangavali:CLID-614

Conversation

@nidangavali

@nidangavali nidangavali commented May 20, 2026

Copy link
Copy Markdown
Contributor

Description

Add an integration test for operator incremental mirroring (mirrorToDisk). The test verifies that oc-mirror produces archives containing only new content on subsequent runs when the ImageSetConfig changes.

The test runs mirrorToDisk twice against the same workspace with different ISCs:

  • Step 1: mirrors operator foo pinned at version 0.2.0
  • Step 2: expands foo to version range 0.2.0–0.3.1 and adds a new package bar (stable channel)

It then validates that the second archive contains only the incremental data,new blobs from the expanded version range and the newly added package with no overlap from the first archive.

Github / Jira issue: CLID-614

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Code Improvements (Refactoring, Performance, CI upgrades, etc)
  • Internal repo assets (diagrams / docs on github repo)
  • This change requires a documentation update on openshift docs

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration.

Expected Outcome

  • Step 1 produces a tar archive containing blobs for foo at version 0.2.0
  • Step 2 produces a tar archive containing only new blobs (additional foo versions and bar package)
  • No blob from the initial archive appears in the incremental archive
  • The initial archive does not contain any bar entries
  • The incremental archive contains the expected repositories from the updated ISC

Summary by CodeRabbit

Summary by CodeRabbit

  • Tests
    • Added new integration-test helper utilities to log mirror results and analyze tar archives (including blob and repository-level summaries).
    • Added new operator incremental-mirroring test fixtures for step-based scenarios.
    • Expanded integration coverage to verify that repeated mirroring runs produce archives containing only newly introduced blobs and expected repository contents.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 20, 2026
@openshift-ci-robot

openshift-ci-robot commented May 20, 2026

Copy link
Copy Markdown

@nidangavali: This pull request references CLID-614 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Github / Jira issue:

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Code Improvements (Refactoring, Performance, CI upgrades, etc)
  • Internal repo assets (diagrams / docs on github repo)
  • This change requires a documentation update on openshift docs

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration.

Expected Outcome

Please describe the outcome expected from the tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci
openshift-ci Bot requested review from adolfo-ab and aguidirh May 20, 2026 11:12
@coderabbitai

coderabbitai Bot commented May 20, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 1677acc3-335a-4015-bcce-ad6c2649d9a1

📥 Commits

Reviewing files that changed from the base of the PR and between 75c6e8e and 2f5faab.

📒 Files selected for processing (4)
  • tests/integration/helpers_test.go
  • tests/integration/incremental_mirroring_test.go
  • tests/integration/testdata/imagesetconfigs/operators/isc-operator-incremental-initial.yaml
  • tests/integration/testdata/imagesetconfigs/operators/isc-operator-incremental-update.yaml
✅ Files skipped from review due to trivial changes (2)
  • tests/integration/testdata/imagesetconfigs/operators/isc-operator-incremental-update.yaml
  • tests/integration/testdata/imagesetconfigs/operators/isc-operator-incremental-initial.yaml
🚧 Files skipped from review as they are similar to previous changes (2)
  • tests/integration/helpers_test.go
  • tests/integration/incremental_mirroring_test.go

Walkthrough

Adds integration test for operator config-based incremental mirroring (CLID-614). Two ISC YAML fixtures define initial and update mirror configurations for operator foo and bar. Four new helper functions support tar archive inspection and result logging. A new two-step Ginkgo test runs MirrorToDisk with each ISC against the same workspace and asserts that the incremental tar contains only blob paths not present in the initial tar.

Changes

Operator Incremental Mirroring Integration Test (CLID-614)

Layer / File(s) Summary
ISC test fixtures for initial and update steps
tests/integration/testdata/imagesetconfigs/operators/isc-operator-incremental-initial.yaml, tests/integration/testdata/imagesetconfigs/operators/isc-operator-incremental-update.yaml
Step-1 fixture pins operator foo to the beta channel at version 0.2.0; step-2 fixture expands foo's version range to 0.2.00.3.1 and adds operator bar on the stable channel.
Tar inspection and result logging helpers
tests/integration/helpers_test.go
Adds logOcMirrorResult (exit code, duration, stdout/stderr), logTarSummary (size, entry count, blob count, repository prefixes), collectTarBlobPaths (normalized entry names under a prefix), and expectTarDoesNotContainPath (substring assertion).
CLID-614 test implementation and documentation updates
tests/integration/incremental_mirroring_test.go
File-level and CLID-655 comments updated for clarity. New Describe("operator incremental mirrorToDisk") block runs MirrorToDisk twice, preserves the step-1 tar, and asserts the step-2 tar's blob paths are disjoint from step-1; also verifies step-1 tar excludes /bar.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Suggested reviewers

  • adolfo-ab
🚥 Pre-merge checks | ✅ 13 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Test Structure And Quality ⚠️ Warning The operator incremental test lacks SpecTimeout, has 2 Expect statements without failure messages (lines 125, 148), and no timeout on long-running MirrorToDisk calls. Add SpecTimeout(5*time.Minute) to It() block, add messages to Expect(err).NotTo(HaveOccurred()) calls on lines 125 and 148, ensuring consistency with the --since test above.
Ipv6 And Disconnected Network Test Compatibility ⚠️ Warning Test requires external connectivity to quay.io (public registry) and lacks [Skipped:Disconnected] marker for disconnected environments. Add [Skipped:Disconnected] to the test name or use a local/mirrored test catalog: e.g., It("should produce a second tar with only incremental blob content [Skipped:Disconnected]", func() { ... })
✅ Passed checks (13 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically identifies the main change: adding a test for operator incremental mirroring (CLID-614), which is the primary focus of the changeset.
Docstring Coverage ✅ Passed Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed All 5 Ginkgo test declarations in incremental_mirroring_test.go use stable, descriptive, static test names with no dynamic content (no pod/node/namespace names, timestamps, UUIDs, or generated suff...
Microshift Test Compatibility ✅ Passed The new tests in this PR are integration tests (tests/integration/incremental_mirroring_test.go) that test the oc-mirror CLI tool in isolation with a local test registry. They do not interact with...
Single Node Openshift (Sno) Test Compatibility ✅ Passed These are oc-mirror integration tests that run locally against a test registry, not OpenShift e2e tests. They do not interact with any cluster infrastructure and make no multi-node assumptions; SNO...
Topology-Aware Scheduling Compatibility ✅ Passed PR adds only integration tests and test fixtures (helpers_test.go, incremental_mirroring_test.go, and test data YAML files). No deployment manifests, operator code, or controllers are introduced. C...
Ote Binary Stdout Contract ✅ Passed No OTE Binary Stdout Contract violations found. All output uses GinkgoWriter (correct), called only from test execution blocks (It), with no direct stdout writes (fmt.Print, log.Print, klog, or os....
No-Weak-Crypto ✅ Passed PR adds test code only. No weak cryptography (MD5, SHA1, DES, RC4, 3DES, Blowfish, ECB), custom crypto implementations, or non-constant-time secret comparisons found. SHA256 references are OCI dige...
Container-Privileges ✅ Passed No Kubernetes container manifests with privileged settings, host access, or capability escalation found. PR contains only test code and oc-mirror ImageSetConfiguration fixtures.
No-Sensitive-Data-In-Logs ✅ Passed PR adds test helpers logging oc-mirror exit codes, durations, stdout/stderr, tar archive metrics, and OCI repository names from public test registries. No sensitive data like passwords, tokens, API...

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/incremental_test.go`:
- Around line 36-38: The test currently hardcodes "mirror_000001.tar" (assigned
to initialTar) which can break if sequence naming changes; replace the hardcoded
lookup by discovering produced mirror tar files in workDir with a glob like
"mirror_*.tar", assert the glob returns at least one match, sort the matches (by
filename or modtime) to pick the intended archive (first/earliest for initial,
next for subsequent), assign that path to initialTar (and the later tar variable
used around logTarSummary), and then call logTarSummary with the discovered
path; update any other hardcoded uses of "mirror_00000X.tar" similarly.
- Around line 68-70: The current assertion using
Expect(incrementalBlobs).NotTo(ConsistOf(initialBlobs)) only proves the sets
differ; change the test to assert the incremental tar contains none of the
previously mirrored blobs by verifying there is no intersection between
incrementalBlobs and initialBlobs. Replace the ConsistOf-based assertion with
one that checks incrementalBlobs does not contain any element from initialBlobs
(e.g., assert the intersection length is zero or use a Gomega matcher that
ensures no elements from initialBlobs appear in incrementalBlobs) referencing
the incrementalBlobs and initialBlobs variables and the Expect call.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d60a918f-b59b-49dd-aa3e-496c58791e63

📥 Commits

Reviewing files that changed from the base of the PR and between edb3e8c and 94a346b.

📒 Files selected for processing (4)
  • tests/integration/helpers_test.go
  • tests/integration/incremental_test.go
  • tests/integration/testdata/imagesetconfigs/operators/isc-operator-incremental-initial.yaml
  • tests/integration/testdata/imagesetconfigs/operators/isc-operator-incremental-update.yaml

Comment thread tests/integration/incremental_test.go Outdated
Comment thread tests/integration/incremental_test.go Outdated
@nidangavali

Copy link
Copy Markdown
Contributor Author

/retest

@aguidirh aguidirh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, I just added few comments to it.

Please add a PR description following the template on .github directory and if possible also a commit message with context about the changes for future reference.

Comment thread tests/integration/incremental_test.go Outdated
Comment thread tests/integration/incremental_test.go Outdated
Comment thread tests/integration/incremental_test.go Outdated
Comment thread tests/integration/incremental_test.go Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/incremental_mirroring_test.go`:
- Around line 147-150: The deferred cleanup call to os.RemoveAll(preserveDir)
does not capture or handle its error return value. Modify the defer statement to
use a closure that captures the error returned by os.RemoveAll and logs or
handles it appropriately, ensuring that any cleanup failures are not silently
ignored per the coding guidelines that require never ignoring error returns.
- Line 122: The test "should produce a second tar with only incremental blob
content" is a long-running integration test that lacks a timeout and needs to
pass SpecContext to its mirroring operations. Add a SpecTimeout parameter to the
It() function call to prevent test suite hangs, and update all MirrorToDisk
operation calls within this test (and the related calls at lines 134-135 and
155-156) to accept and use the SpecContext parameter instead of the outer ctx
variable to ensure proper test context handling within Ginkgo.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5907f313-9121-42bd-a4c1-62b6c40967ac

📥 Commits

Reviewing files that changed from the base of the PR and between a84bd83 and 75c6e8e.

📒 Files selected for processing (4)
  • tests/integration/helpers_test.go
  • tests/integration/incremental_mirroring_test.go
  • tests/integration/testdata/imagesetconfigs/operators/isc-operator-incremental-initial.yaml
  • tests/integration/testdata/imagesetconfigs/operators/isc-operator-incremental-update.yaml
✅ Files skipped from review due to trivial changes (1)
  • tests/integration/testdata/imagesetconfigs/operators/isc-operator-incremental-initial.yaml
🚧 Files skipped from review as they are similar to previous changes (2)
  • tests/integration/testdata/imagesetconfigs/operators/isc-operator-incremental-update.yaml
  • tests/integration/helpers_test.go

iscInitial := filepath.Join("operators", "isc-operator-incremental-initial.yaml")
iscUpdate := filepath.Join("operators", "isc-operator-incremental-update.yaml")

It("should produce a second tar with only incremental blob content", func() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify this spec currently lacks SpecTimeout and uses non-spec context in MirrorToDisk calls.
rg -nP --type=go -C2 'It\("should produce a second tar with only incremental blob content",\s*func\(' tests/integration/incremental_mirroring_test.go
rg -nP --type=go -C2 'runner\.MirrorToDisk\(' tests/integration/incremental_mirroring_test.go

Repository: openshift/oc-mirror

Length of output: 1647


Add timeout to long-running incremental mirroring test and pass SpecContext to mirroring calls.

Line 122 defines a long-running integration test with two MirrorToDisk operations but no SpecTimeout, creating risk of suite hangs. Additionally, the test should pass Ginkgo's SpecContext to the mirroring operations instead of the outer ctx.

Suggested fix
-		It("should produce a second tar with only incremental blob content", func() {
+		It("should produce a second tar with only incremental blob content", SpecTimeout(10*time.Minute), func(specCtx SpecContext) {
@@
-			result, err := runner.MirrorToDisk(ctx, iscInitialPath, workDir, "--remove-signatures=true")
+			result, err := runner.MirrorToDisk(specCtx, iscInitialPath, workDir, "--remove-signatures=true")
@@
-			result, err = runner.MirrorToDisk(ctx, iscUpdatePath, workDir, "--remove-signatures=true")
+			result, err = runner.MirrorToDisk(specCtx, iscUpdatePath, workDir, "--remove-signatures=true")

Also applies to lines 134–135, 155–156.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/incremental_mirroring_test.go` at line 122, The test
"should produce a second tar with only incremental blob content" is a
long-running integration test that lacks a timeout and needs to pass SpecContext
to its mirroring operations. Add a SpecTimeout parameter to the It() function
call to prevent test suite hangs, and update all MirrorToDisk operation calls
within this test (and the related calls at lines 134-135 and 155-156) to accept
and use the SpecContext parameter instead of the outer ctx variable to ensure
proper test context handling within Ginkgo.

Source: Coding guidelines

Comment thread tests/integration/incremental_mirroring_test.go

@adolfo-ab adolfo-ab left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 23, 2026
@openshift-ci

openshift-ci Bot commented Jul 7, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adolfo-ab, aguidirh, nidangavali

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 7, 2026
@nidangavali

Copy link
Copy Markdown
Contributor Author

/verified by @nidangavali

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Jul 7, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@nidangavali: This PR has been marked as verified by @nidangavali.

Details

In response to this:

/verified by @nidangavali

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD bc220b6 and 2 for PR HEAD 2f5faab in total

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD d31e7f4 and 1 for PR HEAD 2f5faab in total

@nidangavali

Copy link
Copy Markdown
Contributor Author

/retest

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD 4cbc362 and 0 for PR HEAD 2f5faab in total

@openshift-ci

openshift-ci Bot commented Jul 13, 2026

Copy link
Copy Markdown

@nidangavali: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/images 2f5faab link true /test images
ci/prow/integration 2f5faab link true /test integration

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

/hold

Revision 2f5faab was retested 3 times: holding

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants