Skip to content

FIX: Have SeedCorpus._original_index track correct indices#105

Merged
yuleisui merged 1 commit into
SVF-tools:mainfrom
hanyuone:seedcorpus-dedup
Jul 21, 2026
Merged

FIX: Have SeedCorpus._original_index track correct indices#105
yuleisui merged 1 commit into
SVF-tools:mainfrom
hanyuone:seedcorpus-dedup

Conversation

@hanyuone

@hanyuone hanyuone commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

When using SeedCorpus, we give it a set of seeds that corresponds 1-to-1 to a batched VerifiableModel's labeled_input (as in this example from ipynb/vnnlib_fuzzer.ipynb):

initial_seeds = [
    LabeledInputTensor(tensor=images[i:i+1], label=labels[i:i+1])
    for i in range(B)
]

However, SeedCorpus does deduplicating, and then stores self._original_index as the indices of the deduplicated list of seeds, instead of the original. For example, if we set the number of samples to 4, the images in labeled_input would correspond to images [0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3]. However, _original_index = [0, 1, 2, 3] because of deduping.

This means that there is no longer a 1-to-1 correspondence between the indices in _original_index and the batched input spec, which means when we fuzz with BOX input specs, we use the wrong lb and ub. Input tensors now "blend together" when fuzzing:

image

To replicate this bug, run the following Python file as a Jupyter notebook, in the ipynb folder of ACT: https://gist.github.com/hanyuone/8a551837e90091ac720aa89c9f397c1c

This PR fixes the above bug by keeping track of original indices when deduplicating.

@codecov

codecov Bot commented Jul 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.51%. Comparing base (a5bf2e2) to head (e040b5b).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #105   +/-   ##
=======================================
  Coverage   73.51%   73.51%           
=======================================
  Files          91       91           
  Lines       20506    20508    +2     
=======================================
+ Hits        15075    15077    +2     
  Misses       5431     5431           
Flag Coverage Δ
bab 45.71% <8.33%> (-0.01%) ⬇️
backend-float32 49.52% <8.33%> (-0.01%) ⬇️
backend-float64 49.56% <8.33%> (-0.01%) ⬇️
frontend 32.91% <100.00%> (+<0.01%) ⬆️
pipeline-fuzz 20.52% <100.00%> (+<0.01%) ⬆️
pipeline-verify 39.47% <8.33%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
act/pipeline/fuzzing/corpus.py 87.96% <100.00%> (+0.22%) ⬆️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a5bf2e2...e040b5b. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@guanqin-123

guanqin-123 commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

it is good to me.

@hanyuone
hanyuone marked this pull request as ready for review July 21, 2026 00:51
@yuleisui
yuleisui merged commit df5bdf4 into SVF-tools:main Jul 21, 2026
8 checks passed
@hanyuone
hanyuone deleted the seedcorpus-dedup branch July 21, 2026 05:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants