Fix shallow file scans when merge base objects are missing#4900
Fix shallow file scans when merge base objects are missing#4900lawrence3699 wants to merge 1 commit intotrufflesecurity:mainfrom
Conversation
|
|
There was a problem hiding this comment.
Pull request overview
Fixes failures when scanning shallow local file:// git clones with --since-commit <base> --branch <head> by not aborting when the merge-base object is missing locally (but commit enumeration from head is still possible).
Changes:
- Treat
plumbing.ErrObjectNotFoundfromMergeBaseduringnormalizeConfigas non-fatal and continue with the resolved base ref/hash. - Add an end-to-end regression test that builds a shallow local clone scenario and asserts scanning finds feature-branch content without pulling in
main-only content.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| pkg/sources/git/git.go | Continue scanning when merge-base resolution fails due to missing objects in shallow clones. |
| pkg/sources/git/git_test.go | Add regression coverage for shallow local file:// clone scanning with base/head branch refs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| func TestScanRepo_ShallowLocalCloneWithBranchBaseRef(t *testing.T) { | ||
| ctx := context.Background() | ||
|
|
||
| sourceRepoPath := setupTestRepo(t, "source-repo") | ||
| assert.NoError(t, exec.Command("git", "-C", sourceRepoPath, "branch", "-M", "main").Run()) | ||
| addTestFileAndCommit(t, sourceRepoPath, "base.txt", "base one") | ||
| addTestFileAndCommit(t, sourceRepoPath, "base.txt", "base two") | ||
|
|
||
| assert.NoError(t, exec.Command("git", "-C", sourceRepoPath, "checkout", "-b", "feature").Run()) | ||
| addTestFileAndCommit(t, sourceRepoPath, "feature.txt", "feature secret\nsecret: AKIA1234567890123456\n") | ||
| assert.NoError(t, exec.Command("git", "-C", sourceRepoPath, "checkout", "main").Run()) | ||
| addTestFileAndCommit(t, sourceRepoPath, "main.txt", "main only") | ||
|
|
||
| originRepoPath := filepath.Join(t.TempDir(), "origin.git") | ||
| assert.NoError(t, exec.Command("git", "clone", "--bare", sourceRepoPath, originRepoPath).Run()) | ||
|
|
||
| ciRepoPath := filepath.Join(t.TempDir(), "ci-repo") | ||
| assert.NoError(t, exec.Command("git", "clone", "--depth=1", "--branch", "feature", "file://"+originRepoPath, ciRepoPath).Run()) | ||
| assert.NoError(t, exec.Command("git", "-C", ciRepoPath, "fetch", "--depth=1", "origin", "main:main").Run()) | ||
|
|
||
| preparedRepoPath, _, err := prepareRepoSinceCommit(ctx, "file://"+ciRepoPath, "", "main", false, false) | ||
| assert.NoError(t, err) |
| // If baseCommit is an ancestor of headCommit, update c.BaseRef to be the common ancestor. | ||
| mergeBase, err := headCommit.MergeBase(baseCommit) | ||
| if err != nil { | ||
| // Shallow local clones can omit the common ancestor object even when git can still | ||
| // enumerate the head-side commits to scan. In that case, keep the resolved base ref |
|
is there maybe already a docker image of this branch that I could use to test? |
|
Hi @lawrence3699, |
Description:
Fixes #4895.
When
trufflehog git file://... --since-commit <base> --branch <head>runs against a shallow local clone,normalizeConfigcan fail while resolving the merge base because the common ancestor object is not present locally.In that case, Git can still enumerate the head-side commits to scan, so this change keeps the resolved base ref instead of aborting the scan when
MergeBasereturnsplumbing.ErrObjectNotFound.The new regression test covers the shallow
file://path end to end and asserts that the scan still reports the feature-side change without pulling in themain-only commit.Checklist:
go test ./pkg/sources/git -run TestScanRepo_ShallowLocalCloneWithBranchBaseRef -count=1,go test ./pkg/sources/git -run 'Test(GitConfigSanitization|ScanRepo_ShallowLocalCloneWithBranchBaseRef|GitConfigSanitizationWithBareRepo|GitConfigSecurityIsolation|GitConfigSanitizationWithStagedChanges|PrepareRepoErrorPaths|NormalizeFileURI|PrepareRepoWithNormalization|PrepareRepoWithNormalizationBare)$' -count=1,go test ./pkg/sources/git -run 'TestSource_Chunks_Integration/remote repo, main ahead of branch' -count=1)./scripts/lint.sh)Note
Medium Risk
Changes commit-range normalization for
--since-commitscans: shallow clones that can’t load merge-base objects will now continue with the resolved base ref, which could subtly affect which commits are included/excluded. Covered by a new end-to-end regression test for shallowfile://clones.Overview
Fixes shallow local
file://scans wherenormalizeConfigpreviously aborted ifMergeBasefailed withplumbing.ErrObjectNotFound; it now treats this as non-fatal and proceeds using the resolved base ref.Adds
TestScanRepo_ShallowLocalCloneWithBranchBaseRefto reproduce the CI-style shallow clone scenario and assert the scan still reports feature-branch content without pulling inmain-only commits.Reviewed by Cursor Bugbot for commit 5c91e5a. Bugbot is set up for automated code reviews on this repo. Configure here.