Skip to content

Detect duplicate JSON keys in flow files#78

Merged
sfc-gh-pvillard merged 1 commit into
mainfrom
feature/duplicate-key-detection
Jul 9, 2026
Merged

Detect duplicate JSON keys in flow files#78
sfc-gh-pvillard merged 1 commit into
mainfrom
feature/duplicate-key-detection

Conversation

@sfc-gh-pvillard

Copy link
Copy Markdown
Collaborator

Closes #77.

Summary

  • Pre-validation before parsing: both flow files are scanned with a streaming JSON parser that has STRICT_DUPLICATE_DETECTION enabled before any POJO deserialization. A dedicated validateNoDuplicateKeys() method handles this using a separate JsonFactory so the existing parsing configuration is untouched.
  • Clear PR comment error: when a duplicate key is found, a [!CAUTION] block is posted with the file path and exact line/column, matching the style of the existing checkstyle violation block.
  • Action exits with code 1: the PR check is blocked until the file is fixed, preventing a silently-wrong diff from giving false confidence to reviewers.
  • snapshotA catch tightened: the overly broad catch (Exception e) for the "no original flow" case is narrowed to catch (IOException e), so duplicate-key errors in flowA are now reported rather than silently treated as a first-version flow.

Example PR comment on detection

### Executing Snowflake Flow Diff for flow: `submitted-changes/flows/my-flow.json`

> [!CAUTION]
> Flow file `submitted-changes/flows/my-flow.json` contains duplicate JSON keys (this typically indicates a merge conflict that was not fully resolved): Duplicate field 'flowContents'
> Line 37, column 17

Test plan

  • All 27 tests pass (mvn test)
  • testDuplicateKeyInFlowBThrowsException - getDiff() throws JsonParseException when flowB has a duplicate key
  • testDuplicateKeyInFlowAThrowsException - getDiff() throws JsonParseException when flowA has a duplicate key
  • testDuplicateKeyReturnsFailureExitCode - run() returns exit code 1 when a duplicate key is present
  • testDuplicateKeyOutputContainsCaution - PR comment output contains the CAUTION callout and the duplicate field name

@sfc-gh-dchaffelson sfc-gh-dchaffelson left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a clean one, Pierre — and a genuinely nasty failure mode to close off. Validating both files up front with a dedicated strict-duplicate parser is the right call: since JsonParseException is an IOException, relying on the deserialization to surface it would have let a duplicate in the before file get swallowed by the "no original flow" handler, so doing the explicit pre-scan before that try/catch is exactly right. The [!CAUTION] block with file + line/column matches the existing style nicely, and I like that it fails the check (exit 1) so a silently-wrong diff can't give false confidence.

Test coverage is spot on — both flowA and flowB directions, the exit code, and the CAUTION output, with a realistic double-flowContents fixture.

Two small non-blocking notes:

  1. Heads-up on merge ordering. This, #76, and #66 all touch FlowDiff.java (and #76 also reworks getDiff / executeFlowDiffForOneFlow), so they'll conflict with each other on merge, not just with main. Worth a rebase whichever lands last.
  2. Minor cosmetic: the ### Executing ... for flow: header prints the after path even when the duplicate is in the before file. The message body names the right file, so it's low-impact — just noting it.

Nice fix — LGTM.

@sfc-gh-dchaffelson sfc-gh-dchaffelson left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — clean fix for a nasty silent-failure mode. Notes in my comment (merge ordering, cosmetic header) are non-blocking.

Resolves #77. Before computing a diff, both flow files are now scanned
with a streaming JSON parser that has STRICT_DUPLICATE_DETECTION enabled.
If a duplicate key is found the action posts a CAUTION callout in the PR
comment with the file path and exact line/column, then exits with code 1
so the PR check is blocked until the file is fixed.

- Add validateNoDuplicateKeys() which uses a separate JsonFactory with
  STRICT_DUPLICATE_DETECTION; re-wraps JsonParseException with the file
  path for a self-contained error message.
- Call it at the start of getDiff() for both pathA (skipped when absent)
  and pathB.
- Catch JsonParseException in executeFlowDiffForOneFlow() and print the
  CAUTION block; set jsonParseError flag for run() to track.
- Track jsonParseError per flow in run() and return RETURN_FAILURE if
  any parse error occurred.
- Tighten the catch for snapshotA from Exception to IOException.
- Add test fixture flow_v9_duplicate_key.json and four new tests.

@sfc-gh-dchaffelson sfc-gh-dchaffelson left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving on the rebased head — duplicate-key detection is intact and the base is now current after the sibling PRs merged. LGTM.

@sfc-gh-pvillard sfc-gh-pvillard merged commit 2de65d3 into main Jul 9, 2026
3 checks passed
@sfc-gh-pvillard sfc-gh-pvillard deleted the feature/duplicate-key-detection branch July 9, 2026 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Detect duplicate JSON keys in flow files

2 participants