[vpj] Preserve KIF input config precedence

[pthirun]

Problem Statement

Spark Kafka-input repush forwarded all original job properties after applying normalized inputs. A stale source broker, source key schema, or Kafka Message Envelope schema could therefore replace the controller-derived value and make Spark diverge from MapReduce behavior.

Solution

Forward general job properties first, then apply the resolved source broker and controller-derived key and Kafka Message Envelope schemas. This preserves custom adapter and other pass-through properties while keeping authoritative normalized values final.

Code changes

• Added new code behind a config. If so list the config names and their default values in the PR description.
• Introduced new log lines.
  • Confirmed if logs need to be rate limited to avoid excessive logging.

Concurrency-Specific Checks

Both reviewer and PR author to verify

• Code has no race conditions or thread safety issues.
• Proper synchronization mechanisms (e.g., synchronized, RWLock) are used where needed.
• No blocking calls inside critical sections that could lead to deadlocks or performance degradation.
• Verified thread-safe collections are used (e.g., ConcurrentHashMap, CopyOnWriteArrayList).
• Validated proper exception handling in multi-threaded code to avoid silent thread termination.

How was this PR tested?

• Local code review completed.
• New unit tests added.
• New integration tests added.
• Modified or extended existing tests.
• Verified backward compatibility (if applicable).

Ran the focused DataWriterSparkJobTest suite, including stale and empty source key schemas, conflicting broker and KME schemas, and custom property forwarding. Ran the repository Spotless check.

Does this PR introduce any user-facing or breaking changes?

• No. You can skip the rest of this section.
• Yes. Clearly explain the behavior change and its impact.

🤖 Generated with GitHub Copilot CLI