feat(spark-3.5): [stacked] Add OPTIMIZE SQL extension (bin-pack + incremental clustering)

[mkuchenbecker]

OpenHouse spark-3.5 SQL Extensions Stack

PR	Base	Content
#660	main	Standalone refactor (decouple spark-3.5 from spark-3.1)
vacuum	#660	VACUUM
(this)	#660	OPTIMIZE (bin-pack + incremental clustering)
analyze	(this)	ANALYZE … COMPUTE CLUSTERING QUALITY

Depends on #660. Independent of the VACUUM PR — bases directly on the standalone
refactor. The ANALYZE PR bases on this one. Review/merge #660 first.

Summary

Adds an OPTIMIZE command to the OpenHouse spark-3.5 SQL extensions:

OPTIMIZE <table> [FULL] [REWRITE MANIFESTS]

• No optimize.cluster.keys → plain bin-pack compaction (system.rewrite_data_files);
  unchanged historical behavior.
• Clustering configured → sort / z-order rewrite of the configured keys, incremental
  by default (only the forward slice of the leading key since the last run, tracked by an
  optimize.cluster.hwm-snapshot-id watermark); FULL reclusters up to the age floor.
  Snapshots younger than optimize.cluster.min-snapshot-age-minutes are held back;
  Iceberg partial progress bounds the snapshots committed per run.
• Durable clustering state (optimize.cluster.config-id + optimize.cluster.state
  JSON intervals) advances atomically with the watermark in a single alterTable, so it
  survives snapshot expiration.
• REWRITE MANIFESTS (system.rewrite_manifests) runs afterwards over the new layout.
• Emits files_before / files_after / files_removed / snapshots_committed rows.

Thin sugar over the CALL path (OptimizeTableExec), with an OpenHouse-table guard
(openhouse.tableId). The extension parser wrapper now implements Iceberg's
ExtendedParser and delegates parseSortOrder, so the sort / z-order sort_order
argument parses (the wrapper is the session's outermost parser).

Changes

• Client-facing API Changes
• Internal API Changes
• Bug Fixes
• New Features
• Performance Improvements
• Code Style
• Refactoring
• Documentation
• Tests

New non-reserved keywords OPTIMIZE / FULL / REWRITE / MANIFESTS.

Testing Done

• Added new tests for the changes made.

• OptimizeTableTest (pure helpers: parseClusterConfig / configId / parseState /
  advanceState) — 9 tests.

• OptimizeStatementTest (real Iceberg, Hadoop catalog): bin-pack reduction, clustering
  writes durable state, incremental second run is a no-op, REWRITE MANIFESTS, FULL,
  non-OpenHouse rejection — 6 tests.

./gradlew :integrations:spark:spark-3.5:openhouse-spark-3.5-runtime_2.12:test            # 9/9
./gradlew :integrations:spark:spark-3.5:openhouse-spark-3.5-itest:statementTest --tests '*OptimizeStatementTest'   # 6/6
# both BUILD SUCCESSFUL

Additional Information

• Large PR broken into smaller PRs, and PR plan linked in the description.