Expose node RealMemory without controller RPC

[fabrizio2210]

Summary

• render the worker's Slurm RealMemory value into an authoritative container environment variable
• publish that value atomically to node-local /run/soperator/node_metadata.env
• make CHECKS_NODE_REAL_MEM_BYTES prefer the local metadata instead of querying the Slurm controller
• retain the existing controller lookup as a compatibility fallback during rolling upgrades
• add Go, shell behavior, metadata-writer, and check-runner coverage

Why

Periodic health checks need the node's allocatable Slurm memory without issuing scontrol show node on every run. The value must come from the same worker resource configuration used to render Slurm RealMemory, so it cannot drift from the node's scheduling configuration.

This is a prerequisite for #2773 and is related to SCHED-1897, but it is kept in a separate PR because local node metadata is an independently useful feature.

Dependency

This PR is based directly on main. The dependent idle-memory health check is stacked separately in #2773 and should be merged after this PR.

#2742 was accidentally merged into #2743. The branch history was rebuilt to restore the independent RPC-free RealMemory scope, but GitHub retained #2743's old pull ref and would not reopen it after the force-push. This PR therefore replaces #2743, while #2773 carries the replacement idle-memory feature diff.

Impact

Workers publish the MiB-rounded Slurm RealMemory value in bytes through a root-owned, atomically replaced metadata file on the existing ephemeral /run volume. Existing checks continue to request CHECKS_NODE_REAL_MEM_BYTES unchanged. Workers that have not yet published the file use the existing Slurm controller lookup, preserving rolling-upgrade compatibility.

Validation

Local validation after rebuilding the branch on current main:

• 8 Slurm-script unit tests passed
• go test ./internal/render/worker ./internal/render/common
• bash -n images/worker/write_soperator_metadata.sh images/worker/slurmd_entrypoint.sh
• git diff --check main...HEAD

Tested previously on a dev cluster:

• both workers contained /run/soperator/node_metadata.env with the same value, permissions 0644, owned by root:root
• the metadata writer was installed as executable and invoked by slurmd_entrypoint.sh
• the deployed check_runner.py contained both the local metadata lookup and controller fallback
• the lookup on both workers logged Node RealMemory from /run/soperator/node_metadata.env and returned 119185342464, confirming the controller fallback was not used
• scontrol show nodes -o reported RealMemory=113664 MiB for both nodes; 113664 × 1048576 = 119185342464 bytes, an exact match