Skip to content

Commit db0ad81

Browse files
committed
Merge sprint-8-os-enforced: OS-enforced validation (Layer A vs Layer B)
Merge sprint-8-os-enforced: OS-enforced validation
2 parents 7229d07 + 636fc58 commit db0ad81

8 files changed

Lines changed: 666 additions & 3 deletions

File tree

.github/workflows/edge-validation.yml

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,5 +26,10 @@ jobs:
2626
- name: Real RLIMIT_AS enforcement tests (Layer B, Linux)
2727
run: pytest tests/unit/test_linux_runner.py -v
2828

29-
- name: Layer A vs Layer B memory enforcement (fixture repo, mem-256)
30-
run: python scripts/edge_validation.py
29+
- name: Layer A vs Layer B memory sweep (256 / 512 / 768 / 2048 / 16384 MB)
30+
run: |
31+
for mb in 256 512 768 2048 16384; do
32+
echo "######## CC_EDGE_BUDGET_MB=${mb} MB ########"
33+
CC_EDGE_BUDGET_MB=$mb python scripts/edge_validation.py
34+
echo ""
35+
done

docs/benchmark-results.md

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# CodeCompass benchmark results
2+
3+
Per-run reports live alongside this file:
4+
5+
- `docs/benchmark-results-baseline.md` / `-rerun.md` — real kubeedge/ianvs baseline
6+
(bge-small + Ollama), plus a run-to-run variance check.
7+
- `docs/benchmark-results-edge.md` — the Sprint 7 edge-constraint sweep (Layer A,
8+
application-level shaping).
9+
10+
## Layer A vs Layer B agreement
11+
12+
**What is compared.** The same fixture benchmark — the `tests/fixtures/sample_repo` repo
13+
with the `data/benchmarks/fixture_qa.yaml` gold set, a **stub LLM provider**, and the real
14+
**bge-small** embedder — is run under a **256 MB memory budget** two ways:
15+
16+
- **Layer A** — the portable, application-level `MemoryGovernor` (Sprint 7): a peak-RSS cap
17+
checked in-process (`constraints/memory.py`). Runs anywhere, including macOS.
18+
- **Layer B** — a real OS address-space cap, `resource.setrlimit(RLIMIT_AS, 256 MB)`,
19+
applied to a forked subprocess running the same workload (`constraints/linux_runner.py`).
20+
**Linux only**: macOS cannot enforce `RLIMIT_AS` (PRD §4.4), so on a Mac it raises
21+
`UnsupportedPlatformError` instead of pretending to enforce anything.
22+
23+
This runs in CI via the manual `edge-validation` workflow
24+
(`.github/workflows/edge-validation.yml`).
25+
26+
**Why the fixture + stub LLM, not a live ianvs re-ingest.** Speed and reliability: the
27+
fixture indexes in seconds and the stub provider removes any Ollama/network dependency, so
28+
the validation job is deterministic and needs no model server in CI.
29+
30+
**Result.**
31+
32+
**Memory-budget sweep (CI run 28006463855; fixture repo + stub LLM + bge-small).** Layer A =
33+
`MemoryGovernor` peak-RSS cap; Layer B = real `RLIMIT_AS` on a forked subprocess (kernel-
34+
confirmed via `getrlimit`). "Stage" is where Layer B failed.
35+
36+
| budget | Layer A (peak RSS) | Layer B (real RLIMIT_AS) | agree? |
37+
| ------ | ------------------ | ------------------------ | ------ |
38+
| 256 MB | FAIL — peak 1076 MB | FAIL @ **import**`regex/_regex…so: failed to map segment from shared object` ||
39+
| 512 MB | FAIL — peak 1069 MB | FAIL @ **import**`libtorch_cpu.so: failed to map segment from shared object` ||
40+
| 768 MB | FAIL — peak 1066 MB | FAIL @ **import**`libtorch_cpu.so: failed to map segment from shared object` ||
41+
| **2048 MB** | **PASS — peak 1060 MB** | **FAIL @ import**`libtorch_cpu.so: failed to map segment from shared object` | **❌ DISAGREE** |
42+
| ≈unconstrained (16384 MB) | PASS — peak 1058 MB | PASS — imported, loaded weights, ran ||
43+
44+
The dependency-free control (`bytearray(400 MB)`) is enforced (clean `MemoryError`) at
45+
256/512/768 and completes at 16384, confirming `RLIMIT_AS` itself works. Layer A and Layer B
46+
agree at every budget **except 2048 MB — the RSS-vs-address-space crossover** (below).
47+
48+
**The citable finding — this stack's import-time address-space floor.** Layer B never reaches
49+
model loading at 256/512/768; it dies during *import*, and the binding cost is PyTorch's
50+
native library, not the model weights:
51+
52+
- **< 256 MB:** can't even mmap the small `regex` C extension.
53+
- **256 → 768 MB:** `regex` maps, but **`libtorch_cpu.so` (PyTorch's core native library)
54+
cannot be mmap'd** — the floor just to *import torch* sits above 768 MB of virtual address
55+
space.
56+
- **(768 MB, 16384 MB]:** imports succeed, the model loads, and the workload runs cleanly.
57+
58+
So for this `sentence-transformers` / `torch` stack the practical memory floor is set by
59+
**importing libtorch**, well before model loading or inference.
60+
61+
**The RSS-vs-address-space crossover (2048 MB), confirmed.** At 2048 MB Layer A **passes**
62+
(peak RSS 1060 MB fits comfortably) while Layer B **fails to even import** (`libtorch_cpu.so`
63+
cannot be `mmap`'d). This is the concrete divergence: *a process can have plenty of headroom
64+
in actual memory use (~1.06 GB resident) yet still fail to even start, because reserving
65+
address space to memory-map large shared libraries (libtorch) is a different resource than
66+
using memory.* Below ~1.08 GB both layers fail (the RSS peak and the import floor both exceed
67+
the budget); above libtorch's virtual floor (by 16384 MB) both pass; in the gap at 2048 MB
68+
only the OS-enforced address-space cap (Layer B) rejects the workload, while the RSS-based
69+
Layer-A simulator green-lights it.
70+
71+
**First CI run + diagnosis — do not trust its headline "DISAGREE".** The first
72+
`edge-validation` run printed Layer B `outcome=error, enforced=False` and "DISAGREE". That
73+
was a *reporting bug in `linux_runner`, not a real enforcement failure*:
74+
75+
- `outcome=error` means the capped child raised a **non-`MemoryError`** exception; the old
76+
code only treated `MemoryError`/kill as "enforced" and never surfaced the exception, so a
77+
genuine under-budget failure was mislabeled "OK".
78+
- The **1087 MB came from the Layer A path — the *unconstrained* parent**, not the capped
79+
Layer-B child. RSS can never exceed virtual size, so a real 256 MB `RLIMIT_AS` fails the
80+
child *long before* 1087 MB; that number never contradicted the cap. (The earlier "they
81+
agree on outcome, RSS vs address-space" note was hand-waving and has been removed.)
82+
- Log corroboration: only **one** "Loading weights 100%" appeared — the unconstrained
83+
Layer-A parent. The capped Layer-B child failed *before* that, during library import (it
84+
could not even mmap a C extension), well before model loading.
85+
86+
**What is now instrumented (so the re-run proves it, not asserts it).** `linux_runner`
87+
logs `getrlimit(RLIMIT_AS)` *after* `setrlimit` (kernel-confirmed soft/hard), the capped
88+
child's PID, and always the failing exception's repr. The script also runs a
89+
**dependency-free control**`bytearray(400 MB)` under the same 256 MB cap — to isolate
90+
RLIMIT_AS itself: if the control is not enforced, the bug is in `linux_runner`; if it is,
91+
RLIMIT_AS works and the model case's `error` is a cap-induced library failure. Agreement is
92+
now decided on *whether the workload completed under the cap*, not only on `MemoryError`.
93+
94+
**Confirmed by the instrumented CI run (run 27951551696).**
95+
96+
- **Control** (`bytearray(400 MB)`, no deps) → `outcome=memory_error`, kernel `RLIMIT_AS`
97+
soft=**256 MB**, child PID 2314, clean `MemoryError`. So **RLIMIT_AS itself enforces**
98+
the runner is not broken.
99+
- **Layer B** (model workload) → `outcome=error`, kernel `RLIMIT_AS` soft=**256 MB**, child
100+
PID 2315. The surfaced exception chain shows the real root cause:
101+
`ImportError: …/regex/_regex.cpython-312-x86_64-linux-gnu.so: failed to map segment from
102+
shared object` — i.e. the dynamic loader could not `mmap` a C-extension shared library
103+
while *importing* `transformers``regex`, under the 256 MB address-space cap. That
104+
ImportError is then re-wrapped (via `from exc`) as the friendly "needs the optional ML
105+
stack" message; the chain preserves both.
106+
107+
**Verdict: Layer A and Layer B AGREE — both reject the 256 MB budget.** Layer A because peak
108+
RSS (~1082 MB) ≫ 256; Layer B because the process cannot even map its shared libraries in
109+
256 MB of address space. The failure is genuinely **cap-induced**, just *earlier* in the
110+
pipeline than expected — during library import, not model loading (RSS never approaches
111+
1082 MB because the child dies during import). The earlier "DISAGREE" was purely a reporting
112+
artifact (a non-`MemoryError` failure misclassified and its chain hidden), now fixed.

pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@ convention = "google"
6363

6464
[tool.ruff.lint.per-file-ignores]
6565
"tests/**" = ["D"] # tests don't require docstrings
66+
"scripts/**" = ["T20"] # CLI/validation scripts print to the console
6667

6768
[tool.mypy]
6869
python_version = "3.12"

scripts/edge_validation.py

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
"""Layer A vs Layer B memory-enforcement validation on the fixture repo (PRD section 4.4).
2+
3+
Runs three things under a 256 MB budget and prints each outcome in full:
4+
Control -- a dependency-free ``bytearray(400 MB)`` under a real RLIMIT_AS cap. This
5+
isolates RLIMIT_AS itself: if the control is not enforced, the bug is in
6+
linux_runner, independent of any model/library.
7+
Layer B -- the same fixture benchmark (bge-small + stub LLM) under the real RLIMIT_AS cap.
8+
Layer A -- the same benchmark under the portable app-level MemoryGovernor (peak-RSS cap).
9+
10+
For every RLIMIT_AS run it prints the kernel-confirmed limit (getrlimit *after* setrlimit),
11+
the capped child's PID, the exit code, and the failing exception's repr -- so a cap-induced
12+
failure is visible, not hidden behind a coarse "enforced" flag. No Ollama and no ianvs
13+
re-ingest. On macOS the RLIMIT_AS runs are reported as skipped (it cannot enforce them).
14+
"""
15+
16+
from __future__ import annotations
17+
18+
import os
19+
import tempfile
20+
from collections.abc import Callable
21+
from pathlib import Path
22+
23+
from codecompass.benchmark.dataset import load_gold_questions
24+
from codecompass.benchmark.runner import BenchmarkRunner
25+
from codecompass.config import ChunkingConfig, EmbeddingConfig, RepoConfig
26+
from codecompass.constraints.linux_runner import (
27+
EnforcementResult,
28+
UnsupportedPlatformError,
29+
run_under_memory_limit,
30+
)
31+
from codecompass.constraints.memory import peak_rss_bytes
32+
from codecompass.constraints.profiles import ConstraintProfile
33+
from codecompass.embedding.registry import build_embedder
34+
from codecompass.ingestion.github_source import build_file_documents
35+
from codecompass.pipeline import QueryPipeline, index_documents
36+
from codecompass.retrieval.retriever import Retriever
37+
from codecompass.store.milvus_lite import MilvusLiteStore
38+
39+
_BUDGET_MB = int(os.environ.get("CC_EDGE_BUDGET_MB", "256"))
40+
_FIXTURE = Path("tests/fixtures/sample_repo")
41+
_GOLD_SET = Path("data/benchmarks/fixture_qa.yaml")
42+
_EMBEDDING = EmbeddingConfig(model_name="BAAI/bge-small-en-v1.5", dimension=384)
43+
44+
45+
class _StubProvider:
46+
"""Offline LLM provider so the validation needs no Ollama."""
47+
48+
def generate(self, prompt: str) -> str:
49+
"""Return a fixed answer."""
50+
return "Validation stub answer."
51+
52+
53+
def _allocate_400mb() -> None:
54+
"""Dependency-free 400 MB allocation -- isolates RLIMIT_AS from any model/library."""
55+
block = bytearray(400 * 1024 * 1024)
56+
block[-1] = 1
57+
58+
59+
def _build_runner(db_dir: Path) -> tuple[BenchmarkRunner, MilvusLiteStore]:
60+
"""Ingest the fixture with bge-small and wire a stub-provider benchmark runner."""
61+
embedder = build_embedder(_EMBEDDING)
62+
store = MilvusLiteStore(db_dir / "idx.db", dimension=embedder.dimension)
63+
documents = build_file_documents(_FIXTURE, RepoConfig())
64+
index_documents(documents, embedder=embedder, store=store, chunking_config=ChunkingConfig())
65+
pipeline = QueryPipeline(Retriever(embedder, store, top_k=5), _StubProvider())
66+
return BenchmarkRunner(pipeline, top_k=5), store
67+
68+
69+
def _benchmark_workload() -> None:
70+
"""Run the fixture benchmark once (bge load + index + score) -- the memory-heavy work."""
71+
with tempfile.TemporaryDirectory() as tmp:
72+
runner, store = _build_runner(Path(tmp))
73+
try:
74+
runner.score_once(load_gold_questions(_GOLD_SET))
75+
finally:
76+
store.close()
77+
78+
79+
def _enforce(func: Callable[[], None]) -> EnforcementResult | None:
80+
"""Run ``func`` under the real RLIMIT_AS cap; None when not on Linux."""
81+
try:
82+
return run_under_memory_limit(func, memory_mb=_BUDGET_MB)
83+
except UnsupportedPlatformError:
84+
return None
85+
86+
87+
def _run_layer_a() -> tuple[int, int, int]:
88+
"""Layer A: run the workload under the MemoryGovernor mem-256 cap.
89+
90+
Returns ``(failure_count, repeats, observed_peak_rss_mb)``.
91+
"""
92+
profile = ConstraintProfile(name=f"mem-{_BUDGET_MB}", memory_mb=_BUDGET_MB)
93+
with tempfile.TemporaryDirectory() as tmp:
94+
runner, store = _build_runner(Path(tmp))
95+
try:
96+
result = runner.run(load_gold_questions(_GOLD_SET), profile, repeats=1)
97+
finally:
98+
store.close()
99+
return result.failure_count, result.repeats, peak_rss_bytes() // (1024 * 1024)
100+
101+
102+
def _print_enforcement(label: str, result: EnforcementResult | None) -> None:
103+
"""Print a full RLIMIT_AS result: outcome, kernel-confirmed cap, PID, exit, detail."""
104+
if result is None:
105+
print(f"{label} @ {_BUDGET_MB} MB: SKIPPED (not Linux; RLIMIT_AS unenforceable here).")
106+
return
107+
soft = result.applied_soft_bytes
108+
applied_mb = soft // (1024 * 1024) if soft is not None else None
109+
print(
110+
f"{label} @ {_BUDGET_MB} MB: outcome={result.outcome} "
111+
f"(completed={result.completed}, enforced={result.enforced}); "
112+
f"kernel RLIMIT_AS soft={applied_mb} MB; child_pid={result.child_pid}; "
113+
f"exit={result.exit_code}; detail={result.detail}"
114+
)
115+
if result.traceback_text:
116+
print(f" ----- {label}: full exception chain (child PID {result.child_pid}) -----")
117+
for line in result.traceback_text.rstrip().splitlines():
118+
print(f" {line}")
119+
print(" ----- end exception chain -----")
120+
121+
122+
def main() -> None:
123+
"""Run Control + Layer B first (parent stays torch-free at fork), then Layer A; report."""
124+
print(f"== Layer A vs Layer B memory enforcement @ {_BUDGET_MB} MB (fixture repo, stub LLM) ==")
125+
control = _enforce(_allocate_400mb)
126+
layer_b = _enforce(_benchmark_workload)
127+
failures, repeats, peak_mb = _run_layer_a()
128+
layer_a_completed = failures == 0
129+
130+
print("")
131+
_print_enforcement("Control (bytearray 400 MB, no deps)", control)
132+
_print_enforcement("Layer B (model workload)", layer_b)
133+
print(
134+
f"Layer A (MemoryGovernor peak-RSS cap): failures={failures}/{repeats}, "
135+
f"observed peak RSS={peak_mb} MB vs budget {_BUDGET_MB} MB -> "
136+
f"{'OK (ran)' if layer_a_completed else 'FAIL (rejected)'}"
137+
)
138+
139+
if control is not None and not control.completed and not control.enforced:
140+
# Only a control that neither completed (fit the budget) nor was enforced (a clean
141+
# MemoryError) is suspicious -- that means the pure-allocation control itself errored.
142+
print(
143+
f"WARNING: the dependency-free control errored (outcome={control.outcome}) -- "
144+
"RLIMIT_AS may be misbehaving in linux_runner; investigate the Layer-B result."
145+
)
146+
if layer_b is None:
147+
print("Agreement: PENDING Layer B (not on Linux) -- run the edge-validation workflow.")
148+
return
149+
if layer_a_completed != layer_b.completed:
150+
print(
151+
f"Agreement: DISAGREE -- Layer A completed={layer_a_completed}, "
152+
f"Layer B completed={layer_b.completed}. Inspect the Layer-B detail above."
153+
)
154+
elif layer_a_completed:
155+
print(f"Agreement: AGREE -- both layers ran the workload under {_BUDGET_MB} MB.")
156+
elif layer_b.enforced:
157+
print(
158+
f"Agreement: AGREE -- both rejected the workload under {_BUDGET_MB} MB "
159+
"(Layer B via a clean MemoryError/kill)."
160+
)
161+
else:
162+
print(
163+
f"Agreement: LIKELY AGREE -- both failed to run under {_BUDGET_MB} MB, but Layer B's "
164+
"outcome=error is a non-MemoryError; confirm from the detail above that it is "
165+
"cap-induced (e.g. a failed mmap/import), not an unrelated failure."
166+
)
167+
168+
169+
if __name__ == "__main__":
170+
main()

src/codecompass/constraints/__init__.py

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,17 @@
1-
"""Edge-constraint simulation (PRD section 4.4, Layer A: portable, deterministic)."""
1+
"""Edge-constraint simulation (PRD section 4.4): Layer A (portable) + Layer B (OS-enforced)."""
22

33
from codecompass.constraints.cpu import apply_cpu_limit
4+
from codecompass.constraints.docker_runner import (
5+
DockerResult,
6+
DockerUnavailableError,
7+
docker_available,
8+
run_under_docker_memory_limit,
9+
)
10+
from codecompass.constraints.linux_runner import (
11+
EnforcementResult,
12+
UnsupportedPlatformError,
13+
run_under_memory_limit,
14+
)
415
from codecompass.constraints.memory import (
516
MemoryBudgetExceeded,
617
MemoryGovernor,
@@ -18,13 +29,20 @@
1829
__all__ = [
1930
"CANONICAL_ORDER",
2031
"ConstraintProfile",
32+
"DockerResult",
33+
"DockerUnavailableError",
34+
"EnforcementResult",
2135
"MemoryBudgetExceeded",
2236
"MemoryGovernor",
2337
"NetworkShaper",
2438
"ShapedProvider",
39+
"UnsupportedPlatformError",
2540
"apply_cpu_limit",
41+
"docker_available",
2642
"load_profile",
2743
"load_profiles_dir",
2844
"normalize_ru_maxrss",
2945
"peak_rss_bytes",
46+
"run_under_docker_memory_limit",
47+
"run_under_memory_limit",
3048
]

0 commit comments

Comments
 (0)