diff --git a/README.md b/README.md index 2bd6e32..bc840ff 100644 --- a/README.md +++ b/README.md @@ -123,7 +123,7 @@ pip install 'git+https://github.com/intellerce/agentcodec.git@v0.3.0' Python ≥ 3.10. Local model experiments need [Ollama](https://ollama.com) or a vLLM/SGLang endpoint. Cloud experiments need provider API keys (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, etc.). A `.env` is loaded automatically by the benchmark runner; the library facade reads keys directly from `os.environ`. -For SemKNN routing you additionally need a SemKNN backend reachable via HTTPS — the public package ships only the client. See [§privacy & data flow](#privacy--data-flow) and [COMMERCIAL.md](./COMMERCIAL.md) for hosted / self-host options. +For SemKNN routing you additionally need a SemKNN backend reachable via HTTPS — the public package ships only the client. It defaults to the public hosted endpoint `https://agentcodec.intellerce.com`; override it with the `AGENTCODEC_SEMKNN_SERVER_URL` env var (or `router.server_url` in YAML) to point at your own backend. See [§privacy & data flow](#privacy--data-flow) and [COMMERCIAL.md](./COMMERCIAL.md) for hosted / self-host options. --- @@ -1211,15 +1211,17 @@ Every technique below is dispatched by `agentcodec.dispatch.dispatch()` and shar | Ours | HARQ (ARQ retry / combine) | `harq_cc`, `harq_ir` | 2 | | Ours | Turbo decoding | `turbo` | 1 | | Ours | Fountain (rateless) | `fountain`, `fountain_soft` | 2 | -| Ours | FEC (forward error correction) | `fec_0.75`, `fec_0.50`, `fec_0.33` | 3 | -| Ours | Diversity combining | `diversity_sc`, `diversity_mrc`, `diversity_egc`, `diversity_sc_N`, `diversity_mrc_discrete_N`, `diversity_spatial`, `diversity_frequency`, `diversity_time`, `diversity_mrc_soft`, `diversity_mrc_discrete_N_soft` | 10 | +| Ours | FEC (forward error correction) | `fec_0.75`, `fec_0.50`, `fec_0.33`, `fec_0.25` | 4 | +| Ours | Diversity combining | `diversity_sc`, `diversity_mrc`, `diversity_egc`, `diversity_sc_N`, `diversity_mrc_discrete_N`, `diversity_spatial`, `diversity_frequency`, `diversity_time`, `diversity_mrc_soft`, `diversity_mrc_discrete_N_soft` | 9¹ | | Ours | ACM routing (adaptive coding/modulation) | `acm`, `acm_soft`, `acm_learned` | 3 | | Prior | Baselines | `self_consistency`, `self_refine`, `chain_of_verification`, `best_of_n`, `weighted_bon`, `cisc`, `mixture_of_agents` | 7 | | | **Reliability techniques** | | **28** | | | **+ uncoded `baseline` reference** | | **1** | -| | **Dispatchable total** | | **29** | +| | **Dispatchable total** | | **30** | -Communication-theoretic subtotal: **21** across the six families. Prior-method baselines: **7**. That's **28 reliability techniques**; the uncoded `baseline` (single pass, no redundancy) is the reference they're all measured against, which brings the dispatch enum to **29** entries. +¹ `diversity_spatial` is behaviorally identical to `diversity_mrc` (both = MRC combining over multiple model channels), so it's dispatchable but not counted as a distinct technique. The diversity family is really two independent axes: **what varies** (spatial = different models, frequency = different prompts, time = different temperatures) × **how branches are combined** (SC = pick best, MRC = quality-weighted synthesis, EGC = equal-weight) — any source can pair with any combiner. + +Communication-theoretic subtotal: **21** across the six families. Prior-method baselines: **7**. That's **28 reliability techniques**; the uncoded `baseline` (single pass, no redundancy) is the reference they're all measured against, which — plus the `diversity_spatial` alias — brings the dispatch enum to **30** entries. > **Routers live at the strategy layer, not in this list — but they're still techniques.** In communication theory, adaptive coding-modulation (ACM) *is* a technique; it's one of our six families. The library simply realizes ACM at **two layers**: the self-contained `acm` / `acm_soft` / `acm_learned` entries are dispatchable *leaf* techniques (probe difficulty → dispatch in-band) and so live in `KNOWN_TECHNIQUES`, while the `semknn` / `acm_table` / `acm_linear` *routers* — SemKNN being the flagship cost-aware one that delivers the ~56% number — **orchestrate the other techniques** and so are configured via `strategy.type` (see [§Strategies: fixed vs routed](#strategies-fixed-vs-routed) and the [router table](#routers-strategy-layer) below) rather than dispatched by name. They're techniques in the comm-theory sense; they just aren't *leaf* entries in the dispatch catalog, which is why the count of **29 dispatchable entries** doesn't include them. @@ -1243,7 +1245,7 @@ SemKNN is the recommended router — see [§SemKNN](#semknn--the-recommended-rou | `harq_ir` | Hybrid ARQ Incremental Redundancy — retry with critic feedback. | `max_rounds` (5) | | `turbo` | Iterative SISO decoding — generator + critic exchange extrinsic info. | `max_iterations` (5) | | `fountain` | Rateless: keep generating samples until the judge is satisfied. | `max_samples` (8) | -| `fec_0.75`, `fec_0.50`, `fec_0.33` | Forward error correction at three code rates. | `code_rate` (from name) | +| `fec_0.75`, `fec_0.50`, `fec_0.33`, `fec_0.25` | Forward error correction at four code rates (1/2/3/4 parity calls). | `code_rate` (from name) | | `diversity_sc` | Selection Combining — pick the highest-scored branch. | (one branch per channel) | | `diversity_mrc` | Maximal Ratio Combining — quality-weighted synthesis. | (one branch per channel) | | `diversity_egc` | Equal Gain Combining — equal-weight consensus. | — | diff --git a/agentcodec/dispatch.py b/agentcodec/dispatch.py index 974dcd0..ff3a25f 100644 --- a/agentcodec/dispatch.py +++ b/agentcodec/dispatch.py @@ -347,12 +347,15 @@ def _baseline(task: TaskItem, channel: AgentChannel, scorer: QualityScorer) -> R "best_of_n", "weighted_bon", "cisc", "mixture_of_agents", "diversity_sc", "diversity_mrc", "diversity_egc", "diversity_sc_N", "diversity_mrc_discrete_N", - "diversity_spatial", "diversity_frequency", "diversity_time", + # NOTE: "diversity_spatial" is intentionally omitted — it's an alias for + # diversity_mrc (same MRC combining; see dispatch branch above) and is not + # counted as a distinct public technique. It still dispatches correctly. + "diversity_frequency", "diversity_time", "diversity_mrc_soft", "diversity_mrc_discrete_N_soft", "harq_cc", "harq_ir", "turbo", "fountain", "fountain_soft", - "fec_0.75", "fec_0.50", "fec_0.33", + "fec_0.75", "fec_0.50", "fec_0.33", "fec_0.25", "acm", "acm_soft", "acm_learned", ) @@ -417,6 +420,7 @@ def _baseline(task: TaskItem, channel: AgentChannel, scorer: QualityScorer) -> R "fec_0.75", "fec_0.50", "fec_0.33", + "fec_0.25", "diversity_mrc_soft", "diversity_mrc_discrete_N_soft", "acm", diff --git a/agentcodec/techniques/acm_learned.py b/agentcodec/techniques/acm_learned.py index 7672e0c..c3c10ed 100644 --- a/agentcodec/techniques/acm_learned.py +++ b/agentcodec/techniques/acm_learned.py @@ -73,6 +73,7 @@ "fec_0.75": {"code_rate": 0.75}, "fec_0.50": {"code_rate": 0.50}, "fec_0.33": {"code_rate": 0.33}, + "fec_0.25": {"code_rate": 0.25}, "diversity_sc_N": {"num_samples": 5}, "diversity_mrc_discrete_N": {"num_samples": 5}, "self_consistency": {"num_samples": 5}, diff --git a/examples/00_drop_in_openai.py b/examples/00_drop_in_openai.py index e4678cb..0a18e28 100644 --- a/examples/00_drop_in_openai.py +++ b/examples/00_drop_in_openai.py @@ -146,6 +146,12 @@ def layer_3_full_module(): client = OpenAI( api_key=API_KEY, base_url=BASE_URL, reliability=mod, ) + # `model=` is required by the OpenAI SDK signature (this is a drop-in + # wrapper), but with a full ReliabilityModule injected it does NOT pick + # the channel models — the module's own `models=[MODEL_A, MODEL_B]` above + # drives the calls. Here `model` is inert: it's only stamped onto + # `resp.model` as a label. (It *does* matter in passthrough / preset-string + # layers, which is why every layer keeps the same call shape.) resp = client.chat.completions.create( model=MODEL_A, messages=[ diff --git a/setup.sh b/setup.sh index e37f247..6206d88 100644 --- a/setup.sh +++ b/setup.sh @@ -1,8 +1,14 @@ #!/usr/bin/env bash # Set up a local development environment for the public AgentCodec package. # -# ./setup.sh # create .venv, install package + dev deps -# ./setup.sh --remote # also install the remote-semknn extra (BGE encoder) +# ./setup.sh # create .venv, install package + dev + provider SDKs +# ./setup.sh --heavy # also install benchmark + remote-semknn (torch, etc.) +# +# The default install includes the openai/anthropic/ollama provider SDKs and +# the eval extra so the full test suite and examples run without "module not +# found" surprises. The heavy extras (torch, transformers, matplotlib, pandas, +# scikit-learn, datasets) are opt-in via --heavy since most contributors don't +# need them and they make a clean install much larger. # # Re-runnable; uses uv (https://docs.astral.sh/uv/) for fast installs. @@ -15,10 +21,21 @@ if ! command -v uv >/dev/null 2>&1; then exit 1 fi -EXTRAS="dev" -if [[ "${1:-}" == "--remote" ]]; then - EXTRAS="dev,remote-semknn" -fi +# Default: dev tooling + every provider SDK + eval stats. Lets all tests run +# without pulling in torch/matplotlib/datasets. +EXTRAS="dev,openai,anthropic,ollama,eval" +case "${1:-}" in + --heavy) + # `all` bundles the provider SDKs plus remote-semknn/benchmark/eval. + EXTRAS="dev,all" + ;; + "") + ;; + *) + echo "Unknown option: $1 (expected --heavy)" >&2 + exit 1 + ;; +esac # Create the venv (Python >= 3.10 per pyproject.toml). if [[ ! -d .venv ]]; then @@ -29,5 +46,6 @@ fi uv pip install --python .venv/bin/python -e ".[${EXTRAS}]" echo +echo "Installed extras: ${EXTRAS}" echo "Done. Activate with: source .venv/bin/activate" echo "Run tests with: ./.venv/bin/pytest tests/"