Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
123 changes: 97 additions & 26 deletions architecture/supported-subset.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ warns (ignored, behavior-neutral inside a single pod) or raises
## Top-level keys

- **Supported:** `services` (required, non-empty), `version`, `name`,
`networks`, `secrets`.
`networks`, `volumes`, `secrets`, `configs`.
- **Ignored (warns):** `networks` — all services share the pod's single
network namespace, so top-level network definitions have no effect.
- **Extension fields:** any key prefixed `x-` is accepted and ignored
Expand Down Expand Up @@ -170,51 +170,122 @@ colon-less entry that is not absolute (e.g. `./cache`) is malformed and raises.
relative) or `environment:` (a host environment variable name), both as a
plain string. `external: true` gets its own rejection message (compose's
"must already exist" secrets have no analogue here); any other unrecognized
key raises generically (`_validate_secret_def`, `compose2pod/secrets.py`).
key raises generically (`_validate_def`, `compose2pod/store.py`, via the
`SECRET` `StoreKind` in `compose2pod/secrets.py`).
- **Per-service `secrets:` references:** short form (`- name`) or long form
(a mapping with `source` and optionally `target`, `uid`, `gid`, `mode`).
`source` must name a top-level secret; an unknown `source` raises at
`validate()` time (`_ref_source`/`validate_secrets`,
`compose2pod/secrets.py`).
`validate()` time (`_ref_source`/`validate_all`, `compose2pod/store.py` --
`validate_all` dispatches per store kind through the internal
`_validate_kind`).
- **Closure-scoped creation:** only secrets referenced (by `source`) from
somewhere in the target service's dependency closure are ever created, so a
top-level secret nothing in the closure references never becomes a
`podman secret create` call (`referenced_secret_names`, driven by the same
`startup_order` closure used to decide which services run at all).
`podman secret create` call (`referenced_names`, `compose2pod/store.py`,
driven by the same `startup_order` closure used to decide which services
run at all).
- **Creation:** each referenced secret becomes one pod-namespaced
`podman secret create <pod>-<name> ...` line, emitted right after
`podman pod create` and before any `podman run` (`emit_script`,
`compose2pod/emit.py`). A `file:` source resolves
`Path(project_dir, file)` through `to_shell()`, so a `${VAR}` in the path
expands live when the script runs, exactly like other interpolated fields.
An `environment:` source instead pipes `printf '%s' "${VAR-}"` into
`podman secret create ... -`, where the `${VAR-}` means an *unset* host
variable yields an empty secret rather than failing the script
(`secret_create_lines`).
`podman pod create` and before any `podman run` by `all_create_lines`
(`compose2pod/store.py`, called from `emit_script`, `compose2pod/emit.py`).
A `file:` source resolves `Path(project_dir, file)` through `to_shell()`,
so a `${VAR}` in the path expands live when the script runs, exactly like
other interpolated fields. An `environment:` source instead pipes
`printf '%s' "${VAR-}"` into `podman secret create ... -`, where the
`${VAR-}` means an *unset* host variable yields an empty secret rather than
failing the script (`create_lines`, `compose2pod/store.py`).
- **Mounting:** each service reference becomes a
`--secret source=<pod>-<name>,target=<target>` flag on that service's
`podman run`, where `target` defaults to the secret's own name when the
reference doesn't give one (short form, or long form without `target`).
`uid`/`gid`/`mode` are only added when the long form gives them explicitly;
`mode` renders as a 4-digit octal string when given as a Python int
(`0o400` becomes `"0400"`) and passes through verbatim when given as a
string (`secret_flags`). When `uid`/`gid`/`mode` are omitted, podman itself
applies its own defaults: the secret is mounted at `/run/secrets/<target>`,
owned `0:0`, mode `0444`.
`podman run`, assembled per service by `all_flags` (`compose2pod/store.py`,
called from `emit_script`, `compose2pod/emit.py`), where `target` defaults
to the secret's own name when the reference doesn't give one (short form,
or long form without `target`). `uid`/`gid`/`mode` are only added when the
long form gives them explicitly; `mode` renders as a 4-digit octal string
when given as a Python int (`0o400` becomes `"0400"`) and passes through
verbatim when given as a string (`flags`, `compose2pod/store.py`). When
`uid`/`gid`/`mode` are omitted, podman itself applies its own defaults: the
secret is mounted at `/run/secrets/<target>`, owned `0:0`, mode `0444`.
- **Teardown:** the EXIT trap that force-removes the pod also runs
`podman secret rm <pod>-<name> ...` for every referenced secret, so the
store never outlives the pod even when the script exits abnormally
(`emit_script`).
(`all_teardown_names`, `compose2pod/store.py`, called from `emit_script`,
`compose2pod/emit.py`).
- **Variable interpolation:** an `environment:` source's variable name, and
any `${VAR}` inside a `file:` path, both count toward the CLI's
informational stderr note of variables the generated script expands at run
time (`secret_referenced_variables`, folded into `referenced_variables()`
in `compose2pod/emit.py`); see Variable interpolation, below, for the note
itself.
time (`referenced_variables`, `compose2pod/store.py`, assembled across
store kinds by `all_referenced_variables` and folded into
`referenced_variables()` in `compose2pod/emit.py`); see Variable
interpolation, below, for the note itself.
- Everything else raises `UnsupportedComposeError` rather than silently doing
nothing: `external: true`, an unknown `source`, a definition with neither
or both of `file`/`environment`, and an unrecognized long-form key.

## Configs

- **Top-level `configs:` definitions:** each entry must be a mapping with
exactly one of `file:` (a host path, resolved against `--project-dir` when
relative), `environment:` (a host environment variable name), or `content:`
(inline literal text), all as a plain string. `external: true` gets its own
rejection message, mirroring secrets; any other unrecognized key raises
generically (`_validate_def`, `compose2pod/store.py`, via the `CONFIG`
`StoreKind` in `compose2pod/configs.py`).
- **Per-service `configs:` references:** short form (`- name`) or long form
(a mapping with `source` and optionally `target`, `uid`, `gid`, `mode`).
`source` must name a top-level config; an unknown `source` raises at
`validate()` time (`_ref_source`/`validate_all`, `compose2pod/store.py`).
- **Closure-scoped creation:** only configs referenced (by `source`) from
somewhere in the target service's dependency closure are ever created, so a
top-level config nothing in the closure references never becomes a
`podman secret create` call -- the same closure-scoped-creation rule
secrets follow (`referenced_names`, `compose2pod/store.py`).
- **Creation and delivery:** configs are delivered through podman's secret
store, exactly like secrets, but pod- *and kind*-namespaced with a
`config-` prefix (store name `<pod>-config-<name>`, vs. a secret's bare
`<pod>-<name>`), so a config never collides with a same-named secret. Each
referenced config becomes one `podman secret create <pod>-config-<name>
...` line, emitted right after `podman pod create` and before any
`podman run` by `all_create_lines` (`compose2pod/store.py`, called from
`emit_script`, `compose2pod/emit.py`). A `file:` source resolves
`Path(project_dir, file)` through `to_shell()`; an `environment:` source
pipes `printf '%s' "${VAR-}"` into `podman secret create ... -`; and a
`content:` source pipes the literal text through `to_shell()` into that
same `podman secret create ... -` form, so a `${VAR}` written inside
`content:` stays live and expands against the generated script's own
runtime environment when the script runs -- the same deferred-interpolation
model as `file:`/`environment:` (`create_lines`, `compose2pod/store.py`).
- **Mounting:** each service reference becomes a
`--secret source=<pod>-config-<name>,target=<target>` flag on that
service's `podman run`, assembled per service by `all_flags`
(`compose2pod/store.py`, called from `emit_script`, `compose2pod/emit.py`).
Unlike a secret, whose default `target` is its own name (mounted by podman
under `/run/secrets/<target>`), a config's default `target` is the
container-root absolute path `/<name>` (`CONFIG.default_target`,
`compose2pod/configs.py`). A long-form `target` must be an absolute path
(start with `/`); a relative target raises
(`CONFIG.require_absolute_target`, checked by `_check_target`,
`compose2pod/store.py`). `uid`/`gid`/`mode` behave exactly as for secrets:
only added when the long form gives them explicitly, `mode` renders as a
4-digit octal string when given as a Python int and passes through
verbatim when given as a string (`flags`, `compose2pod/store.py`).
- **Teardown:** the EXIT trap that force-removes the pod also runs
`podman secret rm <pod>-config-<name> ...` for every referenced config, so
the store never outlives the pod even when the script exits abnormally --
byte-for-byte the same teardown parity as secrets (`all_teardown_names`,
`compose2pod/store.py`, called from `emit_script`, `compose2pod/emit.py`).
- **Variable interpolation:** an `environment:` source's variable name, any
`${VAR}` inside a `file:` path, and any `${VAR}` inside `content:` all
count toward the CLI's informational stderr note of variables the
generated script expands at run time (`referenced_variables`,
`compose2pod/store.py`, assembled across store kinds by
`all_referenced_variables` and folded into `referenced_variables()` in
`compose2pod/emit.py`); see Variable interpolation, below, for the note
itself.
- Everything else raises `UnsupportedComposeError` rather than silently
doing nothing: `external: true`, an unknown `source`, a definition with
not exactly one of `file`/`environment`/`content`, an unrecognized
long-form key, and a relative long-form `target`.

## Variable interpolation

compose2pod does not resolve Compose Spec `${VAR}` references at generation
Expand Down
13 changes: 13 additions & 0 deletions compose2pod/configs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
"""The config StoreKind: a podman secret mounted at the container-root path /<name>."""

from compose2pod.store import StoreKind


CONFIG = StoreKind(
label="config",
top_key="configs",
prefix="config-",
sources=frozenset({"file", "environment", "content"}),
default_target=lambda name: f"/{name}",
require_absolute_target=True,
)
15 changes: 8 additions & 7 deletions compose2pod/emit.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@
from compose2pod.graph import depends_on, hostnames, startup_order
from compose2pod.healthcheck import health_cmd, interval_seconds
from compose2pod.keys import SERVICE_KEYS, Token, _Expand, _key_value_pairs
from compose2pod.secrets import referenced_secret_names, secret_create_lines, secret_flags, secret_referenced_variables
from compose2pod.shell import to_shell, variable_names
from compose2pod.store import all_create_lines, all_flags, all_referenced_variables, all_teardown_names
from compose2pod.stores import STORE_KINDS


HEALTHY_WAIT_BUDGET_SECONDS = 120
Expand Down Expand Up @@ -99,7 +100,7 @@ def run_flags(name: str, svc: dict[str, Any], pod: str, hosts: list[str], projec
for key, spec in SERVICE_KEYS.items():
if key in svc:
flags += spec.emit(svc[key])
flags += secret_flags(svc, pod)
flags += all_flags(svc, pod, STORE_KINDS)
return flags


Expand Down Expand Up @@ -186,7 +187,6 @@ def emit_script(compose: dict[str, Any], options: EmitOptions) -> str:
services = compose["services"]
hosts = hostnames(services)
order = startup_order(services, options.target)
secret_names = referenced_secret_names(services, order)
completion_gated = {
dep
for svc in services.values()
Expand All @@ -196,12 +196,13 @@ def emit_script(compose: dict[str, Any], options: EmitOptions) -> str:

lines = [_SCRIPT_HEADER]
teardown = f"podman pod rm -f {shlex.quote(options.pod)} >/dev/null 2>&1 || true"
if secret_names:
stores = " ".join(f"{options.pod}-{name}" for name in secret_names)
store_names = all_teardown_names(services, order, options.pod, STORE_KINDS)
if store_names:
stores = " ".join(store_names)
teardown += f"; podman secret rm {stores} >/dev/null 2>&1 || true"
lines.append(f"trap '{teardown}' EXIT")
lines.append(f"podman pod create --name {shlex.quote(options.pod)}")
lines.extend(secret_create_lines(compose, options.pod, options.project_dir, secret_names))
lines.extend(all_create_lines(compose, order, options.pod, options.project_dir, STORE_KINDS))
waited: set[str] = set()
for name in order:
for dep, condition in depends_on(services[name]).items():
Expand Down Expand Up @@ -235,5 +236,5 @@ def referenced_variables(compose: dict[str, Any], options: EmitOptions) -> list[
for token in _run_tokens(name, services, options, hosts):
if isinstance(token, _Expand):
names.update(variable_names(token.value))
names |= secret_referenced_variables(compose, options.project_dir, referenced_secret_names(services, order))
names |= all_referenced_variables(compose, order, options.project_dir, STORE_KINDS)
return sorted(names)
1 change: 1 addition & 0 deletions compose2pod/keys.py
Original file line number Diff line number Diff line change
Expand Up @@ -197,4 +197,5 @@ def _emit_ulimits(value: Any) -> list[Token]: # noqa: ANN401 - Compose values a
"hostname",
"container_name",
"secrets",
"configs",
}
7 changes: 4 additions & 3 deletions compose2pod/parsing.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,14 @@
from compose2pod.graph import depends_on, hostnames
from compose2pod.healthcheck import has_healthcheck, interval_seconds
from compose2pod.keys import SERVICE_KEYS, STRUCTURAL_KEYS
from compose2pod.secrets import validate_secrets
from compose2pod.store import validate_all
from compose2pod.stores import STORE_KINDS


SUPPORTED_SERVICE_KEYS = set(SERVICE_KEYS) | STRUCTURAL_KEYS
IGNORED_SERVICE_KEYS = {"ports", "restart", "stdin_open", "tty", "stop_signal", "stop_grace_period"}
SUPPORTED_HEALTHCHECK_KEYS = {"test", "interval", "timeout", "retries", "start_period"}
SUPPORTED_TOP_LEVEL_KEYS = {"services", "version", "name", "networks", "volumes", "secrets"}
SUPPORTED_TOP_LEVEL_KEYS = {"services", "version", "name", "networks", "volumes", "secrets", "configs"}
DEPENDS_ON_CONDITIONS = {"service_started", "service_healthy", "service_completed_successfully"}


Expand Down Expand Up @@ -127,5 +128,5 @@ def validate(compose: dict[str, Any]) -> list[str]:
warnings.extend(_validate_service(name, svc))
hostnames(services) # validate hostname/container_name/networks shapes at the gate
_validate_depends_on(services)
validate_secrets(compose)
validate_all(compose, STORE_KINDS)
return warnings
Loading