feat(s101): native S-101 (S-100 Part 10a) chart loading

docs/docs/architecture.md

@@ -6,21 +6,26 @@ sidebar_position: 7
 
 # Architecture
 
-This page explains how tile57 turns an S-57 chart — an ENC *cell*, in the
-spec's vocabulary — into vector tiles, how the codebase is layered, and the
-memory design that keeps it small.
+This page explains how tile57 turns an ENC chart — a *cell*, in the spec's
+vocabulary — into vector tiles, how the codebase is layered, and the memory
+design that keeps it small.
 
 ## The pipeline
 
-A chart flows through these stages, all inside the engine:
+A chart flows through these stages, all inside the engine. Both source formats
+converge on the same S-101 feature + attribute records — a native S-101 dataset
+produces them directly; an S-57 chart is adapted into them:
 
 ```
-S-57 ENC cell (.000)
-   │  decode the binary container     src/iso8211/   (module: iso8211)
-   ▼
-S-57 feature + geometry model         src/s57/       (module: s57)
-   │  adapt S-57 → S-101 features     src/s101/ adapter.zig
-   ▼
+S-101 ENC (.000)          S-57 ENC cell (.000)
+   │ decode 8211             │ decode 8211            src/iso8211/  (iso8211)
+   ▼                         ▼
+S-100 spatial + feature   S-57 feature + geometry     src/s57/      (s57)
+   records                   model
+   │ assemble to the         │ adapt S-57 → S-101      src/s101/ native.zig
+   │ S-101 model             │ features                          / adapter.zig
+   └───────────┬─────────────┘
+               ▼
 S-101 feature + attribute records
    │  apply S-101 portrayal           src/portray/ + embedded Lua 5.4
    ▼                            (vendor/S-101_Portrayal-Catalogue)
@@ -35,11 +40,16 @@ render Surface                        src/render/surface.zig
    └─► pixel surfaces: PNG raster · vector PDF · terminal text (src/render/)
 ```
 
-1. **Decode (ISO 8211).** S-57 charts use the ISO 8211 binary container format;
-   the decoder reads its raw records and fields.
-2. **Build the S-57 model.** Features (depth areas, buoys, coastlines, …), their
-   attributes, and their geometry (assembled from the vector topology) become a
-   queryable in-memory model.
+1. **Decode (ISO 8211).** Both S-101 and S-57 charts use the ISO 8211 binary
+   container format; the decoder reads its raw records and fields. The chart
+   format is detected from the file's own record schema.
+2. **Build the S-101 model.** A native S-101 dataset (S-100 Part 10a) is read
+   straight into S-101 feature + attribute records — its own in-band code tables
+   already carry the S-101 class and attribute names, and its complex attributes
+   are explicit, so no conversion is needed. An S-57 chart instead builds the
+   S-57 feature + geometry model (depth areas, buoys, coastlines, …) and adapts
+   it into the same S-101 records. Geometry (assembled from the vector topology)
+   and attributes become a queryable in-memory model either way.
 3. **Apply S-101 portrayal.** The official IHO S-101 Portrayal Catalogue — the
    real Lua rule files — runs in embedded Lua 5.4 and decides how to draw each
    feature: symbol, colour, line style, conditional symbology. Zig implements the
@@ -102,7 +112,7 @@ libc/Lua) and target-agnostic:
 |--------|------|
 | `iso8211` | the ISO/IEC 8211 container reader (the bottom layer; std-only) |
 | `s57` | the S-57 chart parser + geometry model (reads 8211 records through `iso8211`) |
-| `s101` | the S-101 catalogue, the S-57 → S-101 adapter, and the portrayal instruction stream |
+| `s101` | the S-101 catalogue, the native S-101 dataset reader, the S-57 → S-101 adapter, and the portrayal instruction stream |
 | `portray` | the embedded-Lua S-101 runner (links libc) |
 | `tiles` | MVT + MLT encoders, gzip, the PMTiles container, web-mercator tile math |
 | `render` | the Surface contract, the resolver (colours, display gates), and the pixel machinery (Canvas, PNG, PDF, ASCII) |

docs/docs/cli.md

@@ -147,6 +147,20 @@ edition, update, issue date, agency, bbox. `cell` summarises one chart.
 GeoJSON FeatureCollection. `catalog` decodes an exchange-set catalogue into
 its entries (file, title, bbox).
 
+### `s101`
+
+```
+tile57 s101 <file.000> [--features N]
+```
+
+Inspects a native S-101 (S-100 Part 10a) chart: confirms detection, prints the
+coordinate factors and record counts, the in-band code-table sizes, a
+feature-class histogram, and the assembled geometry summary. It then runs the
+portrayal rules and reports how many features drew, were empty, or errored. Use
+`--features N` to also dump the first N features with their attributes. Charts
+are auto-detected everywhere — `png`, `bake`, and the C API read a native S-101
+`.000` transparently — so this command is for inspection, not a separate load path.
+
 ### `inspect` / `tiledump`
 
 ```

docs/docs/intro.md

@@ -16,15 +16,16 @@ real-world navigation.** See [Known limitations](./limitations.md).
 
 :::
 
-**tile57** is a nautical chart engine. It reads IHO **S-57** ENC cells — each
-cell is one electronic navigational chart, compiled for one scale; NOAA and
-other hydrographic offices publish thousands — converts each chart's features
-to the newer **S-101** data model, and runs the official IHO **S-101
-Portrayal Catalogue** in embedded Lua to decide how each feature is drawn,
-producing **S-101 symbology** (the successor to S-52). The conversion is an
-interim step, and a best-effort one — S-57 has no perfect S-101 translation
-(see [Known limitations](./limitations.md)); the goal is S-101 throughout,
-reading native S-101 charts directly as hydrographic offices publish them.
+**tile57** is a nautical chart engine. It reads both native IHO **S-101** ENC
+charts — the S-100-based successor format hydrographic offices are beginning to
+publish — and legacy IHO **S-57** ENC cells (each cell is one electronic
+navigational chart, compiled for one scale; NOAA and others publish thousands).
+The format is detected from the chart file: a native S-101 dataset feeds the
+portrayal model directly, while an S-57 chart is first converted to the S-101
+data model — a best-effort step, since S-57 has no perfect S-101 translation
+(see [Known limitations](./limitations.md)). Either way, tile57 runs the
+official IHO **S-101 Portrayal Catalogue** in embedded Lua to decide how each
+feature is drawn, producing **S-101 symbology** (the successor to S-52).
 
 The primary output is **vector tiles**, fetched by `(z, x, y)`:
 [MapLibre Tiles](https://github.com/maplibre/maplibre-tile-spec) (MLT, the

docs/docs/limitations.md

@@ -22,14 +22,31 @@ See the warning on the [introduction](./intro.md). NOAA ENC charts are U.S. publ
 domain and not for navigation; this renderer adds its own gaps on top.
 :::
 
+## Native S-101 charts
+
+A native S-101 (S-100 Part 10a) chart is read directly into the S-101 portrayal
+model with no conversion — the format is auto-detected from the file, so `png`,
+`bake`, and the C API accept a native S-101 `.000` transparently, and its
+sequential `.001…` update files are applied (record-level insert / delete /
+modify by identity). On the SHOM France test datasets and the S-164 update
+conformance set every feature portrays (no errors), including complex attributes
+such as light sectors and buoy topmarks. The cursor-pick report serves each
+feature's S-101 class and full attribute tree (nested complex attributes
+included), with text preserved as UTF-8. One area is not yet wired for native
+S-101:
+
+- **Feature-to-feature associations.** `FASC`/`INAS` records are parsed but the
+  StructureEquipment relationship (e.g. a light *supportedBy* its beacon) is not
+  yet resolved for the query report's related-feature grouping. It carries no
+  portrayal weight for the test charts, so rendering is unaffected.
+
 ## S-57 → S-101 conversion
 
-tile57's input is S-57 but its portrayal rules are S-101, so every chart passes
-through an S-57 → S-101 adapter (`src/s101/adapter.zig`) before the rules run.
-The adapter is an interim solution — the goal is S-101 throughout, reading
-native S-101 charts directly as hydrographic offices publish them. S-57 has no
-perfect S-101 translation; the adapter follows the IHO S-65 conversion
-guidance, and the result is **best effort**:
+An S-57 chart's portrayal rules are S-101, so an S-57 chart passes through an
+S-57 → S-101 adapter (`src/s101/adapter.zig`) before the rules run (a native
+S-101 chart skips this — it is already in the S-101 model). S-57 has no perfect
+S-101 translation; the adapter follows the IHO S-65 conversion guidance, and the
+result is **best effort**:
 
 - **Missing S-101 content stays missing.** S-101 attributes and features with
   no S-57 source are never invented; rules that test them take their fallback

src/chart.zig

@@ -27,6 +27,7 @@ const scene = @import("scene");
 const portray = @import("portray");
 const bake_enc = @import("scene").bake_enc;
 const catalogue = @import("s101").catalogue;
+const s101 = @import("s101");
 const tile = @import("tiles").tile;
 const render = @import("render");
 const sprite = @import("sprite");
@@ -326,6 +327,29 @@ fn streamRead(ls: *LazySource, lc: *LazyCell) bool {
     return true;
 }
 
+/// A parsed .000 chart: the geometry cell plus, for a NATIVE S-101 dataset, its
+/// pre-built portrayal records (so portray bypasses the S-57 -> S-101 adapter).
+const CellLoad = struct { cell: s57.Cell, adapted: ?[]const s101.adapter.Adapted = null };
+
+/// Parse a .000 chart, auto-detecting S-101 vs S-57 from the file itself, and apply
+/// its sequential `.001…` update chain. A native S-101 dataset (S-100 Part 10a)
+/// assembles via s101.native; an S-57 cell parses via s57. Returns null on failure.
+fn parseAnyCell(base: []const u8, updates: []const []const u8) ?CellLoad {
+    if (s101.dataset.detect(base)) {
+        const l = s101.native.parseDataset(gpa, base, updates) catch return null;
+        return .{ .cell = l.cell, .adapted = l.adapted };
+    }
+    const cell = s57.parseCellWithUpdates(gpa, base, updates) catch return null;
+    return .{ .cell = cell };
+}
+
+/// Portray a cell three ways, using the native adapted set (S-101) when present,
+/// else the S-57 adapter.
+fn portrayVariantsAny(arena: std.mem.Allocator, cell: *const s57.Cell, adapted: ?[]const s101.adapter.Adapted, dir: []const u8) !portray.CellPortrayal {
+    if (adapted) |ad| return portray.portrayCellVariantsAdapted(arena, cell, ad, dir);
+    return portray.portrayCellVariants(arena, cell, dir);
+}
+
 // Parse + portray a lazy cell if not already loaded, and stamp its LRU tick.
 fn lazyEnsureLoaded(ls: *LazySource, lc: *LazyCell) void {
     ls.tick += 1;
@@ -334,11 +358,12 @@ fn lazyEnsureLoaded(ls: *LazySource, lc: *LazyCell) void {
     if (lc.streaming and lc.base.len == 0) {
         if (!streamRead(ls, lc)) return;
     }
-    var cell = s57.parseCellWithUpdates(gpa, lc.base, lc.updates) catch return;
+    const loaded = parseAnyCell(lc.base, lc.updates) orelse return;
+    var cell = loaded.cell;
     cell.name = lc.name; // pick-report source-cell badge (gpa-owned, lives with the source)
     if (gpa.create(std.heap.ArenaAllocator)) |p| {
         p.* = std.heap.ArenaAllocator.init(gpa);
-        if (portray.portrayCellVariants(p.allocator(), &cell, ls.rules_dir)) |cp| {
+        if (portrayVariantsAny(p.allocator(), &cell, loaded.adapted, ls.rules_dir)) |cp| {
             lc.portrayal = cp.base;
             lc.portrayal_plain = cp.plain;
             lc.portrayal_simplified = cp.simplified;
@@ -492,14 +517,17 @@ pub fn openPmtilesPath(io: std.Io, path: []const u8) !*Chart {
 // Parse (+ apply updates) + portray one cell into a CellBackend. Reads the bytes
 // but does not take ownership. Portrayal failure is non-fatal (classify() fallback).
 fn buildCellBackend(base: []const u8, updates: []const []const u8, dir: []const u8) ?CellBackend {
-    const cell = s57.parseCellWithUpdates(gpa, base, updates) catch return null;
-    var cb = CellBackend{ .cell = cell, .cscl = s57.peekScale(gpa, base) orelse 0 };
+    const loaded = parseAnyCell(base, updates) orelse return null;
+    // Use the parsed cell's compilation scale (S-57 DSPM CSCL, or a native S-101
+    // chart's DataCoverage display scale) — `peekScale` reads only the S-57 DSPM and
+    // returns 0 for native, which would mis-band a native chart in the live compositor.
+    var cb = CellBackend{ .cell = loaded.cell, .cscl = loaded.cell.params.cscl };
     const pa = gpa.create(std.heap.ArenaAllocator) catch return cb;
     pa.* = std.heap.ArenaAllocator.init(gpa);
     cb.portray_arena = pa;
     // Real M_COVR data-coverage polygons for the host to report as chart coverage.
     cb.coverage = cb.cell.mcovrCoverage(pa.allocator());
-    if (portray.portrayCellVariants(pa.allocator(), &cb.cell, dir)) |cp| {
+    if (portrayVariantsAny(pa.allocator(), &cb.cell, loaded.adapted, dir)) |cp| {
         cb.portrayal = cp.base;
         cb.portrayal_plain = cp.plain;
         cb.portrayal_simplified = cp.simplified;
@@ -604,16 +632,19 @@ pub fn bakeChartBytes(cell_path: []const u8, rules_dir: ?[]const u8) !?[]u8 {
     var cov_arena = std.heap.ArenaAllocator.init(gpa);
     defer cov_arena.deinit();
     var coverage_json: ?[]const u8 = null;
+    // Parse native-aware (S-101 or S-57): the band scale + the archive's coverage
+    // sidecar both come from the real cell, so a native S-101 chart bands by its
+    // DataCoverage display scale (not the S-57-misparsed cscl=0 approach default).
     var cscl: i32 = s57.peekScale(gpa, cf.base) orelse 0;
-    if (s57.parseCellWithUpdates(gpa, cf.base, cf.updates)) |parsed| {
-        var cell = parsed;
+    if (parseAnyCell(cf.base, cf.updates)) |loaded| {
+        var cell = loaded.cell;
         defer cell.deinit();
         cscl = cell.params.cscl;
         const stem = std.fs.path.stem(std.fs.path.basename(cell_path));
         const band: u8 = @intFromEnum(bake_enc.bandOf(cscl));
         const cc = scene.coverage.fromCell(cov_arena.allocator(), &cell, stem, band);
         coverage_json = scene.coverage.encodeJson(cov_arena.allocator(), cc) catch null;
-    } else |_| {}
+    }
 
     // The cell's band window, plus the extend_min fill DOWN to z0: sub-band tiles
     // (scamin-thinned by the scene cull) let the compositor pull this cell up into
@@ -2348,7 +2379,8 @@ const BakeWork = struct {
         _ = scratch; // owns persistent backends via its own arenas / `gpa`
         const c: *BakeWork = @ptrCast(@alignCast(uptr));
         const src = c.sources[i];
-        var cell = s57.parseCellWithUpdates(gpa, src.base, src.updates) catch return;
+        const loaded = parseAnyCell(src.base, src.updates) orelse return;
+        var cell = loaded.cell;
         cell.name = src.name; // pick-report badge (borrowed for the bake call)
         const b = cell.bounds() orelse {
             cell.deinit();
@@ -2363,7 +2395,7 @@ const BakeWork = struct {
         const pa: ?*std.heap.ArenaAllocator = gpa.create(std.heap.ArenaAllocator) catch null;
         if (pa) |p| {
             p.* = std.heap.ArenaAllocator.init(gpa);
-            if (portray.portrayCellVariants(p.allocator(), &cell, c.rules_dir)) |cp| {
+            if (portrayVariantsAny(p.allocator(), &cell, loaded.adapted, c.rules_dir)) |cp| {
                 portrayal = cp.base;
                 portrayal_plain = cp.plain;
                 portrayal_simplified = cp.simplified;

src/iso8211/iso8211.zig

@@ -230,37 +230,65 @@ fn parseFieldControls(a: Allocator, ddr: Record, field_control_length: usize) ![
     return list.items;
 }
 
-/// Parse format controls like "(A,I(4),B(40))" into subfield defs.
+/// Parse format controls like "(A,I(4),B(40))" into subfield defs. ISO 8211 allows
+/// nested parenthesized groups with an optional leading repeat count — e.g. UKHO's
+/// "(b11,b14,7A,A(8),3A,(b11))" and "(b11,(3b24))" — where the group's formats repeat
+/// `repeat` times. (These defs are informational: s57/s101 decode records by their
+/// fixed schema, so the value here is that a legal DDR parses; a group must NOT be
+/// mistaken for a `(width)` and fed to the integer parser — SHOM cells lack nesting,
+/// UKHO cells have it.)
 pub fn parseSubfields(a: Allocator, fmt_in: []const u8) ![]SubfieldDef {
     var list = std.ArrayList(SubfieldDef).empty;
     var fmt = std.mem.trim(u8, fmt_in, " ");
     if (fmt.len == 0) return list.items;
     if (fmt[0] == '(') fmt = fmt[1..];
     if (fmt.len > 0 and fmt[fmt.len - 1] == ')') fmt = fmt[0 .. fmt.len - 1];
+    try parseFmtList(a, &list, fmt);
+    return list.items;
+}
 
+/// Parse one comma-separated format list into `list`, recursing into parenthesized
+/// groups. Each recursion consumes a shorter (parens-stripped) slice, so it terminates.
+fn parseFmtList(a: Allocator, list: *std.ArrayList(SubfieldDef), fmt: []const u8) !void {
     var i: usize = 0;
     while (i < fmt.len) {
-        // optional leading repeat count, e.g. "2A(3)" or "3I"
+        // optional leading repeat count, e.g. "2A(3)", "3I", "2(A,I)"
         var repeat: usize = 1;
         const rstart = i;
         while (i < fmt.len and fmt[i] >= '0' and fmt[i] <= '9') i += 1;
-        if (i > rstart) repeat = try asciiInt(fmt[rstart..i]);
+        if (i > rstart) repeat = asciiInt(fmt[rstart..i]) catch 1;
         if (i >= fmt.len) break;
-        const ftype = fmt[i];
-        i += 1;
-        var width: usize = 0;
-        if (i < fmt.len and fmt[i] == '(') {
+
+        if (fmt[i] == '(') {
+            // A nested group: find its balanced ')' and expand its contents `repeat` times.
+            const gstart = i + 1;
+            var depth: usize = 1;
+            var j = gstart;
+            while (j < fmt.len and depth > 0) : (j += 1) {
+                if (fmt[j] == '(') depth += 1 else if (fmt[j] == ')') depth -= 1;
+            }
+            const inner = fmt[gstart .. j - @intFromBool(depth == 0)]; // drop the closing ')'
+            var k: usize = 0;
+            while (k < repeat) : (k += 1) try parseFmtList(a, list, inner);
+            i = j;
+        } else {
+            const ftype = fmt[i];
             i += 1;
-            const wstart = i;
-            while (i < fmt.len and fmt[i] != ')') i += 1;
-            width = try asciiInt(fmt[wstart..i]);
-            if (i < fmt.len) i += 1; // skip ')'
+            var width: usize = 0;
+            if (i < fmt.len and fmt[i] == '(') {
+                i += 1;
+                const wstart = i;
+                while (i < fmt.len and fmt[i] != ')') i += 1;
+                // Tolerant: a width is informational and unused; a mis-aligned scan
+                // over binary notation ("b24") must not abort the whole DDR parse.
+                width = asciiInt(fmt[wstart..i]) catch 0;
+                if (i < fmt.len) i += 1; // skip ')'
+            }
+            var k: usize = 0;
+            while (k < repeat) : (k += 1) try list.append(a, .{ .format_type = ftype, .width = width });
         }
-        var k: usize = 0;
-        while (k < repeat) : (k += 1) try list.append(a, .{ .format_type = ftype, .width = width });
         if (i < fmt.len and fmt[i] == ',') i += 1;
     }
-    return list.items;
 }
 
 /// Parse a whole ISO 8211 file from in-memory bytes (borrowed; keep alive).