feat(s101): native S-101 (S-100 Part 10a) chart loading#16
Merged
Conversation
Add src/s101/dataset.zig: an ISO 8211-based reader for native S-101 .000 files. It reuses the shared iso8211 container and decodes the S-100 General Feature Model — DSSI coordinate factors + record counts, the in-band code tables (FTCS/ATCS/ITCS/ARCS mapping numeric codes to S-101 class/attribute names), and the point/multipoint/curve/composite-curve/surface spatial records plus feature records (FRID/FOID/ATTR/SPAS/FASC/MASK). ATTR decodes the complex-attribute tree via the PAIX parent-position links. detect() discriminates S-101 from S-57 on the DDR field schema (the S-101-only FTCS code-table field). Add a 'tile57 s101' subcommand to inspect a dataset. Validated on all 8 SHOM France test cells: parsed record counts match the DSSI counts exactly and every feature code resolves to a class name; a real S-57 cell is correctly not detected.
Add src/s101/native.zig. parseDataset(gpa, bytes) maps a native S-101 dataset
onto the two things the renderer consumes, bypassing the S-57->S-101 adapter:
* a geometry-carrying s57.Cell shell — S-100 points -> VC nodes, curves ->
VE edges (begin/end from PTAS + interior C2IL), multipoints -> VI sounding
vectors; features resolve their SPAS (and, for surfaces, RIAS rings and
composite-curve members) into FSPT-style edge/node refs with ORNT/USAG/MASK.
Every existing geometry, masking, and boolean accessor then works unchanged.
* []adapter.Adapted built directly — class name from the FTCS table, the CNode
attribute tree reconstructed from ATTR's PAIX parent-position links (native
S-101 already carries the complex attributes the adapter has to synthesize
from flat S-57 attributes). A small class->objl / name->attr-code surrogate
covers only the classes scene.zig special-cases (soundings, depth contours,
dangers, coastline masking, lights).
Validated on all 8 SHOM France cells: features==adapted, vectors==edges+
soundingVecs, nodes==points; area/line/point/sounding geometry all resolve
through the s57.Cell accessors with bounds on the French coast.
…peline
Wire the native S-101 path into every .000 load site so the same API renders
S-101 and S-57 charts transparently:
* portray: add portrayCellWithAdapted / portrayCellVariantsAdapted — the
native entry points that portray a pre-built adapter.Adapted set, skipping
adapter.adaptCell. portrayCellWith / portrayCellVariants now delegate to them.
* chart.zig: parseAnyCell() detects S-101 vs S-57 from the file and routes to
s101.native.parseDataset or s57.parseCellWithUpdates; the lazy loader, the
query backend, and the bake worker use it + portrayVariantsAny.
* render.zig (tile57 png/pdf): same detect-and-route for the live-context path.
* s57.Cell gains a flag; scene.zig's 'unknown feature -> QUESMRK1'
fallback is gated on !cell.native (a native feature always has a valid S-101
class, so a null portrayal stream means 'nothing to draw', not 'unknown').
* native.zig: mark the shell native; exclude SOUNDG + geometry-less features
from the adapted set (scene emits soundings directly).
A French S-101 cell now renders a full S-52 chart (depth areas, soundings,
contours, coastline, buoys/beacons, wrecks, dangers, text, routes); a known
S-57 cell renders unchanged.
Map the S-101 scaleMinimum attribute onto the S-57 SCAMIN code (133) so native features are scale-culled by the existing S-52 gate — a low-zoom view of a French cell drops from 11159 to 2093 draw ops, matching S-52 generalization (small-scale soundings/marks hidden until zoomed in). The 'tile57 s101' command now also runs the rules and reports ok/empty/ERROR portrayal-stream counts by class: all 8 France cells portray with 0 errors and 0 empty streams.
tile57 reads both native S-101 (S-100 Part 10a) and legacy S-57 charts, auto- detected from the .000 file. Update intro + architecture pipeline to show the two converging entry paths, document the 'tile57 s101' inspect command, and note in limitations what native S-101 does not yet wire (update files; feature/info association surfacing in queries).
parseWithUpdates merges a native S-101 base cell with its sequential update files at the record level, by (RCNM, RCID): RUIN 1=insert, 2=delete, 3=modify (S-100 Part 10a). A feature MODIFY applies the per-association SAUI edits (2=drop the matching spatial association, 1=add) and replaces attributes/FASC when the update carries them. Because each file (base and every update) has its OWN in-band code tables, class/attribute NAMES are resolved per file at decode time and stored on the records; all kept bytes are duped into the dataset arena, so the per-file ISO readers are transient (the Dataset no longer holds one). detect + parse are unchanged for a base-only cell. Wired through s101.native.parseDataset, chart.zig parseAnyCell, render.zig, and the 'tile57 s101' inspector (all auto-apply the sibling .001.. chain). Validated on the S-164 10100AA_X01SW conformance set (edition-1.0 base + updates 1-5): the merged result is byte-for-byte identical to an independent record-level merge simulation (all 795 features, every class), and portrays with 0 errors. Unit tests cover the rcid merge index and the SAUI modify path.
Update the limitations note: native S-101 charts apply their .001.. update chain (record-level insert/delete/modify), validated on the S-164 conformance set. Feature/info association surfacing in the query report remains the one gap.
The cursor-pick / feature-info report read S-57 f.objl + f.attrs, which for a native S-101 cell are only surrogates — so a feature's real S-101 attributes (including complex/information text like a French leading-line note) never appeared, and any that did risked mangled encoding. native.zig now builds, per feature, the S-101 class name + a JSON of the S-101 attribute tree (simple values + nested complex attributes) straight from the CNode, and stores them on the s57.Cell shell (pick_class / pick_json). The JSON encoder escapes only the JSON-mandatory characters and passes UTF-8 bytes through untouched, so accented text survives intact (e.g. "...doivent etre considerees comme approximatives et simplifiees..." renders with correct accents, no replacement characters). scene.zig's FeatureMeta serves these for native cells (via pickClass/pickJson) in place of acronymByObjl + encodeS57Attrs; S-57 cells are unchanged. Verified on the SHOM France cells: the pick report now carries the full S-101 attribute tree with correct UTF-8.
…TF-8) Update the limitations note: the cursor-pick report now surfaces each native feature's S-101 class and full attribute tree (nested complex attributes, UTF-8 text). The remaining gap is StructureEquipment feature-to-feature association resolution for related-feature grouping.
A native S-101 cell had no compilation scale, so bakeChartBytes (which read the scale by parsing the file as S-57) fell back to cscl=0 -> the approach band [z11,13] for EVERY native chart. A small-scale chart with a large coverage then baked an enormous fine-zoom tile pyramid: an Arctic overview test cell (101AA00AA1NPOL3, a 95 KB / 238-feature chart spanning 35deg x 9deg) took >200 s and 1.35 trillion instructions, ~47% of it in memset (per-tile buffers) + gzip. native.zig now reads the chart's scale from the DataCoverage feature's optimumDisplayScale (falling back to maximum/minimum) and sets it as the cell's cscl. bakeChartBytes and buildCellBackend parse native-aware (parseAnyCell) so the band + coverage come from the real cell, not an S-57 misparse. Result: NPOL3 200s+ -> 1.1s; 101US002EC03M 13.6s -> 0.4s; NPOLA/B 11s -> 0.1s; harbor charts unchanged. Bands now match the S-101 navigational purpose (US2->general, US3->coastal, US4->approach, US5->berthing), and equal the S-57 CSCL for the same chart (US2EC03M = 1:1,200,000 both ways).
UKHO S-101 datasets write nested parenthesized groups in the ISO 8211 field
format controls — e.g. (b11,b14,7A,A(8),3A,(b11)) and (b11,(3b24)) — where SHOM
cells use a flat list. parseSubfields treated a nested '(' as a subfield width
and fed the group body ('3b24') to the integer parser -> BadAsciiInt, which
aborted iso.parse for the WHOLE cell (every UKHO chart failed to load).
parseSubfields now recurses into balanced groups, and the width parse is tolerant
(a mis-aligned scan over binary notation like 'b24' yields width 0 instead of
throwing). The subfield defs are informational — s57/s101 decode records by their
fixed schema — so the only requirement is that a legal DDR parses.
Validated on 27 UKHO GB charts (all parse, 0 unresolved codes, 0 portrayal
errors) and the SHOM FR charts incl. their .001/.002 update chains; S-57 cells
and the iso8211 tests are unchanged.
S-57 ENC attribute text is ISO 8859-1 (Latin-1) at the standard lexical level — a French SHOM chart's national name 'La Crabière Est' carries a lone 0xE8 for 'è'. parseATTF stored the ATVL bytes verbatim, so the tile / pick report / rendered label carried invalid UTF-8 and a client showed the replacement char. Transcode in parseATTF (covering ATTF and, via mergeNatf, the national NATF names where accents live): a value already valid UTF-8 (ASCII or a UTF-8 producer) is kept as-is; otherwise each byte is taken as a Latin-1 codepoint and UTF-8-encoded. Verified on FR471610 — the baked tile now carries 'Crabière' as C3 A8; S-57 ASCII charts and native S-101 (already UTF-8) are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reads native S-101 ENC charts directly, auto-detected from the .000 file, feeding the portrayal pipeline without the S-57→S-101 adapter (native data already speaks the S-101 vocabulary). The public API is unchanged — png, bake, and the C ABI accept a native S-101 chart transparently.
Validation: SHOM France + UKHO GB production datasets (incl. update chains) and the IHO S-164 update conformance set — all load with 0 unresolved codes / 0 portrayal errors; S-57 rendering unchanged; every chart bakes ≤1.7s.