Skip to content

Latest commit

 

History

History
171 lines (146 loc) · 36.2 KB

File metadata and controls

171 lines (146 loc) · 36.2 KB

Format decode progress

Generated by tools/report_format_decode_progress.py --write from schemas/archive_content_capabilities.v1.json. Do not edit by hand: change the manifest entry and regenerate, so the status a modder reads and the status the Archive Browser reports can never disagree.

Where we are

Of 141 known file formats, 89 are engine formats (Pearl Abyss or licensed middleware) that had to be reverse engineered, and 77 of those actually appear in the shipped build. That last number is the honest denominator: a format the game does not contain cannot be modded and should not count against progress.

Scope Formats Read coverage Write coverage
Engine formats the build ships 77 41.7% 26.0%
Weighted by archive file count 1,383,187 files 65.9% 53.8%
Engine formats (proprietary + middleware) 89 41.9% 27.5%
Pearl Abyss formats only 82 42.6% 28.0%
All formats 141 54.3% 33.0%

Coverage is a weighted mean, not a file count: decode full = 1.0, decode partial = 0.6, decode surface = 0.3, decode none = 0.0; write full = 1.0, write constrained = 0.5, write none = 0.0.

By area

Area Formats Read Write
animation_scene 15 ########............ 42.0% #####............... 23.3%
audio_video 18 #########........... 47.2% ##.................. 8.3%
material_metadata 62 #########........... 46.0% #######............. 33.1%
model_mesh_physics 19 ###########......... 54.2% #######............. 36.8%
texture_image 12 ################.... 80.0% ###................. 16.7%
user_interface_text 15 ##################.. 88.7% ################.... 80.0%

What works, and what is left

Engine formats only, worst first. Open formats are listed at the end for completeness.

Format Files Read Write Priority What is left
.hkx 58,031 partial constrained high structural edits (topology, counts, references, strings, arrays) blocked pending semantic rebuild proof - no new collision shapes or ragdoll bodies
.levelinfo 14,478 surface none high placement records not parsed - level layout cannot be edited
.paac 520 partial constrained high chart node structure around the strings is not parsed, so only same-length retargets are allowed
.palevel 21,119 surface none high placement records not parsed - level layout cannot be edited
.parg 750 partial none high the pointer-addressed heap walk does not complete, so nested values are not read; there is no writer for this extension yet. This is the particle render schema, so closing it opens VFX modding The heap walk diverges from .prefab in the pointee trailer: where a prefab repeats the consumed length, these files carry a padding byte and then a small value of 1, 2 or 3 (measured as 256/512/768 when read one byte early across 747 of the 882 files), so it is a kind tag rather than a length. That is where a future attempt should start.
.pat 1,397 partial none high no builder, and LOD1+ plus unrecognised vertex layouts stay explicitly undecoded - static world geometry is view-only
.prefab 47,343 partial constrained high 38% of archive prefabs do not walk to completion (62.0% of a seeded 1,500-file sample, up from 58.5% once the blob's close was understood as unread pointee length fields rather than a trailer grammar). The collection-header width rule behind much of that is not determined by member metadata. Telemetry over 7,121 collection sites in 2,500 random prefabs: every collection member carries flags=7, 4 of the 12 distinct (member name, flags) tuples require both the narrow and the wide header form, and 6,183 of 7,121 sites are ambiguous because both widths yield a plausible count -- so the shipped plausibility heuristic chooses arbitrarily at 87% of sites and is unsound even where a walk completes. Two hypotheses are dead: width from the element type in the type table (circular, kind 7 declares the generic ReflectObjectPtr) and width from member metadata (refuted by the telemetry). The discriminator is upstream in component selection, and it is not the mask arithmetic. At every one of 247 sampled component-selection failures the file states no type index (type_index out of range, 100%), so the walk is guessing by declaration order. The masks themselves look explicable: strip the top set bit and all of them fit the members available -- 0x0110 needs 5 against 5, 0x091a needs 9 against 11, 0x0100 needs 0 against 8. That top-bit-as-sentinel reading was implemented and measured, and it is rejected: completion moved 59.6% to 59.8% (+8 of 4,000), the mask errors were merely replaced by fresh ones and by blob-string-length desyncs further on. It fits the arithmetic without being the rule. The external source needed for marker-1 component identity looks like it is already in the archives: .prefabdata_xml, 2,613 files, is an XML serialisation of the same reflection data and 399 of 400 sampled pair with a .prefab binary of the same stem. Each names its root type and its members directly -- for example HeadPrefabData with SkeletonVariationName, FacialAnimationIntensityMask, FacialAnimationMask, MorphTargetSet, SkeletonMorphMask and EmotionAnimationSet. That is the same pairing trick that confirmed .paprojdesc against its XML. That pairing was built and measured, and it does not help: of 2,609 stems that pair, 2,607 (99.9%) already walk to completion, and only 2 fail. The XML-paired set is almost exactly the easy subset -- small character prefabs with simple roots such as HeadPrefabData and NudePrefabData -- so the 40% of the corpus that fails has essentially no XML counterpart and the oracle gives no leverage on marker-1 component identity. Ruled out rather than assumed. A better lead came out of the executable: the reflection system behind this container appears to be Havok's, not a Pearl Abyss one. CrimsonDesert.exe carries hkReflect RTTI throughout -- MutableTypeReg, BuiltinTypeReg, AttributeArrayWithOwnership, Version::PatchRegistry, Version::PatchSet, ClonerCallback -- with a Pearl Abyss layer above it (AutoReflectBase@pa, ReflectObjectIterator@pa, IReflectObjectResourceBase@pa, CustomAttributeReflectObject@pa, EnumReflectPropertyBase@pa). ReflectObject appears 737 times and SceneObject 504. If component identity is resolved through a Havok type registry rather than through declaration order, then marker-1 identity may be recoverable from that registry, and hkReflect is documented middleware rather than a format that has to be derived from the bytes. The version is Havok_2024, which appears 107 times in the executable, and the hope that this makes it documented middleware is wrong. The classic Havok reflection registry names are absent: hkTypeInfoRegistry, hkBuiltinTypeRegistry and hkDynamicClassNameRegistry all occur zero times, while hkReflect occurs 191 times and hkSerialize 54. So this is the newer hkReflect system rather than the hkClass one that public Havok documentation and the community .hkx tooling both target, and it postdates the withdrawal of Havok's public SDK docs. Practically that means component resolution has to be reverse engineered from this binary like anything else, and existing Havok knowledge is aimed at the wrong system. Reading the resolution out of the runtime is now ruled out too, for a reason that reframes the format: most of the component types in shipped prefabs do not exist in the runtime. Of 488 distinct component type names sampled from 600 archive prefabs, only 75 (15%) appear as strings in CrimsonDesert.exe, and the absent ones include HDAComponent, HoudiniOutput, HoudiniParameterBool, HoudiniParameterInt, HoudiniParameterResourcePath and HoudiniSubnetInputSpline -- Houdini digital-asset authoring types the game never instantiates. The 75 that are present are scattered rather than tabled: only 8 of 74 consecutive names sit within 80 bytes of each other, so there is no contiguous registry to read. So .prefab carries authoring-time reflection data much of which has no runtime counterpart, which is the same pattern as .papr being a 3ds Max rig export, and marker-1 component identity cannot be recovered from a runtime type registry that does not contain the types. Exhaustive backtracking over candidate components, using 'the blob closes exactly' as the oracle rather than any external source, was also tried and does not work: over 600 prefabs it closed 5 of 228 failures, +0.8%, with 4 of the 5 at a single choice point. So the failures are not mis-picked candidates among plausible ones -- either no assignment closes the blob or the walk dies before reaching a choice point. Seven hypotheses have now been retired with measurements. Separately, the worry that local collection ambiguity makes a completed walk untrustworthy was tested and does not hold: over 1,500 prefabs the shipped narrow-first reader completes 916 files, 913 of which contain at least one locally ambiguous site, yet forcing the wide reading at those sites leaves only 11 completing. A wrong width desynchronises everything after it and the blob then fails to close, so completion is globally determined even where individual sites are not, and it is strong evidence of a correct parse. Array resizing is implemented and on: duplicate or remove a whole collection element, from the Prefab Inspector's row menu. It still has no oracle -- over 3,000 sampled prefabs only 6 pairs share a type signature and differ by one object, and those are unrelated assets rather than the same asset with a child added (cd_gimmick_steam_engine_train_rail_0001 against cd_tower_wooden_set13_06), so nothing equivalent to the 10,066 same-asset path pairs that validated the path rewriter exists. It is validated the other way instead, exactly as this entry previously said it would have to be: duplicate an element, remove the duplicate, demand the original bytes back -- 796 of 796 byte-exact over a 1,500-file sample. Every result is read back before it is returned and refused unless the walk completes, the collection carries the new count, the file is the size the splice implies, every pointer still satisfies value == offset + 4, and the copy reads back the size of its source. The hazard that had to be ruled out was the 8-byte owner preceding each pointer record: measured over 1,500 prefabs it is either NULL_OWNER or a small ordinal, and none of the 706 non-null owners fell inside the data blob, so a splice does not relocate them. The 11 files that complete both ways are genuinely undetermined and are the only known population where a complete walk may still be wrong Value editing is scoped rather than blocked on this: offset-keyed fixed-size writes with a source digest and expected-old bytes, on objects whose component type the file stated
.bnk 3,186 surface none medium HIRC event/action tables not parsed, so sounds can be swapped but not added
.meshinfo 35,310 surface none medium count/offset tables are unproven, which is why mesh replacement treats .meshinfo as read-only - physics bounds and socket context cannot be edited
.motionblending 1,637 surface none medium blend graph not parsed
.paa_metabin 162,179 surface none medium record layout not parsed
.pab 257 partial none medium unknown and truncated variants fall back to a best-effort scan, and there is no writer - bones cannot be added, removed or renamed
.padxil 89,824 surface none medium the bytecode itself is not disassembled here; that needs dxc/dxil-dis, and there is no route to recompile an edited shader back into the cache
.pae 6,109 surface none medium parameter tables not parsed - VFX authoring is closed
.paem 560 surface none medium parameter tables not parsed - VFX authoring is closed
.paschedule 4,143 surface none medium schedule entries not parsed - NPC routines cannot be retimed or rerouted
.paschedulepath 3,613 surface none medium schedule entries not parsed - NPC routines cannot be retimed or rerouted
.paseq 4,688 surface none medium track and event layout not parsed - cutscene authoring is closed
.paseqc 2,938 surface none medium track and event layout not parsed - cutscene authoring is closed
.pasg 40 partial none medium the pointer-addressed heap walk does not complete, so nested values are not read; there is no writer for this extension yet The heap walk diverges from .prefab in the pointee trailer: where a prefab repeats the consumed length, these files carry a padding byte and then a small value of 1, 2 or 3 (measured as 256/512/768 when read one byte early across 747 of the 882 files), so it is a kind tag rather than a length. That is where a future attempt should start.
.pastage 3,321 surface none medium track and event layout not parsed - cutscene authoring is closed
.pathc 1 partial none medium no writer, and the payload beyond the parsed header is unmodelled
.pcg 92 partial none medium the pointer-addressed heap walk does not complete, so nested values are not read; there is no writer for this extension yet. Convex collision hulls, so a writer would allow custom collision
.uianiminit 883 surface none medium record layout not parsed
.wem 375,762 partial constrained medium only uncompressed PCM is re-encoded; Vorbis/Opus streams cannot be authored
.ani 2 none none low unknown; rare in shipped archives
.app_xml 5,645 full full low outside the structured sidecar editor, so edits are unchecked text edits
.binarygimmick 13,461 surface none low record layout not parsed
.binarygimmickcacheddata 1 none none low container identified, record layout unknown
.binarygimmickframeevent 1 none none low container identified, record layout unknown
.binarystring 1 none none low these entries are encrypted with a key CDMW does not have; nothing can be decoded until that is solved
.dat 17 none none low container identified, record layout unknown
.imp 1,489 none none low container identified, record layout unknown
.impostor 45 none none low container identified, record layout unknown
.linkedsceneobject 1 none none low these entries are encrypted with a key CDMW does not have; nothing can be decoded until that is solved
.material 400 full full low no schema model, so edits are unchecked text edits
.mi 24 full full low no schema model, so edits are unchecked text edits
.paacdesc 1 none none low container identified, record layout unknown
.paasmt 1 none none low these entries are encrypted with a key CDMW does not have; nothing can be decoded until that is solved
.paatt 223 none none low container identified, record layout unknown
.pabc 459 surface none low field semantics unknown
.pabgb 134 surface none low field semantics unknown
.pabgh 134 surface none low field semantics unknown
.pabv 101 surface none low field semantics unknown
.paccd 1,644 surface none low record layout not parsed
.pai 2 none none low unknown; rare in shipped archives
.pamhc 1 surface none low record layout not parsed
.pamlod 32,884 full full low topology-changing rebuild is LOD0-only; LOD1+ can be read but not re-authored
.pampg 15,291 surface none low field semantics unknown
.pamt 33 none none low nested container copies, not content to decode
.pappt 1 surface none low record layout not parsed
.papr 20 full full low no evidence the runtime reads this format: papr appears zero times as a standalone token across all 40+ shipped binaries where pac, pab, paseq, pamlod, meshinfo and pathc all appear, its internal Local_Euler / ExposeTransform vocabulary appears in no binary in ASCII or UTF-16, and 1,015 character descriptor and model files reference it nowhere. It reads as a 3ds Max rig export left in the archives. The live equivalents are plain XML already: character/descriptors/jiggledescriptor.xml and character/descriptors/posemodifierdata/posemodifierdata.xml. nothing: all 20 rigs parse, tile to their declared entry_count, rebuild byte for byte, and decode every configuration block. cd_m0001_00_circusmachine_boss was the last holdout at 236 starts against a declared 237; the entry it missed is unparented, which the scan had read as a malformed name
.paproj 9 none none low record layout unknown, but the loader is located and the shape of the problem is now clear. CrimsonDesert.exe carries paproj twice, paprojdesc once, ProjectileInfo 44 times and 79 MSVC RTTI class names matching .?AVProjectile@pa@@. Those names map one to one onto the file names: CommonProjectile to commonprojectile.paproj, CommonProjectile_AttachToActor to projectileinfo_attachtoactor.paproj, CommonProjectile_Repeat to _repeat, CommonProjectile_Wave to _wave, and ProjectileShotData to projectileshotinfo. So each file is a table of one projectile subclass, not one shared record type -- which is why no single record size fits the set, and why only projectileshotinfo showed a clean period (156 bytes from about offset 284, confirmed by its 312 harmonic and by -1.0 markers at consistent offsets). Ruled out along the way: the leading u32 does not track file size; no header/record pair in 8..128 bytes divides any file evenly at its declared count; .paac charts name projectile effects only by asset path, never by table index; .paatt is attack info and carries no projectile vocabulary at all. RTTI structures located for ProjectileShotData in CrimsonDesert.exe: TypeDescriptor at rva 0x05f6a810, complete object locator at rva 0x053c168c, class hierarchy descriptor at rva 0x053c0df0. The hierarchy declares one entry and it is the class itself, so ProjectileShotData has no base class and its layout is flat -- no inherited prefix to account for, which fits projectileshotinfo.paproj being the one file with a clean fixed period (156 bytes). Note the class is polymorphic, so the in-memory object starts with an 8-byte vptr that cannot be in the file; the on-disk record is a serialised form, not a memcpy of the class, and field offsets will be shifted by at least that. The vftable is at rva 0x04f22848, reached from the COL, and disassembling around the code that installs it shows the mechanism that actually defines the on-disk layout: a field-by-field deserialiser, not a memcpy. Its shape is 'lea rdx, [rdi + offset]; mov r8d, size; call qword ptr [rax + 8]; test al, al; je fail' repeated once per field, where rdi is the object, rbx is a stream whose vtable slot +8 is Read(dest, size), and the function returns bool. Four fields are legible in the tail seen so far -- +0x10 as 4 bytes, +0x14 as 4, +0x18 as 1, +0x19 as 1 -- which also settles that field sizes are explicit in the code rather than inferred. ProjectileShotData's field list is now complete. Its vftable holds a matching read/write pair at slots 2 and 3 (rva 0x01ea11e0 and 0x01ea1260) and both serialise the same four fields: +0x10 u32, +0x14 u32, +0x18 u8, +0x19 u8 -- ten bytes of payload in an object whose first 0x10 bytes are the vptr and base data that never reach the file. Slots 7/8 and 12/13 are the same pattern for neighbouring classes with 7 and 5 fields, and slots 4, 9 and 14 point into .rdata rather than code, which marks where this vftable ends and the next begins. Note the consequence: ten serialised bytes cannot be the top-level record of projectileshotinfo.paproj, whose period measures 156 bytes, so ProjectileShotData is a component inside a larger record rather than the record itself. Three sibling shot-data classes are now walked as well, and their vftables sit 0x28 apart at 0x04f22848, 0x04f22870, 0x04f22898 and 0x04f228c0, which confirms they are consecutive siblings rather than one class: ProjectileShotData reads +0x10 u32, +0x14 u32, +0x18 u8, +0x19 u8; ProjectilePhysicsShotData reads +0x20 u32, +0x24 u32, +0x28 u32 then four u8 at +0x2c..+0x2f; ProjectileGuideShotData reads five u32 at +0x20..+0x30. The owning class is not among them. ClientProjectile_Shot (vftable 0x04b64768) is the likeliest container but its walk mis-parses, returning a single implausible 515-byte read, so its serialiser needs following by hand rather than by the triple scanner
.paprojdesc 1 full full low nothing: the single shipped file is fully accounted for and cross-checked against its XML
.pas 2 full full low no schema model, so edits are unchecked text edits
.paschedulectx 1 none none low container identified, record layout unknown
.paseqh 1 none none low these entries are encrypted with a key CDMW does not have; nothing can be decoded until that is solved
.pashv 24 none none low container identified, record layout unknown
.pasound 16 none none low container identified, record layout unknown
.pbd 584 none none low container identified, record layout unknown
.pma 2 full full low no schema model, so edits are unchecked text edits
.prefabdata_xml 2,613 full full low outside the structured sidecar editor, so edits are unchecked text edits
.questgaugecount 1 none none low these entries are encrypted with a key CDMW does not have; nothing can be decoded until that is solved
.road 1,697 surface none low spline data not parsed
.roadidx 2 none none low container identified, record layout unknown
.roadsector 5,787 surface none low spline data not parsed
.save 328 none none low these entries are encrypted with a key CDMW does not have; nothing can be decoded until that is solved
.seqmt 614 surface none low record layout not parsed
.spline 59 full full low no schema model, so edits are unchecked text edits
.spline2d 87 full full low no schema model, so edits are unchecked text edits
.3ds none none none the shipped build contains no entry with this extension, so there is nothing here to decode; previously recorded as: third-party interchange; import goes through glTF/GLB instead
.fbx none none none the shipped build contains no entry with this extension, so there is nothing here to decode; previously recorded as: third-party interchange; import goes through glTF/GLB instead
.hkt partial constrained none the shipped build contains no entry with this extension, so there is nothing here to decode; previously recorded as: same structural gate as .hkx
.nav surface none none the shipped build contains no entry with this extension, so there is nothing here to decode; previously recorded as: navmesh geometry not parsed - NPCs cannot be taught new ground
.paa 316,059 full full none the u32 at the head of a packed clip's prelude, the bounds frames and the two bytes before each float translation key are carried verbatim rather than modelled, so they can be preserved but not authored from nothing
.pac 12,962 full full none Nothing outstanding.
.pac_xml 12,886 full full none Nothing outstanding.
.pagbg surface none none the shipped build contains no entry with this extension, so there is nothing here to decode; previously recorded as: field semantics unknown
.paloc 14 full full none the category field is an opaque id; describe_categories reports the dominant key prefix per category from the data rather than the engine's own name for it
.pam 51,466 full full none Nothing outstanding.
.pam_xml full full none Nothing outstanding.
.pami 67,759 full full none Nothing outstanding.
.pamlod_xml full full none Nothing outstanding.
.patx none none none the shipped build contains no entry with this extension, so there is nothing here to decode; previously recorded as: container purpose unknown
.prefab_xml full full none the shipped build contains no entry with this extension, so there is nothing here to decode; previously recorded as: outside the structured sidecar editor, so edits are unchecked text edits
.shader full full none no schema model, so edits are unchecked text edits
.technique 16 full full none Nothing outstanding.
.texture none none none the shipped build contains no entry with this extension, so there is nothing here to decode; previously recorded as: wrapper purpose unknown - likely a texture descriptor or streaming stub
.ui none none none nothing: the game's UI is authored as HTML and CSS under ui/, which CDMW already reads and writes. There is no binary widget format to decode

A dash in Files means the shipped build contains no entry with that extension. Those rows are carried for compatibility, not because there is work in them.

Open formats

.aac (surface/none), .avi (surface/none), .bk2 (surface/none), .bmp (full/none), .cfg (full/full), .css (full/full), .csv (full/full), .dae (full/none), .dds (full/full), .flac (surface/none), .gif (full/none), .glb (full/full), .gltf (full/full), .h (full/full), .hdr (surface/none), .hpp (full/full), .html (full/full), .ies (full/none), .ini (full/full), .jpeg (full/none), .jpg (full/none), .json (full/full), .log (full/full), .lua (full/full), .m4a (surface/none), .m4v (surface/none), .mdl (none/none), .mesh (none/none), .model (none/none), .mov (surface/none), .mp3 (full/none), .mp4 (full/none), .mpeg (surface/none), .mpg (surface/none), .mtl (full/full), .obj (full/full), .ogg (full/none), .png (full/full), .tga (full/none), .thtml (full/full), .tif (full/none), .tiff (full/none), .ttf (full/none), .txt (full/full), .wav (full/full), .webm (surface/none), .webp (surface/none), .wma (surface/none), .wmv (surface/none), .xml (full/full), .yaml (full/full), .yml (full/full).

Highest-value gaps

Every format marked high: closing it opens a modding category that is currently at zero.

  • .hkx (read partial, write constrained) — structural edits (topology, counts, references, strings, arrays) blocked pending semantic rebuild proof - no new collision shapes or ragdoll bodies
  • .levelinfo (read surface, write none) — placement records not parsed - level layout cannot be edited
  • .paac (read partial, write constrained) — chart node structure around the strings is not parsed, so only same-length retargets are allowed
  • .palevel (read surface, write none) — placement records not parsed - level layout cannot be edited
  • .parg (read partial, write none) — the pointer-addressed heap walk does not complete, so nested values are not read; there is no writer for this extension yet. This is the particle render schema, so closing it opens VFX modding The heap walk diverges from .prefab in the pointee trailer: where a prefab repeats the consumed length, these files carry a padding byte and then a small value of 1, 2 or 3 (measured as 256/512/768 when read one byte early across 747 of the 882 files), so it is a kind tag rather than a length. That is where a future attempt should start.
  • .pat (read partial, write none) — no builder, and LOD1+ plus unrecognised vertex layouts stay explicitly undecoded - static world geometry is view-only
  • .prefab (read partial, write constrained) — 38% of archive prefabs do not walk to completion (62.0% of a seeded 1,500-file sample, up from 58.5% once the blob's close was understood as unread pointee length fields rather than a trailer grammar). The collection-header width rule behind much of that is not determined by member metadata. Telemetry over 7,121 collection sites in 2,500 random prefabs: every collection member carries flags=7, 4 of the 12 distinct (member name, flags) tuples require both the narrow and the wide header form, and 6,183 of 7,121 sites are ambiguous because both widths yield a plausible count -- so the shipped plausibility heuristic chooses arbitrarily at 87% of sites and is unsound even where a walk completes. Two hypotheses are dead: width from the element type in the type table (circular, kind 7 declares the generic ReflectObjectPtr) and width from member metadata (refuted by the telemetry). The discriminator is upstream in component selection, and it is not the mask arithmetic. At every one of 247 sampled component-selection failures the file states no type index (type_index out of range, 100%), so the walk is guessing by declaration order. The masks themselves look explicable: strip the top set bit and all of them fit the members available -- 0x0110 needs 5 against 5, 0x091a needs 9 against 11, 0x0100 needs 0 against 8. That top-bit-as-sentinel reading was implemented and measured, and it is rejected: completion moved 59.6% to 59.8% (+8 of 4,000), the mask errors were merely replaced by fresh ones and by blob-string-length desyncs further on. It fits the arithmetic without being the rule. The external source needed for marker-1 component identity looks like it is already in the archives: .prefabdata_xml, 2,613 files, is an XML serialisation of the same reflection data and 399 of 400 sampled pair with a .prefab binary of the same stem. Each names its root type and its members directly -- for example HeadPrefabData with SkeletonVariationName, FacialAnimationIntensityMask, FacialAnimationMask, MorphTargetSet, SkeletonMorphMask and EmotionAnimationSet. That is the same pairing trick that confirmed .paprojdesc against its XML. That pairing was built and measured, and it does not help: of 2,609 stems that pair, 2,607 (99.9%) already walk to completion, and only 2 fail. The XML-paired set is almost exactly the easy subset -- small character prefabs with simple roots such as HeadPrefabData and NudePrefabData -- so the 40% of the corpus that fails has essentially no XML counterpart and the oracle gives no leverage on marker-1 component identity. Ruled out rather than assumed. A better lead came out of the executable: the reflection system behind this container appears to be Havok's, not a Pearl Abyss one. CrimsonDesert.exe carries hkReflect RTTI throughout -- MutableTypeReg, BuiltinTypeReg, AttributeArrayWithOwnership, Version::PatchRegistry, Version::PatchSet, ClonerCallback -- with a Pearl Abyss layer above it (AutoReflectBase@pa, ReflectObjectIterator@pa, IReflectObjectResourceBase@pa, CustomAttributeReflectObject@pa, EnumReflectPropertyBase@pa). ReflectObject appears 737 times and SceneObject 504. If component identity is resolved through a Havok type registry rather than through declaration order, then marker-1 identity may be recoverable from that registry, and hkReflect is documented middleware rather than a format that has to be derived from the bytes. The version is Havok_2024, which appears 107 times in the executable, and the hope that this makes it documented middleware is wrong. The classic Havok reflection registry names are absent: hkTypeInfoRegistry, hkBuiltinTypeRegistry and hkDynamicClassNameRegistry all occur zero times, while hkReflect occurs 191 times and hkSerialize 54. So this is the newer hkReflect system rather than the hkClass one that public Havok documentation and the community .hkx tooling both target, and it postdates the withdrawal of Havok's public SDK docs. Practically that means component resolution has to be reverse engineered from this binary like anything else, and existing Havok knowledge is aimed at the wrong system. Reading the resolution out of the runtime is now ruled out too, for a reason that reframes the format: most of the component types in shipped prefabs do not exist in the runtime. Of 488 distinct component type names sampled from 600 archive prefabs, only 75 (15%) appear as strings in CrimsonDesert.exe, and the absent ones include HDAComponent, HoudiniOutput, HoudiniParameterBool, HoudiniParameterInt, HoudiniParameterResourcePath and HoudiniSubnetInputSpline -- Houdini digital-asset authoring types the game never instantiates. The 75 that are present are scattered rather than tabled: only 8 of 74 consecutive names sit within 80 bytes of each other, so there is no contiguous registry to read. So .prefab carries authoring-time reflection data much of which has no runtime counterpart, which is the same pattern as .papr being a 3ds Max rig export, and marker-1 component identity cannot be recovered from a runtime type registry that does not contain the types. Exhaustive backtracking over candidate components, using 'the blob closes exactly' as the oracle rather than any external source, was also tried and does not work: over 600 prefabs it closed 5 of 228 failures, +0.8%, with 4 of the 5 at a single choice point. So the failures are not mis-picked candidates among plausible ones -- either no assignment closes the blob or the walk dies before reaching a choice point. Seven hypotheses have now been retired with measurements. Separately, the worry that local collection ambiguity makes a completed walk untrustworthy was tested and does not hold: over 1,500 prefabs the shipped narrow-first reader completes 916 files, 913 of which contain at least one locally ambiguous site, yet forcing the wide reading at those sites leaves only 11 completing. A wrong width desynchronises everything after it and the blob then fails to close, so completion is globally determined even where individual sites are not, and it is strong evidence of a correct parse. Array resizing is implemented and on: duplicate or remove a whole collection element, from the Prefab Inspector's row menu. It still has no oracle -- over 3,000 sampled prefabs only 6 pairs share a type signature and differ by one object, and those are unrelated assets rather than the same asset with a child added (cd_gimmick_steam_engine_train_rail_0001 against cd_tower_wooden_set13_06), so nothing equivalent to the 10,066 same-asset path pairs that validated the path rewriter exists. It is validated the other way instead, exactly as this entry previously said it would have to be: duplicate an element, remove the duplicate, demand the original bytes back -- 796 of 796 byte-exact over a 1,500-file sample. Every result is read back before it is returned and refused unless the walk completes, the collection carries the new count, the file is the size the splice implies, every pointer still satisfies value == offset + 4, and the copy reads back the size of its source. The hazard that had to be ruled out was the 8-byte owner preceding each pointer record: measured over 1,500 prefabs it is either NULL_OWNER or a small ordinal, and none of the 706 non-null owners fell inside the data blob, so a splice does not relocate them. The 11 files that complete both ways are genuinely undetermined and are the only known population where a complete walk may still be wrong Value editing is scoped rather than blocked on this: offset-keyed fixed-size writes with a source digest and expected-old bytes, on objects whose component type the file stated

Rubric

decode

  • full — Record layout is known and checked against the shipped corpus. No unexplained spans in the supported path.
  • partial — The primary payload is decoded, but variants, later LODs, or regions of the file remain unmodelled.
  • surface — Strings, references, and headers are recovered. The record layout is not parsed.
  • none — Raw bytes only.

write

  • full — CDMW can produce game-loadable bytes for this format.
  • constrained — Only edits proven safe are permitted; the rest are gated off deliberately.
  • none — No writer exists.

origin

  • proprietary — Pearl Abyss format. Everything known about it was reverse engineered here.
  • third_party — Licensed middleware format (Havok, Wwise). Documented externally, not by the game.
  • open — Public format with a published specification.
  • unknown — Provenance unconfirmed; may not be engine content.

priority

  • high — Closing this gap unlocks a modding category that is currently at zero.
  • medium — Real value, but a workaround or an adjacent format covers part of it.
  • low — Rare, cosmetic, or already served by another route.
  • none — Nothing meaningful left to do.