generic-devkit: catalog/publish support for the generic BPP - #17
Open
nirmay wants to merge 12 commits into
Open
Conversation
Adds a catalogPull handler alongside bapTxnReceiver/bapTxnCaller on the onix-bap container, serving /catalog/pull as a DS-internal trigger that fetches and verifies a provider node's manifest -> index -> catalog chain.
Uses {{networkId}} and {{bpp_uri}} collection variables, consistent with
the existing Discovery/Transaction requests. Adds bap_base_url since
catalog/pull is a root-level path, not nested under /bap/caller like the
other requests.
Aligns with beckn.yaml Context.receiverId naming (the PN's DID) rather than reusing bppUri. Value is still the PN's URI for now -- real DID resolution isn't implemented, this is a field-name alignment only.
Pins onix-bap/onix-bpp to fidedocker/onix-adapter:1.7.5 (built from beckn-onix's catalog-crawler branch) instead of the implicit :latest, so the team gets the catalog/pull handler without rebuilding locally.
Adds a "Catalog Crawler" section covering what /catalog/pull does, how to trigger it (Postman or curl, using networkId=beckn.one/testnet), and sample COMPLETED/FAILED responses -- so this is discoverable from starter-kit itself, not just beckn-onix's own plugin README.
Reference schema for the details payload of a catalog-index DeDi file (registry.name: "beckn-catalogs"): one record per catalogId, describing its version, lifecycle status, visibility, and the digest/location of its content (plain, unsigned Beckn Catalog JSON at parts[].url). Matches the draft-07, flat, additionalProperties:false style of nfh-trust-labs/DeDi's own Beckn_subscriber.json / Beckn_subscriber_reference.json, intended as a starting point to eventually move there.
The published schema described a single catalog entry from a
pre-file-spec design: visibility as public/{networks}, a flat parts[]
list with no signature at all, no index wrapper. The current file spec's
catalog index has since moved to a signed, versioned model entirely --
this rewrite matches it:
- Top-level document: {participantId, version, next_update, catalogs[]},
not a single entry.
- Each catalog entry is either ACTIVE (catalogType, optional networkIds/
authMethods/schemaTypes/crawlHint, baseline + changes) or RETIRED
(catalogId, status, retiredAt only, enforced via if/then/else so a
tombstone can't also carry files).
- baseline/changes entries are file-entry objects requiring version, with
url/size/digest/signature only required together -- allowing the spec's
lightweight forward-pointer form ({"version": 43}, no file yet).
- Each file's signature is its own object {keyId, value, validUntil} --
the tuple it signs (per its description) now includes size, matching
the file spec's binding update.
- authMethods reshaped to the current method/header/signedHeaders/
freshnessSeconds form (no more algorithm/challenge fields).
Verified against the file spec's own worked example (all three catalog
entries -- ACTIVE with a forward-pointer change, ACTIVE restricted with
authMethods, and a RETIRED tombstone) and against a battery of documents
that should be rejected (the old single-entry shape, a RETIRED entry
carrying a baseline, a fileEntry missing digest, a bad digest pattern,
and the old authMethods shape).
Wires the catalog-publisher plugin into the generic BPP devkit: new catalogPublish module in generic-bpp.yaml, docker-compose updates, Postman requests for the /catalog/publish endpoint, and README docs. Tested end-to-end against the generic devkit.
…ngrok tunnel Adds a /catalog/* static route to beckn-router's Caddyfile, serving ../data/catalog (onix-bpp's catalogPublish outputRoot) read-only through the same port-9000 tunnel the BAP/BPP routes already use -- no second port or tunnel needed. Sets catalogBaseURL in generic-bpp.yaml to that tunnel's URL + /catalog so generated manifest/index/catalog files carry real, publicly fetchable URLs instead of the pending-artifact-store:// placeholder.
Adds schemaValidator (schemav2validator) to the catalogPublish module in generic-bpp.yaml, so submitted catalogs are validated against catalog/publish's real message.catalogs[]:Catalog[] schema before they're published. No type/location set, matching the other modules in this file, so it resolves the canonical protocol-specifications-v2 beckn.yaml via becknConstants. checkPolicy (opapolicychecker) intentionally left out for now.
…to beckn Renames the on-host data dir, container mount, and Caddy static route for catalogPublish's output from catalog to beckn throughout: data/catalog -> data/beckn, /catalog (container) -> /beckn, /srv/catalog -> /srv/beckn, Caddyfile's /catalog/* route -> /beckn/*, and outputRoot/catalogBaseURL in generic-bpp.yaml updated to match. The /catalog/publish API path itself is unrelated and untouched.
… pinned-image compose too
catalogPull is being developed independently now; the beckn-onix
catalog-publisher branch no longer registers that handler type, so
generic-bap.yaml's catalogPull module crashed the BAP on startup
("invalid module: catalogPull"). Removes that module block and its
Postman/README references.
Adds manifestSubscriberId to generic-bpp.yaml's catalogPublisher
config -- the DeDi-native namespace/registry/recordName identifier the
node-manifest self-lookup requires, kept separate from subscriberId
(used for keyset/signing) since the two have different formats.
bppTxnReceiver/bppTxnCaller keep their original bpp.example.com
identity (keys + Postman bpp_id variable) -- only catalogPublish's own
keyManager block and manifestSubscriberId use the new
staging.p-node.fabric.nfh.global test identity, so the regular
select/init/confirm transaction flow keeps signing correctly.
docker-compose-generic.yml (the pinned fidedocker/onix-adapter image,
as opposed to -local.yml's locally-built beckn-onix:latest) was
missing the volume mounts and doc comment catalog-publish needs --
added those too so it's ready once a release image with the
catalogPublish handler is pinned there.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
catalog/publishDS-internal trigger into the generic devkit's BPP (generic-bpp.yaml), backed by beckn-onix'scatalogpublisherplugin/catalogPublishHandler./beckn/*static route ininstall/Caddyfile, backed by a../data/becknbind mount (renamed from an earlier/cataloglayout).schemav2validator) for incoming catalogs, and a manifest-link check (manifestloader) that warns (without writing to the real DeDi manifest) when this publisher's catalog index isn't yet declared there.Beckn Catalog IndexJSON Schema matching the current file spec.catalogPull/catalog-crawler wiring from this branch — that work is proceeding independently, and the handler type it depended on no longer exists on thebeckn-onixcatalog-publisherbranch this devkit points at (its absence was crashing the BPP on startup:invalid module: catalogPull).bppTxnReceiver/bppTxnCallerkeep the originalbpp.example.comidentity so the regular transaction flow keeps signing correctly;catalogPublish's ownkeyManagerblock andmanifestSubscriberIduse a separatestaging.p-node.fabric.nfh.global/ DeDi-registered test identity.docker-compose-generic.yml(pinnedfidedocker/onix-adapterimage) gets the same volume mounts as-local.ymlso it's ready once a release image with thecatalogPublishhandler is pinned there.Test plan
docker compose -f generic-devkit/install/docker-compose-generic-local.yml upbrings up BAP/BPP cleanly (noinvalid modulecrash).discover → ... → confirmtransaction flow still signs/verifies correctly (BPP identity unchanged).POST /catalog/publish(Postman2 — Catalog Publishing / publish) succeeds and writes togeneric-devkit/data/beckn/./beckn/*.beckn-onix:1.7.5is pushed withcatalogPublishsupport,docker-compose-generic.yml(pinned image) also works end-to-end.🤖 Generated with Claude Code