Various Improvements

[HeDo88TH]

• Added experimental unified viewer and 3d tiles support
• Added import from url
• Improved photogrammetry UI

[Copilot]

Pull request overview

This PR expands the processing/import capabilities of the Registry backend by introducing a single-file URL import flow (with SSRF hardening, size/speed guards, and extension policy), adding per-tool feature gating, and migrating photogrammetry support from NodeODM to NodeODX with direct extraction into datasets (plus processing node discovery/status endpoints).

Changes:

• Add SSRF-hardened single-file URL import (verify/probe + import-file heavy tool) and enforce extension policy across ingestion paths.
• Introduce config-driven heavy tool gating (hide/disable tools; role/org allowlists) and surface gating state in tool/feature DTOs.
• Replace NodeODM integration with NodeODX; upgrade photogrammetry to extract results into datasets and expose NodeODX node/options/status via system endpoints.

Reviewed changes

Copilot reviewed 50 out of 50 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
Registry.Web/Utilities/ProcessingPlatformServiceCollectionExtensions.cs	Switch NodeODM DI registrations to NodeODX and register tool gating service.
Registry.Web/Utilities/ImportServiceCollectionExtensions.cs	Register guarded downloader and new import-file heavy tool.
Registry.Web/Startup.cs	Register IFileUrlImportManager for URL import orchestration.
Registry.Web/Services/Ports/IFileUrlImportManager.cs	New web-layer port for URL import verify/import orchestration.
Registry.Web/Services/Managers/PushManager.cs	Add extension-policy enforcement during push commit.
Registry.Web/Services/Managers/ObjectsManager.cs	Enforce extension policy for uploads/stream commits/transfers/copies.
Registry.Web/Services/Managers/FileUrlImportManager.cs	Implement verify/import orchestration and credential encryption for job payload.
Registry.Web/Services/Import/GuardedHttpDownloader.cs	Add SSRF-hardened probe/download with size/disk/speed guards.
Registry.Web/Services/Import/FileImportPolicy.cs	Add pure helpers for URL validation + safe filename/path derivation.
Registry.Web/Services/Import/ArchiveUrlImportSource.cs	Reuse guarded downloader, add archive content validation, apply extension policy and path sanitization.
Registry.Web/Services/HeavyTasks/Tools/PhotogrammetryTool.cs	Migrate to NodeODX; extract results into dataset/new dataset and enqueue builds/cache invalidation.
Registry.Web/Services/HeavyTasks/Tools/FileUrlImportTool.cs	New worker-side heavy tool for importing a single file from URL into dataset.
Registry.Web/Services/HeavyTasks/Tools/ArchiveExtractTool.cs	Apply extension policy + segment sanitization during extraction.
Registry.Web/Services/HeavyTasks/Ports/IHeavyToolGating.cs	New port for computing effective per-caller heavy tool gating state.
Registry.Web/Services/HeavyTasks/NodeOdm/NodeOdxNodeRegistry.cs	NodeODX node registry bound from config.
Registry.Web/Services/HeavyTasks/NodeOdm/NodeOdxModels.cs	NodeODX endpoint + task/info/options models.
Registry.Web/Services/HeavyTasks/NodeOdm/NodeOdxClient.cs	NodeODX HTTP client + parsers + options retrieval.
Registry.Web/Services/HeavyTasks/NodeOdm/NodeOdmModels.cs	Remove NodeODM models.
Registry.Web/Services/HeavyTasks/NodeOdm/INodeOdxNodeRegistry.cs	Rename/update node registry interface to NodeODX.
Registry.Web/Services/HeavyTasks/NodeOdm/INodeOdxClient.cs	Rename/update node client interface to NodeODX and add options endpoint.
Registry.Web/Services/HeavyTasks/Adapters/HeavyToolGating.cs	Implement feature gating evaluation (availability/role/org allowlists).
Registry.Web/Services/HeavyTasks/Adapters/HeavyTaskQuota.cs	Add per-user concurrency cap for import-file tasks.
Registry.Web/Services/Adapters/ConfigurationDataBuilder.cs	Add Tool Gating section and JSON field helper; update various field list initializers.
Registry.Web/Models/DTO/UrlImportDto.cs	Add DTOs for URL import verify/import requests and results.
Registry.Web/Models/DTO/TaskDtos.cs	Extend tool DTO to include gating flags and message.
Registry.Web/Models/DTO/ProcessingNodeStatusDto.cs	Add DTO for NodeODX node status endpoint.
Registry.Web/Models/DTO/ProcessingNodeDto.cs	Add DTO for configured processing node list endpoint.
Registry.Web/Models/DTO/NodeOdxOptionDto.cs	Add DTO for NodeODX options endpoint.
Registry.Web/Models/DTO/FeaturesDto.cs	Extend tool info DTO with gating flags/message.
Registry.Web/Models/ControllerBaseEx.cs	Mark extension-block rejections as non-retriable.
Registry.Web/Models/Configuration/ProcessingPlatformSettings.cs	Add NodeODX config, tool gating config, and URL import concurrency cap setting.
Registry.Web/Models/Configuration/ImportSettings.cs	Add per-file import cap, speed guard knobs, and extension allow/block-list policy.
Registry.Web/Models/Configuration/HeavyToolConfig.cs	Add per-tool gating configuration model.
Registry.Web/Middlewares/DownloadLimitMiddleware.cs	Include build artifact downloads in download concurrency limiter.
Registry.Web/Exceptions/ExtensionBlockedException.cs	New non-retriable exception for extension policy violations.
Registry.Web/Controllers/TasksController.cs	Enforce gating server-side on submit and return gating state in tool listings.
Registry.Web/Controllers/SystemController.cs	Add NodeODX node/options/status endpoints and compute gating state in features payload.
Registry.Web/Controllers/DatasetImportController.cs	Add REST endpoints for URL import verify/import under dataset route.
Registry.Web/appsettings-default.json	Add default blocked extension list configuration under Import.
Registry.Web.Test/ProcessingPlatformTests.cs	Add unit tests for HeavyToolGating behavior.
Registry.Web.Test/NodeOdxTests.cs	Rename/update tests to NodeODX and extend coverage.
Registry.Web.Test/ImportSettingsTests.cs	Add tests for extension policy and per-file size cap selection.
Registry.Web.Test/HeavyTaskQuotaTests.cs	Add tests for per-user import-file concurrency cap.
Registry.Web.Test/GuardedHttpDownloaderTests.cs	Add tests for probe fallback, size caps, and low-speed guard.
Registry.Web.Test/FileUrlImportToolTests.cs	Add validation/plan tests for import-file tool.
Registry.Web.Test/FileImportPolicyTests.cs	Add tests for URL validation and filename sanitization.
Registry.Ports/Archives/IArchiveExtractor.cs	Add content-based archive validation API.
Registry.Adapters/Archives/SharpCompressArchiveExtractor.cs	Implement IsValidArchive via content probing.
Registry.Adapters.Test/SharpCompressArchiveExtractorTest.cs	Add tests for archive content validation.

[Copilot]

Pull request overview

Copilot reviewed 51 out of 51 changed files in this pull request and generated 7 comments.