[CMPT-6415]: propagate pipelines-api latest changes by sunny2get · Pull Request #604 · datarobot-oss/cli

sunny2get · 2026-06-24T18:59:27Z

RATIONALE

Propagates the public API changes from pipelines-api. The following actions have user-facing surface:

Image schema: pip/conda/baseImage/nvidia fields (partially landed in a prior commit; this PR completes it with the conda anyOf codec and CLI flags)
Pipeline ↔ image linkage: image_id form field on create/update, linkedImage block on pipeline detail
inputSetTemplate on pipeline detail
Optional imageId override on run dispatch
New GET /pipelines/{id}/source endpoint → dr pipeline source subcommand

CHANGES

`internal/pipeline/image.go`

Add CondaSpec and CondaValue types with custom MarshalJSON/UnmarshalJSON for the anyOf[[]string, CondaSpec] wire format
Add Conda, BaseImage, Nvidia to ImageCreateRequest, ImageUpdateRequest, and ImageDefinition
Update CreateImage and UpdateImage signatures with new params

`cmd/pipeline/image/create` and `cmd/pipeline/image/update`

Add --conda, --conda-channel, --base-image, --nvidia flags
Validation: at least one of --package (pip) or --conda is required; --conda-channel requires at least one --conda package

`internal/pipeline/pipeline.go`

Add LinkedImageBlock struct; add ImageID, LinkedImage, InputSetTemplate to Pipeline
Thread imageID through CreatePipeline and UpdatePipeline as multipart form field

`cmd/pipeline/create` and `cmd/pipeline/update`

Add optional --image flag

`internal/pipeline/run.go`

Add ImageID, ImageVersion to Run; add ImageID to RunCreateRequest; thread through CreateRun

`cmd/pipeline/run/create`

Add optional --image flag to override the pipeline's linked image at dispatch time

`internal/pipeline/source.go` + `cmd/pipeline/source/cmd.go` (new)

GetPipelineSource wrapping GET /pipelines/{id}[/versions/{v}]/source
dr pipeline source --pipeline <id> [--scope locked --version N] [--output-format json]

Output

PrintImageHuman: versions table now shows PIP, CONDA, BASE IMAGE columns
printPipelineHuman: shows linked image and input template when present
PrintRunHuman: shows image ID and version when present

Note

Medium Risk
Touches pipeline create/update/run dispatch and image versioning semantics (replacement vs append); incorrect image specs or overrides could affect execution environments, but changes are API-aligned with validation on image commands.

Overview
Aligns the CLI with recent pipelines-api schema changes for execution images, pipeline–image linkage, runs, and source inspection.

Execution images now use a full definition model (pip, conda, baseImage, nvidia) instead of a flat package list. image create/update add --conda, --conda-channel, --base-image, and --nvidia; updates are documented as full replacement specs (not append). CondaValue handles the API’s anyOf conda wire format.

Pipelines accept optional --image on create and update (image_id multipart field). Detail/create output surfaces linked image, image ID, and input set template when the API returns them.

Runs support optional --image to override the linked execution image at dispatch; run display includes image ID/version.

New dr pipeline source fetches draft or locked pipeline source.py (JSON or raw text), with a soft message on 404.

^{Reviewed by Cursor Bugbot for commit bae297d. Configure here.}

cursor

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit bae297d. Configure here.}

github-actions · 2026-06-25T00:58:53Z

🔐 Fork smoke tests started by maintainer

⏳ Security scans passed. Running smoke tests...

Commit: b5754a354c07a71634e4fc0bd524dfc7b4c542d7
View run

Copilot

Pull request overview

This PR updates the DataRobot CLI pipeline functionality to match recent pipelines-api public schema changes, including richer execution image definitions, pipeline↔image linkage, run-time image overrides, and a new command to fetch a pipeline’s uploaded source.

Changes:

Expanded execution image support to include pip, conda (anyOf wire format), baseImage, and nvidia, and updated create/update flows and output rendering accordingly.
Added image association fields to pipeline create/update/detail responses and added an optional image override when dispatching runs.
Introduced dr pipeline source to fetch and print pipeline source for draft or locked versions (text or JSON).

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
internal/pipeline/source.go	Adds client wrapper for pipeline source endpoints.
internal/pipeline/run.go	Threads optional `imageId` through run dispatch and response models.
internal/pipeline/run_test.go	Updates run dispatch tests for the new `imageId` parameter.
internal/pipeline/run_output.go	Surfaces run image ID/version in JSON and human output.
internal/pipeline/pipeline.go	Adds pipeline linked-image/input-template fields and threads `image_id` through multipart create/update.
internal/pipeline/pipeline_output.go	Displays linked image and input template in human output; shows image ID in create response.
internal/pipeline/image.go	Adds conda anyOf codec + new image definition schema; updates create/update request shapes and update flow.
internal/pipeline/image_test.go	Updates image tests for new schema and update GET-then-PATCH behavior.
internal/pipeline/image_output.go	Updates image output (PIP/CONDA/BASE IMAGE columns) and adds conda formatting/helpers.
internal/pipeline/image_output_test.go	Adds tests for CondaValue codec and output rendering.
docs/commands/pipelines-reference.md	Updates reference docs for new flags/fields and adds `pipeline source`.
docs/commands/pipeline.md	Updates narrative docs for image definition model, pipeline image linkage, run override, and source command.
cmd/pipeline/update/cmd.go	Adds `--image` flag and passes image ID through pipeline update.
cmd/pipeline/source/cmd.go	Adds `dr pipeline source` command implementation and 404 handling.
cmd/pipeline/run/create/cmd.go	Adds `--image` override flag for run dispatch.
cmd/pipeline/image/update/cmd.go	Adds conda/base-image/nvidia flags + validation; updates call to new UpdateImage signature.
cmd/pipeline/image/create/cmd.go	Adds conda/base-image/nvidia flags + validation; updates call to new CreateImage signature.
cmd/pipeline/image/cmd.go	Updates parent command help text for the updated image model.
cmd/pipeline/create/cmd.go	Adds `--image` flag and passes image ID through pipeline create.
cmd/pipeline/cmd.go	Registers the new `pipeline source` subcommand.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

github-actions · 2026-06-25T01:05:03Z

✅ All smoke tests passed! (Fork PR)

✅ Security Scan: success
✅ Linux: success
✅ Windows: success

View run details

The pipelines-api replaced the flat `packages` field with a nested `definition` object containing `pip`, `baseImage`, and `nvidia` fields. This commit updates the CLI client types, output rendering, update semantics (GET-first to resolve canonical name), tests, and command help text to match the new schema. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

- Add conda/baseImage/nvidia fields to image create and update (action 120): - CondaSpec/CondaValue types with custom anyOf JSON marshal/unmarshal - --conda, --conda-channel, --base-image, --nvidia CLI flags - Validation: at least one of pip or conda required; channels require deps - Add linkedImage and inputSetTemplate to pipeline detail (actions 113, 111) - Thread imageId through pipeline create/update multipart form (action 113) - Add optional --image override to `dr pipeline run create` (action 116) - Add `dr pipeline source` subcommand for GET /pipelines/{id}/source (action 119) Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

- image section: add conda/base-image/nvidia flags, update description - pipeline create/update: document --image flag - run create: document --image override flag - add source subcommand docs (dr pipeline source) - add source endpoint to quick reference table - document conditional Image/inputSetTemplate fields in get output Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

When conda is specified as a structured CondaSpec (channels + deps), the human-readable versions table was silently dropping the channels and only showing deps. Add formatCondaCell to render the full value: - plain list: "scipy,numpy" - channels only: "[conda-forge]" - channels + deps: "[conda-forge] numpy=1.21.*" Also add unit tests covering CondaValue marshal/unmarshal round-trips, formatCondaCell variants, and human/JSON output assertions. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

- Clear Channels in CondaValue.UnmarshalJSON plain-list branch to prevent stale state on reuse - formatCondaCell: treat emptyValuePlaceholder as absent deps so channel-only spec renders as [chan] not [chan] — - joinPackages: return emptyValuePlaceholder for empty slice so PIP column shows — instead of blank - Update image.go doc comment and cmd/pipeline/image Long help to reflect conda/base-image/nvidia support Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

API can return conda: [] for pip-only images; skip the field in the DTO when both Deps and Channels are empty to avoid emitting "conda": []. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

ajalon1

Looks pretty reasonable. Some general questions given the pipelines API has changed again. Are you concerned with users dealing w. backwards incompatible changes? I assume not given there's still a feature gate.

Also, I made some enhancements to --output-format json in #582 and I think you should consider using them, specifically PrintJSONEnvelope().

ajalon1 · 2026-06-25T19:12:59Z

 	cmd.Flags().StringVar(&description, "description", "", "Optional description for the pipeline")
 	cmd.Flags().StringVar(&name, "name", "", "Optional human-readable display name; defaults to the title-cased @pipeline function name")
 	cmd.Flags().StringVar(&mode, "mode", "", "Pipeline mode: draft or locked")
+	cmd.Flags().StringVar(&imageID, "image", "", "Execution image ID to associate with this pipeline")


I assume users will know how to find this execution image ID.

ajalon1 · 2026-06-25T19:13:25Z

+supplied package definition. The image may be referenced by pipelines once
 its first version reaches the READY state.

+At least one of --package (pip) or --conda must be provided.


ajalon1 · 2026-06-25T19:17:13Z

+			if conda != nil && len(conda.Deps) == 0 {
+				return errors.New("--conda-channel requires at least one --conda package")


Is the inverse true? --conda requires --conda-channel? It doesn't make sense to me, I can see using the default conda channel for packages, but just want to be sure.

ajalon1 · 2026-06-25T19:36:47Z

+func handleSourceError(err error, pipelineID string) error {
+	var httpErr *drapi.HTTPError
+
+	if errors.As(err, &httpErr) && httpErr.StatusCode == http.StatusNotFound {
+		fmt.Println(tui.DimStyle.Render("No source available for pipeline: " + pipelineID))
+
+		return nil
+	}
+
+	return err
+}


when output format is JSON, I think you should either raise an error or use PrintJSONEnvelope() and add an error message within the envelope object.

sunny2get changed the title ~~feat(pipeline): propagate pipelines-api actions 108-121 schema changes~~ [YOLO]: propagate pipelines-api actions 108-121 schema changes Jun 24, 2026

sunny2get changed the title ~~[YOLO]: propagate pipelines-api actions 108-121 schema changes~~ [CMPT-6415]: propagate pipelines-api actions 108-121 schema changes Jun 24, 2026

cursor Bot reviewed Jun 24, 2026

View reviewed changes

Comment thread internal/pipeline/image_output.go

sunny2get changed the title ~~[CMPT-6415]: propagate pipelines-api actions 108-121 schema changes~~ [CMPT-6415]: propagate pipelines-api latest changes Jun 24, 2026

sunny2get force-pushed the sunny/image-schema-pip-migration branch from b990d83 to 47ebb32 Compare June 24, 2026 23:57

sunny2get requested review from ajalon1 and Copilot June 25, 2026 00:55

Copilot started reviewing on behalf of sunny2get June 25, 2026 00:58 View session

Copilot AI reviewed Jun 25, 2026

View reviewed changes

Comment thread internal/pipeline/image_output.go Outdated

Comment thread internal/pipeline/image.go

Comment thread cmd/pipeline/image/cmd.go

Comment thread internal/pipeline/image.go Outdated

Comment thread internal/pipeline/image_output.go

sunny2get force-pushed the sunny/image-schema-pip-migration branch from 2d083ca to 09c2e36 Compare June 25, 2026 19:02

sunny2get and others added 6 commits June 25, 2026 16:13

fix(pipeline): omit conda field in JSON when image has no conda packages

e2898ef

API can return conda: [] for pip-only images; skip the field in the DTO when both Deps and Channels are empty to avoid emitting "conda": []. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

sunny2get force-pushed the sunny/image-schema-pip-migration branch from 09c2e36 to e2898ef Compare June 25, 2026 20:13

ajalon1 reviewed Jun 26, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[CMPT-6415]: propagate pipelines-api latest changes#604

[CMPT-6415]: propagate pipelines-api latest changes#604
sunny2get wants to merge 6 commits into
datarobot-oss:mainfrom
sunny2get:sunny/image-schema-pip-migration

sunny2get commented Jun 24, 2026 •

edited

Loading

Uh oh!

cursor Bot left a comment

Uh oh!

Uh oh!

github-actions Bot commented Jun 25, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jun 25, 2026

Uh oh!

ajalon1 left a comment

Uh oh!

ajalon1 Jun 25, 2026

Uh oh!

ajalon1 Jun 25, 2026

Uh oh!

ajalon1 Jun 25, 2026

Uh oh!

ajalon1 Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		if conda != nil && len(conda.Deps) == 0 {
		return errors.New("--conda-channel requires at least one --conda package")

Uh oh!

Conversation

sunny2get commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

RATIONALE

CHANGES

internal/pipeline/image.go

cmd/pipeline/image/create and cmd/pipeline/image/update

internal/pipeline/pipeline.go

cmd/pipeline/create and cmd/pipeline/update

internal/pipeline/run.go

cmd/pipeline/run/create

internal/pipeline/source.go + cmd/pipeline/source/cmd.go (new)

Output

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Jun 25, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jun 25, 2026

Uh oh!

ajalon1 left a comment

Choose a reason for hiding this comment

Uh oh!

ajalon1 Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

ajalon1 Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

ajalon1 Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

ajalon1 Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sunny2get commented Jun 24, 2026 •

edited

Loading

`internal/pipeline/image.go`

`cmd/pipeline/image/create` and `cmd/pipeline/image/update`

`internal/pipeline/pipeline.go`

`cmd/pipeline/create` and `cmd/pipeline/update`

`internal/pipeline/run.go`

`cmd/pipeline/run/create`

`internal/pipeline/source.go` + `cmd/pipeline/source/cmd.go` (new)