Skip to content

feat: add dremio space commands and reject single-component paths in folder create#24

Open
sandhyasun wants to merge 7 commits into
dremio:mainfrom
sandhyasun:fix/issue-13-space-commands
Open

feat: add dremio space commands and reject single-component paths in folder create#24
sandhyasun wants to merge 7 commits into
dremio:mainfrom
sandhyasun:fix/issue-13-space-commands

Conversation

@sandhyasun
Copy link
Copy Markdown

@sandhyasun sandhyasun commented May 13, 2026

Summary

Fixes #13

Adds a dedicated `dremio space` command group (list / get / create / delete) and clarifies top-level vs nested catalog semantics in `dremio folder`.

New `dremio space` commands

  • `create` — runs `CREATE SPACE` SQL. On pre-Space-Plugin clusters that reject it with the legacy sentinel ("Legacy spaces are not supported"), falls back to `CREATE FOLDER` with a single-component path. All other failures propagate directly; "already exists" is rewritten from the internal `Folder [[source, name]]` form to `Space [name] already exists`.
  • `get` / `delete` — reject multi-component paths with a redirect to `dremio folder`.
  • `delete` — resolves the entity first and enforces `containerType == SPACE` to prevent accidentally deleting sources or other top-level catalog entities.

`dremio folder` behavior changes

  • `create ` — prints a deprecation warning and runs `CREATE FOLDER`. On pre-SP clusters this still succeeds (server allows it). On SP-enabled clusters the server rejects it — matching Dremio SQL and v0 REST semantics, so CLI behavior is consistent with those surfaces.
  • `get ` / `delete ` — now reject top-level paths and redirect to `dremio space get` / `dremio space delete`.
  • `create` — propagates SQL job failures as `Error:` instead of printing the raw FAILED JSON.

Net effect

Cluster type `space create` `folder create` (single) nested folders
pre-SP CREATE SPACE → fallback to CREATE FOLDER warning + succeeds unchanged
SP-enabled CREATE SPACE (direct) warning + server error unchanged
  • pre-SP clusters keep working with no change in behavior.
  • SP-enabled clusters enforce `dremio space` for top-level objects, consistent with SQL/REST.
  • All nested folder operations are unchanged on both cluster types.

Test plan

  • Unit tests: all 182 pass (space + folder command coverage)
  • Pre-SP cluster (3165): space create/list/get/delete, fallback, error messages, folder guards
  • SP-enabled cluster (3164): `CREATE SPACE` succeeds directly, `folder create` single-component fails server-side as expected

…n `folder create`

Spaces and folders are distinct catalog concepts but the CLI blurred
them: `dremio folder create "foo"` silently rewrote to CREATE SPACE,
which breaks on projects without Space Plugin and hides the semantic
difference.

Changes:
- `dremio folder create` now rejects single-component paths with a
  clear error pointing users to `dremio space create <name>` instead.
- New `dremio space` command group with `list`, `get`, `create`, and
  `delete` subcommands. `space list` filters the catalog to SPACE
  entities; the others delegate to the existing folder helpers.

Closes dremio#13

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented May 13, 2026

CLA assistant check
All committers have signed the CLA.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d2f3f18572

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/drs/commands/space.py
Comment thread src/drs/commands/space.py Outdated
sandhyasun and others added 4 commits May 13, 2026 16:47
On DCS clusters without Space Plugin enabled,
DCSCoordinatorCatalogServiceImpl.createSpace() rejects CREATE SPACE SQL
with "Legacy spaces are not supported." In that case, fall back to
CREATE FOLDER with a single-component path, which works on those clusters
because the single-component validation was added alongside Space Plugin.

All other failures (entity already exists, permissions, etc.) are
propagated immediately without triggering the fallback.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- folder get/delete reject single-component paths (redirect to space get/delete)
- space get/delete reject multi-component paths (redirect to folder get/delete)
- create_folder propagates SQL job failures as DremioAPIError
- create_space fallback (pre-SP clusters): check fallback result and raise on
  failure; rewrite opaque "Folder [[source, name]] already exists" to
  "Space [name] already exists"

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Prevents dremio space delete from deleting non-space entities (sources,
home folders) that happen to share a single-component name.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
@sandhyasun
Copy link
Copy Markdown
Author

sandhyasun commented May 14, 2026

Still in Draft . Need to look into maintaining compatibility pre/post Space Plugin once more.

- folder create: replace ValueError with deprecation warning + server-driven
  behavior (pre-SP succeeds, SP fails); update help text to match
- folder get/delete: keep single-component redirect; update help text
- folder group: update module docstring and app help to reflect dual purpose
  (nested folders + top-level catalog listing)
- output.py: add warn() for stderr deprecation warnings
- introspect.py: add space.list/get/create/delete entries; fix stale
  folder.create description and sql_template
- tests: add deprecation warning, pre-SP success, and SP failure tests for
  folder create single-component path

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- Add dremio space to command summary table
- Split CRUD table: Spaces row -> space.*, Folders row -> nested-only
- Fix folder create description: uses CREATE FOLDER; single-component paths
  deprecated and may fail on Space-Plugin-enabled clusters
- Add space.py to file tree

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
async def create_folder(client: DremioClient, path: str) -> dict:
"""Create a space (single component) or folder (nested path) using SQL."""
"""Create a folder at the given path using SQL."""
from drs.utils import DremioAPIError
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets move imports to top level

sql = f"CREATE FOLDER {quoted}"
return await run_query(client, sql)
warn(
f"Top-level folder creation is deprecated. "
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. lets not talk about "Space-Pluging-enabled" - its DC and there are no clusters
  2. more importantly we should just fail this operation right away, no ?

Nested paths (e.g. 'Analytics.reports') create a folder inside a space.
Single-component paths attempt top-level creation for compatibility with
pre-Space-Plugin clusters; this is deprecated — use `dremio space create`
instead. On Space-Plugin-enabled clusters, single-component paths will
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets change the comments here - this is DC so there are no clusters or configs

Comment thread src/drs/commands/space.py
async def create_space(client: DremioClient, name: str) -> dict:
"""Create a space.

Uses CREATE SPACE SQL. On pre-Space-Plugin clusters that reject it with
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Comment thread src/drs/commands/space.py
async def get_space(client: DremioClient, name: str) -> dict:
"""Get space metadata by name."""
if len(parse_path(name)) > 1:
raise ValueError(f"'{name}' is a nested path. Use `dremio folder get {name}` for folders.")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of all these ValueError change to an error that extends ValueError - NestedPathUnsupported. That way the function doesn't always assume the error is to be reported to the user

Comment thread src/drs/commands/space.py
Comment on lines +101 to +110
except Exception as exc:
from drs.utils import DremioAPIError

if isinstance(exc, DremioAPIError):
error(str(exc))
raise typer.Exit(1)
if isinstance(exc, ValueError):
error(str(exc))
raise typer.Exit(1)
raise
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be replaced by

except:
    error(...)
    raise typer.Exit(1)

Comment thread src/drs/commands/space.py
try:
return await coro
finally:
await client.close()
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't the caller worry about client.close() ... the caller has created the client, we shouldn't close it here

Comment thread src/drs/commands/space.py
Comment on lines +86 to +89
def _get_client() -> DremioClient:
from drs.cli import get_client

return get_client()
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the wrapper doesn't really help. Do

from contextlib import asynccontextmanager

@asynccontextmanager
async def managed_client():
    client = await get_client()
    try:
        yield client
    finally:
        await client.close()

then _run_command can be removed. See below

Comment thread src/drs/commands/space.py
) -> None:
"""Get metadata for a space by name."""
client = _get_client()
_run_command(get_space(client, name), client, fmt, fields=fields)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change this and otehrs below to

async with _get_client() as client:
    result = asyncio.run(client, name)
    output(result, fmt, fields=fields)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Add 'dremio space' commands and reject single-component paths in 'folder create'

3 participants