[FEATURE]GitLab user/group/project provisioning in the DTaaS CLI

[prasadtalasila]

Describe the Feature

As a DTaaS administrator, I want the DTaaS CLI (cli/) to automatically
provision the GitLab users, groups, and project structure a new user needs —
including setting each user's initial GitLab password — using a provisioning PAT
supplied in dtaas.toml so that onboarding creates the required GitLab
resources under a chosen group without manual, error-prone setup (#1681).

Ex. As a DTaaS administrator, I want to run dtaas admin user add with a
password (via --password or the password column of users.csv) and have the
user's GitLab account created with that password, their group membership set,
and their project created so that their workspace is usable immediately.

Problem Statement

The DTaaS CLI has no GitLab API capability today — its dependencies are click,
tomlkit, python-on-whales, PyYAML, cryptography, and it references GitLab
only as OAuth URLs in dtaas.toml. Onboarding therefore requires GitLab users,
groups, and a project structure to be created by hand (#1681). This feature
adds automated, idempotent provisioning built on dtaas-gitlab-common (see
01-create-dtaas-gitlab-common.md) and on the three-file user model from
#1690 (starting users in dtaas.toml, additional users in
dtaas.users.registry.json, runtime facts in .dtaas.state.json).

Proposed Solution

1. Add the dependency

In cli/pyproject.toml:

[tool.poetry.dependencies]
python = "^3.10"
PyYAML = "^6.0.3"
click = "^8.4.2"
tomlkit = "^0.15.0"
python-on-whales = "^0.81.0"
cryptography = "^49.0.0"
dtaas-gitlab-common = { path = "../lib/dtaas-gitlab-common", develop = true }

(python-gitlab arrives transitively.)

2. Provisioning PAT and target group in dtaas.toml

A provisioning PAT is supplied directly in dtaas.toml and must be able to
create users, groups, and projects (i.e. an admin token, or a group-owner token
with api scope). This is consistent with the existing secret fields already in
dtaas.toml (oauth-client-secret, keycloak-admin-password). The target
group under which the project structure is created is also configured here.

# --- GitLab user/group/project provisioning for new users (issue #1681). -----
# When enabled, 'user add' / 'reconcile' ensure each user's GitLab user,
# group membership, and project exist idempotently under 'group', recording
# facts in .dtaas.state.json.
[gitlab]
provision = true
api_url = "https://gitlab.com"
# Provisioning PAT (api scope) able to create users, groups, and projects.
# Keep dtaas.toml out of version control when it carries a live token, or
# supply the value via the DTAAS_GITLAB_PAT environment variable instead.
pat = "glpat-xxxxxxxxxxxxxxxxxxxx"
# Chosen GitLab group under which the per-user project structure is created.
group = "dtaas"
ssl_verify = true
# Project path/name pattern; {username} is substituted.
project_name = "{username}"
visibility = "private"
# Role each user is granted on their project.
member_access = "maintainer"
# Optional skeleton project imported/forked into each new project.
# template_project = "dtaas/dt-library-template"

3. CLI surface and CSV: user password

Each provisioned user needs an initial GitLab password. It is supplied either
per-invocation via a flag or per-row via the CSV, extending the #1690 command
surface:

dtaas admin user add <username> --email <e> --groups <g,...> \
    --load-balance <bool> --password <p>
dtaas admin user add --file users.csv

The --password flag is optional; when omitted for a single-user add, the CLI
prompts interactively with input hidden (click.prompt(..., hide_input=True, confirmation_prompt=True)) rather than requiring the secret on the command
line, since a flag value is visible in shell history and the process list.

users.csv gains a password column (named to match the Services CLI's
existing credentials.csv), alongside the four #1690 identity fields:

username,email,groups,load_balance,password
user3,user3@example.com,additional;beta-testers,true,S3cur3-p4ss
user4,user4@example.com,,false,An0ther-p4ss

The password is a transient secret: it is validated, passed to
ensure_user to create the GitLab account, and then discarded. It is never
written to dtaas.users.registry.json, .dtaas.state.json, or logs — the
registry keeps only the four #1690 identity fields, and state keeps only the
GitLab resource facts below.

4. New provisioning module: cli/src/pkg/gitlab/

Greenfield — built directly on the shared package:

cli/src/pkg/gitlab/
├── __init__.py
├── client.py       # resolve_client(config) → get_gitlab_client(api_url, pat, ssl_verify)
└── provisioner.py  # ensure_user_resources() using common ensure_* helpers

• client.py resolves the base URL from [gitlab].api_url, reads the PAT from
  [gitlab].pat (or DTAAS_GITLAB_PAT), then calls
  dtaas_gitlab_common.get_gitlab_client(api_url, pat, ssl_verify). Because the
  PAT lives in config, read_pat_from_json is not used here (that helper
  remains for the Services CLI's token file).
• provisioner.py implements
  ensure_user_resources(gl, username, email, password, config) -> (created: bool, msg),
  which first validates the row with
  dtaas_gitlab_common.validate_username/validate_email/validate_password, then
  idempotently, in order:
  1. dtaas_gitlab_common.ensure_user(gl, {"username": username, "email": email, "password": password, "name": username, "skip_confirmation": True}) —
     create the GitLab user with the given password if missing (409 → already
     exists; password left unchanged on an existing account).
  2. dtaas_gitlab_common.ensure_group(gl, config["group"]) — ensure the chosen
     group exists.
  3. dtaas_gitlab_common.ensure_project(gl, name, namespace=group, ...) —
     ensure the project exists under the group (probe → create; >1 →
     GITLAB_DUPLICATE_PROJECT), then add the user as a member at
     member_access.
     Transient calls are wrapped in dtaas_gitlab_common.call_with_backoff. The
     password argument is used only for step 1 and is never logged or persisted.

5. Config accessors for [gitlab]

Extend cli/src/pkg/config.py (Config), mirroring the existing
get_tls/get_set_limits style:

def get_gitlab_section(self): ...
def get_gitlab_provision(self): ...      # bool, default False
def get_gitlab_api_url(self): ...
def get_gitlab_pat(self): ...            # [gitlab].pat, else DTAAS_GITLAB_PAT
def get_gitlab_group(self): ...
def get_gitlab_project_name(self): ...
def get_gitlab_visibility(self): ...
def get_gitlab_member_access(self): ...
def get_gitlab_ssl_verify(self): ...

6. Wire into the user lifecycle (cli/src/pkg/users.py)

Following the #1690 add algorithm, add a GitLab provisioning sub-step beside
container provisioning, gated by get_gitlab_provision():

• In the add path, after the registry write and beside start_user_containers,
  call ensure_user_resources per user when enabled. When disabled, behaviour
  is unchanged.
• Record facts (gitlab_user_id, group, project_id, path_with_namespace,
  created_at, config_hash) in .dtaas.state.json next to the container
  facts, so user list --drifted and user reconcile --repair cover GitLab
  resources the same way they cover containers.
• user reconcile --repair diffs state against a live GitLab probe: missing
  user/group/project → create; duplicate project → report, refuse to create a
  third.

7. New error codes (in the #1690 family)

• GITLAB_UNREACHABLE — API unreachable or PAT rejected.
• GITLAB_PROVISION_FAILED — a user/group/project op failed after retries.
• GITLAB_DUPLICATE_PROJECT — probe found >1 matching project.

GitLab failures are reported independently of container results so an outage in
one does not mask the other.

Alternatives Considered

• Read the PAT from a token file only. Rejected per the chosen design — the
  provisioning PAT is supplied in dtaas.toml (with an env-var override), so a
  single hand-authored config drives provisioning against any GitLab instance,
  bundled or external.
• Implement GitLab access inline in users.py. Rejected — duplicates the
  primitives now centralised in dtaas-gitlab-common.
• Depend on dtaas-services directly. Rejected — couples the app-layer CLI
  to infra release cadence and bundled-GitLab (env URL, root token file)
  assumptions; this feature must also drive external GitLab.
• Provision unconditionally. Rejected — must be feature-flagged
  (provision) so existing deployments and localhost installs are unaffected.

Additional Context

Depends on 01-create-dtaas-gitlab-common.md and follows
02-migrate-dtaas-services.md. Implements #1681 on top of #1690. The
[[users]] starting-user tables referenced by the lifecycle are defined in the
updated dtaas.toml schema from the #1690 design.

Security note: a live provisioning PAT in dtaas.toml should be treated like
the other secrets already in that file — keep the populated file out of version
control (or use the DTAAS_GITLAB_PAT override) and restrict its permissions.

Success Criterion

Describe the expected outcome, using a checklist where appropriate.

Checklist:

• cli/pyproject.toml depends on dtaas-gitlab-common via the
  ../lib/dtaas-gitlab-common path; poetry install/lock succeed.
• cli/src/pkg/gitlab/ uses dtaas_gitlab_common for the client,
  validation, retry, and ensure_user/ensure_group/ensure_project — no
  GitLab client, SSL, or idempotency code is reimplemented in the DTaaS CLI.
• [gitlab].pat (with DTAAS_GITLAB_PAT override) and [gitlab].group
  drive provisioning; Config exposes the [gitlab] accessors.
• With provision = true, dtaas admin user add idempotently ensures each
  user's GitLab user, group membership, and project under group, recording
  them in .dtaas.state.json; with the flag false, behaviour is unchanged.
• dtaas admin user add accepts --password; when omitted for a single-user
  add it prompts with hidden input; the GitLab user is created with that
  password.
• users.csv includes a password column and each row's password is used
  to create that user; a missing/invalid password fails validation before any
  write (USER_VALIDATION_FAILED).
• The password is never written to dtaas.users.registry.json,
  .dtaas.state.json, or logs.
• dtaas admin user reconcile --repair creates missing users/groups/
  projects and reports duplicate projects without creating a third.
• Error codes GITLAB_UNREACHABLE, GITLAB_PROVISION_FAILED,
  GITLAB_DUPLICATE_PROJECT are emitted with actionable messages.
• Test coverage is improved — tests for enabled/disabled flag, create vs.
  already-present vs. duplicate, unreachable/invalid PAT, and reconcile repair
  (GitLab client mocked).
• No qlty issues.