Skip to content

development.md

Latest commit

 

History

History
508 lines (395 loc) · 22.3 KB

development.md

File metadata and controls

508 lines (395 loc) · 22.3 KB
Raw

Development Guide

Diátaxis type: How-to Audience: 🔧 Developers, contributors Prerequisites: Go 1.26+, Node.js 22+, GitLab instance with PAT, Git, Make

Prerequisites

  • Go 1.26+ (download)
  • Node.js 22+ with Corepack for the documentation site and MCP Inspector. The site uses [email protected]; keep pnpm configuration in site/pnpm-workspace.yaml rather than the pnpm field in package.json.
  • GitLab instance with Personal Access Token (api scope)
  • Git for version control
  • Make for build automation (optional but recommended)

Project Structure

See cmd-utilities.md for the full CLI reference of every cmd/ binary (flags, usage, Make targets).

gitlab-mcp-server/
├── cmd/
│   ├── server/                  # MCP server entry point
│   │   ├── main.go              # Signal handling, transport selection
│   │   └── main_test.go         # Server startup and HTTP handler tests
│   ├── audit_1to1/              # Consolidated 1:1 SDK↔API parity audit (-scope structs|actions|metadata)
│   ├── audit_catalog_first/ # ActionSpec catalog coverage inventory
│   ├── audit_discovery_completeness/ # Discovery metadata audit with cluster-aware severity (META-001)
│   ├── audit_doc_coverage/      # docs/reference/tools/*.md vs catalog coverage gaps (DOC-002)
│   ├── audit_dynamic_aliases/   # Dynamic alias collision governance
│   ├── audit_edition_tier/      # Doc-grounded Free/Premium/Ultimate tier audit
│   ├── audit_metrics/           # MCP tool/resource/prompt metrics summary
│   ├── audit_surface_quality/   # Surface quality audit (-view metadata|output|all)
│   ├── audit_test_names/        # Test function naming convention compliance
│   ├── audit_tokens/            # Token overhead audit (+ --compare-schemas sizing spike)
│   ├── eval_mcp_surfaces/       # Model-facing MCP surface evaluation harness
│   ├── audit_string_dupes/              # Finds duplicated string literals missing constants
│   ├── format_md_tables/        # Normalizes Markdown pipe tables
│   ├── gen_action_catalog_manifest/ # Generates ActionSpec manifest
│   ├── gen_docker_tools/        # Generates Docker MCP Registry tools.json
│   ├── gen_llms/                # Generates llms.txt and llms-full.txt
│   ├── gen_stats/               # Regenerates README stats section
│   ├── gen_testing_docs/        # Regenerates testing.md managed sections
│   └── godoc_tool/              # Go doc auditor + fixer (audit/fix subcommands)
├── internal/
│   ├── config/                  # Environment variable loading and validation
│   ├── gitlab/                  # GitLab API client wrapper with TLS support
│   ├── completions/             # Autocomplete handler for 17 argument types
│   ├── progress/                # Progress notification tracker
│   ├── elicitation/             # Interactive user input client
│   ├── toolutil/                # Shared tool utilities (errors, pagination, markdown, logging)
│   ├── testutil/                # Shared test helpers (NewTestClient, RespondJSON)
│   ├── tools/                   # Tool orchestration layer + ~175 internal/tools packages (166 with action_specs.go)
│   │   ├── register.go          # RegisterAll() — catalog-backed individual tool projection
│   │   ├── register_meta.go     # RegisterAllMeta() — catalog-backed meta-tool groups and standalone surfaces
│   │   ├── metatool.go          # Local helpers addMetaTool/addReadOnlyMetaTool wrapping toolutil.DeriveAnnotations + route wrappers
│   │   ├── markdown.go          # markdownForResult delegator to toolutil.MarkdownForResult
│   │   ├── branches/            # Branch management tools (example sub-package)
│   │   ├── issues/              # Issue CRUD tools
│   │   ├── mergerequests/       # MR lifecycle tools
│   │   └── ...                  # ~175 internal/tools packages total
│   ├── resources/               # 45 MCP resource handlers
│   └── prompts/                 # 37 MCP prompt handlers
├── test/e2e/                    # End-to-end integration tests (suite/ + infra)
├── docs/                        # Documentation (this directory)
├── plan/                        # Implementation plans
├── VERSION                      # Single source of truth for project version
├── Makefile                     # Build automation
└── .env                         # Local secrets (gitignored)

Meta-tool counts are additive: 32 base tools, 16 Enterprise/Premium-specific meta-tools for 48 on self-managed GitLab, plus the GitLab.com-only Orbit meta-tool for 49 when Orbit is available.

Architecture

graph TD
    MAIN[cmd/server/main.go] -->|loads| CFG[config.Load]
    MAIN -->|creates| GL[gitlab.NewClient]
    MAIN -->|creates| SRV[mcp.NewServer]
    MAIN -->|selects surface| SURFACE{TOOL_SURFACE}
    SPECS[CollectActionSpecs<br/>domain ActionSpecs] --> CATALOG[BuildActionCatalog]
    MAIN -->|builds| CATALOG
    CATALOG --> IND[individual projection<br/>tools.RegisterAll]
    CATALOG --> META[meta projection<br/>tools.RegisterAllMeta]
    CATALOG --> DYN[dynamic projection<br/>dynamic.RegisterCatalogFindExecuteTools]
    STANDALONE[StandaloneSurfaceToolSpecs<br/>project discovery + interactive flows] -.->|dynamic route injection| DYN
    SURFACE -->|individual| IND
    SURFACE -->|meta| META
    SURFACE -->|dynamic| DYN
    IND --> PROJECTION[Catalog-backed ActionRoute handlers]
    META --> PROJECTION
    DYN --> PROJECTION
    MAIN -->|registers standalone| STANDALONE
    MAIN -->|registers| RES[resources.Register]
    MAIN -->|registers| PROMPTS[prompts.Register]
    MAIN -->|setup| COMP[completions]
    SRV -->|runs| STDIO[StdioTransport]
    SRV -->|runs| HTTP[StreamableHTTPHandler]
    PROJECTION --> GL
    PROJECTION --> PROG[progress]
    STANDALONE --> ELIC[elicitation]
    ELIC --> GL
    RES --> GL
    PROMPTS --> GL
Loading
  1. Config loads settings from .env + environment variables
  2. GitLab Client wraps the official gitlab.com/gitlab-org/api/client-go/v2
  3. Tools are projected from domain-local ActionSpecs through the canonical action catalog
  4. Meta-tools group catalog actions into 32 base tools (48 on self-managed Enterprise/Premium, 49 on GitLab.com Enterprise/Premium with Orbit) (via ADR-0005)
  5. Resources register read-only data via AddResource() / AddResourceTemplate()
  6. Prompts register AI-optimized interactions via AddPrompt()
  7. Capabilities provide completions, progress, and elicitation
  8. Server runs over stdio (default) or HTTP (--http)

See Architecture Overview for detailed diagrams and component descriptions.

Version Management

The project version is defined in the VERSION file at the repository root.

VERSION           # Contains e.g. "1.1.7" — no "v" prefix, no trailing newline
  ├─ Makefile     # Reads VERSION → passes via -ldflags to go build
  ├─ .gitlab-ci   # Reads VERSION → prefers CI_COMMIT_TAG if set
  └─ binary       # Receives version at build time via -X main.version

make version                          # Print version from VERSION file
./dist/gitlab-mcp-server.exe --version    # Print from compiled binary

Building

Local build

make build
# Output: dist/gitlab-mcp-server.exe (Windows) or dist/gitlab-mcp-server (Linux)

Manual build:

go build -ldflags="-X main.version=$(cat VERSION) -X main.commit=$(git rev-parse --short HEAD)" -o dist/gitlab-mcp-server ./cmd/server

Cross-compilation

make build-all
# Produces: linux-amd64, linux-arm64, windows-amd64, windows-arm64

Docker

Build image from source

make docker-build

# Or with explicit version
docker build \
  --build-arg VERSION=$(cat VERSION) \
  --build-arg COMMIT=$(git rev-parse --short HEAD) \
  -t gitlab-mcp-server .

Run locally

make docker-run
# or, for self-managed GitLab:
make docker-run GITLAB_URL=https://gitlab.example.com

Development with live builds

Use the build override to compile from source inside Docker Compose instead of pulling the pre-built image:

docker compose -f docker-compose.yml -f docker-compose.build.yml up -d

Publish to Container Registry

Publish via Makefile or manually:

# Via Makefile
make docker-push

# Or manually
docker login ghcr.io -u "$GITHUB_USER" --password-stdin <<< "$GITHUB_TOKEN"
docker push ghcr.io/jmrplens/gitlab-mcp-server:1.7.1
docker push ghcr.io/jmrplens/gitlab-mcp-server:latest

Testing

Unit Tests

Unit tests live alongside the code in each sub-package. They use net/http/httptest to simulate GitLab API responses — no real GitLab instance needed.

make test            # Standard tests with coverage
make test-race       # Tests with race detector
go test ./internal/... -count=1      # Run all unit tests (~124 packages)
go test ./internal/tools/branches/ -count=1 -v  # Run one domain verbose
go test ./internal/tools/ -run TestBranch -count=1    # Run specific tests

Test pattern (sub-package style)

Each sub-package has its own *_test.go with table-driven tests:

// internal/tools/branches/branches_test.go

func TestCreate_Success(t *testing.T) {
    client, mux := testutil.NewTestClient(t)
    mux.HandleFunc("/api/v4/projects/1/repository/branches", func(w http.ResponseWriter, r *http.Request) {
        testutil.RespondJSON(w, http.StatusOK, `{"name":"feature-x","commit":{"id":"abc123"}}`)
    })

    out, err := Create(context.Background(), client, CreateInput{
        ProjectID: "1",
        Branch:    "feature-x",
        Ref:       "main",
    })
    if err != nil {
        t.Fatalf("unexpected error: %v", err)
    }
    if out.Name != "feature-x" {
        t.Errorf("Name = %q, want %q", out.Name, "feature-x")
    }
}

Shared helpers (internal/testutil/)

Helper Purpose
testutil.NewTestClient(t) Creates mock GitLab client + httptest mux
testutil.RespondJSON(w, code, body) Writes JSON response with status code
testutil.RespondJSONWithPagination() Writes JSON response with pagination headers

End-to-End Tests

E2E tests run against a real GitLab instance via in-memory MCP transport (build tag e2e):

make test-e2e
# or: go test -v -tags e2e -timeout 300s ./test/e2e/suite/

# Compile-only check (no GitLab instance needed)
go test -tags e2e -c -o NUL ./test/e2e/suite/       # Windows
go test -tags e2e -c -o /dev/null ./test/e2e/suite/  # Linux

Docker Mode (Ephemeral GitLab)

Run the full E2E suite against an ephemeral GitLab CE container. Requires Docker and ~4 GB RAM. This mode also enables pipeline/job tests that need a CI runner.

make test-e2e-docker

This single command handles the full lifecycle: start GitLab CE container, wait for readiness, create test user/token, register CI runner, run tests, and tear down.

For manual step-by-step execution, see E2E Docker Mode in the testing guide.

E2E Prerequisites

GITLAB_URL=https://gitlab.example.com
GITLAB_TOKEN=glpat-your-token
GITLAB_SKIP_TLS_VERIFY=true

E2E Test Structure

File Description
test/e2e/suite/setup_test.go Shared state, MCP server setup, helpers, drainSidekiq
test/e2e/suite/fixture_ce_test.go Self-contained GitLab CE resource builders
test/e2e/suite/fixture_ee_test.go Self-contained GitLab EE resource builders
test/e2e/suite/*_test.go 137 domain-specific test files (individual + meta)

MCP Inspector

The MCP Inspector provides a web UI for interactively testing MCP tools, resources, and prompts against a running server.

make inspector       # Compile fresh binary to /tmp, launch Inspector via stdio
make inspector-stop  # Stop Inspector processes and clean up temp binary

This compiles the server to a temporary binary (/tmp/gitlab-mcp-server-inspector), reads credentials from .env, and launches the Inspector at http://127.0.0.1:6274/. The temporary binary is automatically cleaned up on exit.

Prerequisites: Node.js >= 22, .env file with GITLAB_TOKEN. Add GITLAB_URL for self-managed instances.

Linting & Formatting

make lint         # golangci-lint config, format diff, and run
make fmt          # apply configured Go formatters through golangci-lint
make analyze-fix  # apply supported Go and Markdown fixes

Error Handling in Tool Handlers

All error wrapping functions live in internal/toolutil/errors.go. Choose the right function based on this decision tree:

flowchart TD
    start{Is the operation read-only?}
    start -->|list / get / search| wrapErr[WrapErr]
    start -->|create / update / delete| hasHint{Known corrective action?}
    hasHint -->|No| wrapMsg[WrapErrWithMessage]
    hasHint -->|Yes| statusHint{Hint applies to one HTTP status?}
    statusHint -->|Yes| wrapStatus[WrapErrWithStatusHint]
    statusHint -->|No| checkStatus[Check IsHTTPStatus]
    checkStatus --> wrapHint[WrapErrWithHint]
Loading

Quick reference

Function When to use Includes GitLab detail Includes hint
WrapErr Read-only operations No No
WrapErrWithMessage Mutating operations (default) Yes No
WrapErrWithHint Specific error with known fix Yes Yes
WrapErrWithStatusHint Status-specific hint (combines IsHTTPStatus + WrapErrWithHint) Yes Yes (for matching status)

Pattern: Status-specific hints

if toolutil.IsHTTPStatus(err, 409) {
    return Output{}, toolutil.WrapErrWithHint("labelCreate", err,
        "label with this name already exists — use gitlab_label_update to modify it")
}
return Output{}, toolutil.WrapErrWithMessage("labelCreate", err)

Pattern: Single-status hint (shorthand)

// Equivalent to the above but in a single call — returns WrapErrWithMessage for non-409 errors
return Output{}, toolutil.WrapErrWithStatusHint("labelCreate", err, 409,
    "label with this name already exists — use gitlab_label_update to modify it")

Helpers

  • IsHTTPStatus(err, code) — checks if the error chain contains a gl.ErrorResponse with the given HTTP status
  • ContainsAny(err, substrs...) — checks if err.Error() contains any of the given substrings
  • ExtractGitLabMessage(err) — extracts the specific message from gl.ErrorResponse.Message

See Error Handling for the full architecture.

Adding a New Tool

With the catalog-first modular sub-package architecture:

  1. Create sub-package: internal/tools/{domain}/
  2. Create handler file: {domain}.go with typed input/output structs (no domain prefix — package provides namespace)
  3. Create test file: {domain}_test.go with table-driven tests using testutil.NewTestClient
  4. Create ActionSpecs: define ActionSpecs(client, ...) or update the owning aggregation builder with typed ActionRoute constructors and individual projection metadata
  5. Create markdown formatters: register output formatters from the sub-package with toolutil.RegisterMarkdown or toolutil.RegisterMarkdownResult
  6. Regenerate catalog manifest: run make gen-action-catalog-manifest when the source-defined builder set changes, then run make check-action-catalog-manifest
  7. Update documentation: docs/reference/tools/{domain}.md and docs/reference/tools/README.md

Meta-tools and the dynamic toolset share the canonical action catalog built by internal/tools/action_catalog.go. When adding a normal GitLab operation, define the route once inside the owning ActionSpec with typed ActionRoute constructors (RouteAction, DestructiveAction, RouteActionWithRequest, and void variants). The same catalog entry then powers the individual tool projection, visible meta-tool action, gitlab_find_action, gitlab_execute_action, the gitlab://tools manifest, generated LLM files, and audit commands. Do not create package-local RegisterTools functions or dynamic-only copies of ordinary GitLab actions.

See Tool Surfaces And Canonical Action Core for the ownership rules across individual tools, meta-tools, dynamic mode, and the canonical action catalog.

Orbit live-test fixtures

The orbitlive build-tagged live tests in test/e2e/orbit/live_test.go exercise the real https://gitlab.com/api/v4/orbit/* endpoints. They expect two projects in the configured namespace (kg-fixtures and security-fixtures) with a specific shape, plus optional mirror data. The reproduction script, the expected fixture layout, and the indexer caveat (transient error state) are documented in Orbit Live Test Fixtures.

To run the full flow against GitLab.com — token validation, idempotent fixture provisioning, indexer catch-up wait, then the four live test suites (41 subtests) — use the orchestrated target:

# Add a Personal Access Token (api scope) to .env first
echo 'GITLAB_COM_TOKEN=glpat-...' >> .env

# Provision fixtures in your own namespace and run the live tests
make test-e2e-gitlab-com ORBIT_FIXTURES_NAMESPACE=acme-research

# When fixtures are already provisioned, skip setup and run only the tests
GITLAB_COM_TOKEN=glpat-... \
  go test -tags orbitlive -count=1 -v -timeout 300s ./test/e2e/orbit/

make test-e2e-gitlab-com chains four sub-targets: orbit-ensure-token (validates GITLAB_COM_TOKEN is exported), orbit-setup-fixtures (runs scripts/setup-orbit-fixtures.sh), orbit-wait-indexer (polls /api/v4/orbit/graph_status until the indexer reports the projects as indexed), and orbit-run-live-tests (runs go test -tags orbitlive ...). Each sub-target is independently runnable.

Example: Adding a tools sub-package

// internal/tools/branches/branches.go

package branches

type CreateInput struct {
    ProjectID string `json:"project_id" jsonschema:"Project ID or URL-encoded path"`
    Branch    string `json:"branch"     jsonschema:"Branch name to create"`
    Ref       string `json:"ref"        jsonschema:"Source branch or commit SHA"`
}

type Output struct {
    Name   string `json:"name"`
    Commit string `json:"commit"`
    WebURL string `json:"web_url"`
}

func Create(ctx context.Context, client *gitlabclient.Client, input CreateInput) (Output, error) {
    if err := ctx.Err(); err != nil {
        return Output{}, err
    }
    // GitLab API call...
    return Output{}, nil
}
// internal/tools/branches/action_specs.go

package branches

func ActionSpecs(client *gitlabclient.Client) []toolutil.ActionSpec {
    route := toolutil.RouteAction(client, Create).
        WithUsage("Use to create a branch from an existing branch, tag, or commit SHA.")

    return []toolutil.ActionSpec{
        toolutil.NewActionSpec("create", route, toolutil.ActionSpecOptions{
            ReadOnly:     false,
            Idempotent:   false,
            OwnerPackage: "branches",
            IndividualTool: toolutil.IndividualToolSpec{
                Name:        "gitlab_create_branch",
                Title:       "Create branch",
                Description: "Create a new branch in a GitLab project.",
            },
        }),
    }
}

Environment Setup

Local development

  1. Clone the repository
  2. Create .env with your GitLab credentials
  3. Run go mod download
  4. Build: make build
  5. Run: dist/gitlab-mcp-server

IDE setup (VS Code)

Install the Go extension and add to .vscode/mcp.json:

{
  "servers": {
    "gitlab-dev": {
      "type": "stdio",
      "command": "${workspaceFolder}/dist/gitlab-mcp-server.exe",
      "env": {
        "GITLAB_URL": "https://your-gitlab",
        "GITLAB_TOKEN": "glpat-your-token",
        "GITLAB_SKIP_TLS_VERIFY": "true",
        "TOOL_SURFACE": "meta"
      }
    }
  }
}

Git Workflow

  • Conventional commits: feat:, fix:, docs:, test:, refactor:, chore:
  • Feature branches: feature/tool-name, fix/description
  • Main branch: Protected, merge via pull requests

Dependencies

Dependency Version Purpose
github.com/modelcontextprotocol/go-sdk v1.6.1 MCP server framework
gitlab.com/gitlab-org/api/client-go/v2 v2.46.0 Official GitLab REST API client
github.com/joho/godotenv v1.5.1 .env file loading for dev

External References

Resource URL
MCP Specification (2025-11-25) https://modelcontextprotocol.io/specification/2025-11-25/
MCP Go SDK (pkg.go.dev) https://pkg.go.dev/github.com/modelcontextprotocol/go-sdk
MCP Go SDK Repository https://github.com/modelcontextprotocol/go-sdk
GitLab REST API v4 https://docs.gitlab.com/ee/api/rest/
GitLab Go Client (pkg.go.dev) https://pkg.go.dev/gitlab.com/gitlab-org/api/client-go/v2
GitLab Go Client Repository https://gitlab.com/gitlab-org/api/client-go