Skip to content

Async deploy pipeline + port recycling + central config#43

Open
atharv10032 wants to merge 1 commit into
mainfrom
feat/async-deploy-port-recycle
Open

Async deploy pipeline + port recycling + central config#43
atharv10032 wants to merge 1 commit into
mainfrom
feat/async-deploy-port-recycle

Conversation

@atharv10032

Copy link
Copy Markdown
Collaborator

Summary

Three related changes to make the deploy flow correct under real-world conditions:

  • Async deploy. POST /projects/submit returns 202 {deploymentID, status: "pending"} immediately; the build runs in a detached goroutine with a bounded worker pool (8 concurrent) and panic recovery. Clients poll GET /projects/:id (auth-scoped) for state. Running the build inline was timing out browsers / proxies on real repos (docker build + Azure NSG poll runs into minutes).
  • Port allocator overhaul.
    • Reuses db.GetCollection's pooled Mongo client (was opening + tearing down a fresh mongo.Connect/Disconnect per allocation).
    • Atomic claim via InsertOne against a unique index on ports.port (replaces the upsert-with-$setOnInsert that could silently let two concurrent claims both think they won).
    • Recycle pool: DELETE /projects/:containerName now actually frees the port via db.ReleasePort, and allocation tries the recycle pool (lowest available first) before the watermark scan.
    • Fixes the broken DeleteProjectByContainerName query: containernamecontainer_name to match the Project struct's bson tag (the old form never matched, silently orphaning project docs).
  • Central config. New internal/config package loads .env exactly once at startup via sync.Once. Removed the duplicate godotenv.Load calls from utils/jwt.go and utils/port_manager.go's init() (which ran before main.go's load and read empty env vars).

Breaking change for the client

  • Before: POST /projects/submit blocked for minutes, returned 200 {url}.
  • After: POST /projects/submit returns 202 {deploymentID, status: "pending"}. Client polls GET /projects/:id until status is succeeded (read hosted_url) or failed (read deploy_error).

autoship-client side is intentionally out of scope here; will follow in a separate PR.

Operator notes (one-time, before bringing this build up)

db.EnsurePortsIndex will fail at startup if the existing ports collection has duplicate port values (a remnant of the prior buggy upsert). Either drop the collection (loses existing reservations) or dedupe in place before deploying.

Test plan

  • go build ./... and go vet ./... green locally
  • Static deploy: submit, poll until succeeded, hit Blob URL
  • Dynamic deploy: submit, poll through deploying → succeeded, hit hosted URL
  • Concurrent submissions (≥9 at once): first 8 run, 9th queues — no 500s
  • DELETE deployment: verify ports doc flips to status: "available" and Project doc is gone
  • Second deploy reuses the freed port (recycle pool working)
  • AWS regression with CLOUD_PROVIDER=aws

Known follow-ups (FUTURE comments in code)

Pointers for the next pass:

  • services/dynamic.go near finalCmd.Run() — docker bind failure leaves a phantom port reservation; needs bound bool + defer db.ReleasePort.
  • services/dynamic.go line ~227 — cloud.Get().AuthorizePort(hostPort) redundant after GetOrReserveValidFreePort already authorized internally (Azure NSG poll wastes ~10-30s).
  • services/clone.go — clone path static/<user>/<repo> has no deploy id; concurrent same-repo deploys race on the same dir.
  • api/auth.go GitHubCallback — userInfo["login"].(string) panics if login is missing/non-string.
  • api/auth.go GitHubCallback — hardcoded http://localhost:3000/dashboard redirect; needs FRONTEND_URL env var for prod.
  • services/dynamic.go Go branch of GenerateDockerfileinstallCmd builds app then CMD runs whatever startCommand says; incoherent for Go projects.
  • services/dynamic.go containerNametime.Now().Unix() is seconds-resolution; same-repo deploys landing in same second collide.
  • api/auth.go Signup — no input validation; empty email/password create unusable user.

🤖 Generated with Claude Code

- Submit returns 202 with deploymentID; client polls GET /projects/:id.
  Bounded goroutine pool (8) with panic recovery so build/run can take
  minutes without HTTP timeouts.
- Port allocator: reuses pooled Mongo client (was open-per-call),
  atomic InsertOne with unique index (was racy upsert), recycle pool
  via db.ReleasePort on delete (was permanently watermarked).
- Fix DeleteProjectByContainerName field name: containername ->
  container_name (was silently no-op, leaving orphan project docs).
- New internal/config package loads .env once at startup via sync.Once.
  Removed duplicate godotenv.Load from utils/jwt.go and
  utils/port_manager.go's init().
- FUTURE comments added for follow-up items (docker bind cleanup,
  double AuthorizePort, clone-path race, GH callback nil panic,
  hardcoded localhost redirect, Go Dockerfile template, containerName
  collision, signup validation).

Co-Authored-By: Claude Opus 4.7 <[email protected]>
@atharv10032 atharv10032 requested a review from Ashmit-Kumar June 6, 2026 17:14

@Ashmit-Kumar Ashmit-Kumar left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Next: Fix the auth module

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants