Skip to content

Migrate tests to testcontainers-go#107

Closed
meoyawn wants to merge 2 commits into
jschaf:mainfrom
meoyawn:meoyawn/testcontainers
Closed

Migrate tests to testcontainers-go#107
meoyawn wants to merge 2 commits into
jschaf:mainfrom
meoyawn:meoyawn/testcontainers

Conversation

@meoyawn

@meoyawn meoyawn commented May 5, 2026

Copy link
Copy Markdown

setting up a foundation for agentic solving of #99

prompt:

## Postgres Testing Flow Findings

### `internal/pgtest` Package Tests

- `internal/pgtest.NewPostgresSchemaString` connects to `localhost:5555`, creates a random schema, sets `search_path` to that schema, loads the test SQL, and cleanup drops the schema with `DROP SCHEMA ... CASCADE`.
- These tests do not use per-test transactions or rollback.
- Tests are isolated by schema, not by transaction.
- No repo tests currently call `t.Parallel()`, so tests within each package run sequentially.
- `go test ./...` does not pass an explicit `-p`, but Go still runs package test binaries concurrently by default. The default package parallelism is `GOMAXPROCS`.
- Because each test uses a random schema, package-level concurrency is already mostly safe for `pgtest` tests. Subtests often share one connection and one test dataset, so they are not automatically parallelizable without refactoring setup.
- Replacing this flow with `testcontainers-go` is possible by starting one Postgres container in `TestMain` for each package that needs Postgres, then passing that package container connection string into `internal/pgtest`.
- Risk: this repo has many Postgres-using packages, so one container per package can mean many concurrent containers under `go test ./...` unless package parallelism is limited or the helper reuses a shared container.

### Acceptance Tests

- `example/acceptance_test.go` starts one temporary Postgres container with `internal/pgdocker`, then each acceptance subtest creates a random database inside that container.
- These tests do not use per-test transactions or rollback.
- Acceptance isolation is by database, with final cleanup coming from removing the container. Individual databases are not dropped during the test.
- Acceptance subtests are not currently parallelized.
- They are not safely parallelizable as-is because they share one `*pgx.Conn`, run git diff checks, and some cases may write generated files in fixed example directories when `-update` is used.
- Replacing this flow with `testcontainers-go` is possible by starting one Postgres container in the `example` package `TestMain`, then keeping the existing per-subtest database creation or switching to per-subtest schemas.

### Production No-Connection Flow

- `pggen.Generate` uses `internal/pgdocker.Start` when `GenerateOptions.ConnString` is empty. This is production/library behavior, not just test setup.
- This flow builds a custom Docker image from `postgres:13`, copies schema init scripts into `/docker-entrypoint-initdb.d/`, starts a container on a dynamic host port, then removes it on cleanup.
- `TestMain` cannot replace this production path by itself. To remove custom Docker API usage fully, `internal/pgdocker` or the no-`--postgres-connection` path must be rewritten to use `testcontainers-go`.
- A `testcontainers-go` replacement must preserve current production behavior for schema init files, container logs on errors, dynamic port selection, tmpfs or equivalent fast storage, and cleanup.


each of these 3 flows must use `testcontainers-go`.
use `postgres:*-alpine` images. each flow uses it's own version I think. I saw both 13 and 15. Use corresponding version in corresponding flow.
`internal/pgdocker` must be gone (or must become a place of setting up `testcontainers-go`).
`TestMain` must be leveraged as much as possible, exactly one `postgres` container setup and teardown inside a package.

`docker-compose.yml` must be gone. reliance on docker-compose port `5555` must be replaced with `testcontainers-go` in corresponding `TestMain`s.

if any of the test flows run migrations, those migrations must be ran in corresponding `TestMain`s after the postgres container boot.
use `$HOME/workspace/recurring/apps/api/migrations/migration_test.go` as an example for: 1. booting postgres. 2. running migrations. 3. running tests. 4. tearing down postgres.
all 3 test flows must pass.

@jschaf jschaf left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the approach looks okay. The big increase in indirect dependencies isn't amazing.

What do you think of using this instead? https://michael.stapelberg.ch/posts/2024-11-19-testing-with-go-and-postgresql-ephemeral-dbs/

)

// Client is a client to control the running Postgres Docker container.
const DefaultImage = "postgres:13-alpine"

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why Postgres 13?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

internal/pgdocker was using 13

FROM postgres:13

you think we should unify everything to 15? I'm down

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's jump to 17 if it's not too painful. Otherwise, 14 or 15 is fine, as those are both still supported.

13 is end of life.

type Option func(config *pgx.ConnConfig)

const (
postgresImage = "postgres:15-alpine"

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably worth pulling to a test constant so we don't have different versions.

@meoyawn

meoyawn commented May 6, 2026

Copy link
Copy Markdown
Author

Thanks, the approach looks okay. The big increase in indirect dependencies isn't amazing.

What do you think of using this instead? https://michael.stapelberg.ch/posts/2024-11-19-testing-with-go-and-postgresql-ephemeral-dbs/

please take a look at #108

@meoyawn meoyawn closed this May 6, 2026
@meoyawn
meoyawn deleted the meoyawn/testcontainers branch May 6, 2026 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants