Skip to content

Commit 80cf21c

Browse files
authored
chore(test): Clean up testing guide and Makefile testing dependency checks (#9611)
## Summary - **Rename `make deps` → `make check-deps`** with consistent `check-deps-*` prefixes across all 7 check scripts and Makefile targets - **Add Makefile-level OS-dispatched `install-deps-*` targets** for darwin (Homebrew-based) and linux (apt/dnf/pacman detection), replacing script-internal install logic - **Add Docker memory check** (`check-docker-available-memory`) — warns if < 8GB, auto-fixes Docker Desktop settings on macOS when `AUTO_INSTALL=true` - **Wire `check-deps` as prerequisite** for `test` and `test-benchmark` targets so dependency validation runs for all test paths - **Linux Go install prints instructions** instead of auto-installing (avoids destructive `sudo rm -rf /usr/local/go`) - **Update TESTING.md** — replace `make deps` references, expand validation list, strengthen setup messaging ## Test plan - [x] `make check-deps` passes on macOS (local) - [x] `make check-deps` passes on Linux (system-77.local) - [x] `make setup` passes on macOS (local) - [x] `make setup` passes on Linux (system-77.local) - [x] `make check-deps AUTO_INSTALL=true` passes on both platforms - [x] `make help` shows `check-deps` (not old `deps`) - [x] No stale references to old script names (`check-go.sh`, etc.) - [x] Pre-commit hooks pass (trunk formatter auto-fixed one file)
1 parent a301817 commit 80cf21c

12 files changed

Lines changed: 418 additions & 98 deletions

Makefile

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -84,16 +84,16 @@ uninstall: ## Uninstall dgraph binary
8484
dgraph-installed:
8585
$(MAKE) install
8686

87-
.PHONY: deps
88-
deps: ## Check test dependencies (pass AUTO_INSTALL=true to auto-install missing ones)
89-
$(MAKE) -C t deps
87+
.PHONY: check-deps
88+
check-deps: ## Check test dependencies (pass AUTO_INSTALL=true to auto-install missing ones)
89+
$(MAKE) -C t check-deps
9090

9191
.PHONY: setup
9292
setup: ## Install all test dependencies automatically
93-
$(MAKE) deps AUTO_INSTALL=true
93+
$(MAKE) check-deps AUTO_INSTALL=true
9494

9595
.PHONY: test
96-
test: dgraph-installed local-image ## Run tests (default: integration + integration2)
96+
test: check-deps dgraph-installed local-image ## Run tests (default: integration + integration2)
9797
ifdef TAGS
9898
@echo "Running tests with tags: $(TAGS)"
9999
go test -v --tags="$(TAGS)" \
@@ -177,7 +177,7 @@ test-all: ## Every test: all t/ suites + integration2 + upgrade + fuzz
177177
$(MAKE) test-fuzz
178178

179179
.PHONY: test-benchmark
180-
test-benchmark: ## Go benchmarks (i.e. 'go test -bench')
180+
test-benchmark: check-deps ## Go benchmarks (i.e. 'go test -bench')
181181
go test -bench=. -benchmem $(if $(PKG),./$(PKG)/...,./...)
182182

183183
.PHONY: local-image

TESTING.md

Lines changed: 136 additions & 64 deletions
Original file line numberDiff line numberDiff line change
@@ -142,24 +142,28 @@ auto-install them:
142142
make setup
143143

144144
# Check dependencies without installing (reports what's missing)
145-
make deps
145+
make check-deps
146146

147-
# Same as 'make deps' but auto-installs anything missing
148-
make deps AUTO_INSTALL=true
147+
# Same as 'make check-deps' but auto-installs anything missing
148+
make check-deps AUTO_INSTALL=true
149149
```
150150

151151
The check scripts validate:
152152

153-
- Go version (1.21+)
154-
- Docker and Docker Compose versions and memory allocation
153+
- Go version (matches go.mod requirement)
154+
- Docker and Docker Compose versions
155+
- Docker available memory (warns if < 8GB, with auto-fix on macOS)
155156
- gotestsum installation
156157
- ack installation
158+
- Cross-compiler for non-Linux hosts (macOS)
159+
- protoc installation (Linux only)
157160
- Dgraph binary existence and correct architecture
158161

159162
### Required Tools
160163

161-
> **Note:** You don't need to install these manually. Running `make setup` from the repo root
162-
> automatically installs missing dependencies. The commands below are listed for reference.
164+
> **Note:** You do **not** need to install these manually. Running `make setup` or
165+
> `make check-deps AUTO_INSTALL=true` from the repo root automatically checks and installs all
166+
> missing dependencies. The commands below are listed only as reference for what gets installed.
163167
164168
#### 1. Go (1.21+)
165169

@@ -273,6 +277,9 @@ Use `go test` to run one easy test on types package:
273277

274278
```bash
275279
go test -v ./types/... -run TestConvert
280+
281+
# Or with make (runs all unit tests, not just one)
282+
make test-unit
276283
```
277284

278285
**Expected output:**
@@ -294,6 +301,9 @@ ok github.com/dgraph-io/dgraph/v25/types (cached)
294301
295302
```bash
296303
cd t && go build . && ./t --test=TestGQLSchema
304+
305+
# Or with make
306+
make test TEST=TestGQLSchema
297307
```
298308

299309
If both pass, you're ready to run all test types!
@@ -304,28 +314,27 @@ If both pass, you're ready to run all test types!
304314

305315
### Using Make Targets
306316

307-
The simplest way to run tests:
308-
309-
```bash
310-
# Run default tests (~30 min): integration suite + integration2
311-
make test
312-
313-
# Run every test in the repo (all suites + all tag-based tests + fuzz)
314-
make test-all
315-
316-
# Common shortcuts (run 'make help' for full list)
317-
make test-unit # True unit tests only — no Docker, no build tags
318-
make test-integration # Integration tests via t/ runner with Docker (SUITE=integration)
319-
make test-integration-heavy # All heavy tests: systest-heavy + ldbc + load
320-
make test-core # Core tests (i.e. 'make test SUITE=core')
321-
make test-systest # All systest packages: systest-baseline + systest-heavy
322-
make test-vector # Vector search tests (i.e. 'make test SUITE=vector')
323-
make test-integration2 # Integration2 tests via dgraphtest (i.e. 'make test TAGS=integration2')
324-
make test-upgrade # Upgrade tests (i.e. 'make test TAGS=upgrade')
325-
make test-fuzz # Fuzz tests (i.e. 'make test FUZZ=1')
326-
make test-all # Every test: all t/ suites + integration2 + upgrade + fuzz
327-
make test-benchmark # Go benchmarks (i.e. 'go test -bench')
328-
```
317+
The simplest way to run tests is `make test` (default: `integration` suite + `integration2`). Each
318+
`test-*` target is a shortcut for `make test` with specific arguments. The table below shows all
319+
three ways to run each test type.
320+
321+
| Target | `make test` equivalent | Without make |
322+
| ----------------------------- | ----------------------------------------- | ----------------------------------------------------------------------------- |
323+
| `make test` | _(default)_ | `cd t && ./t --suite=integration` then `go test -v --tags=integration2 ./...` |
324+
| `make test-unit` | `make test SUITE=unit` | `cd t && ./t --suite=unit` |
325+
| `make test-integration` | `make test SUITE=integration` | `cd t && ./t --suite=integration` |
326+
| `make test-core` | `make test SUITE=core` | `cd t && ./t --suite=core` |
327+
| `make test-systest` | `make test SUITE=systest` | `cd t && ./t --suite=systest` |
328+
| `make test-vector` | `make test SUITE=vector` | `cd t && ./t --suite=vector` |
329+
| `make test-integration-heavy` | `make test SUITE=systest-heavy,ldbc,load` | `cd t && ./t --suite=systest-heavy,ldbc,load` |
330+
| `make test-integration2` | `make test TAGS=integration2` | `go test -v --tags=integration2 ./...` |
331+
| `make test-upgrade` | `make test TAGS=upgrade` | `go test -v --tags=upgrade ./...` |
332+
| `make test-fuzz` | `make test FUZZ=1` | `go test -v -fuzz=Fuzz -fuzztime=300s ./dql/...` |
333+
| `make test-benchmark` | _(no equivalent)_ | `go test -bench=. -benchmem ./...` |
334+
| `make test-all` | _(no equivalent)_ | Runs `SUITE=all` + `integration2` + `upgrade` + fuzz sequentially |
335+
336+
> **Tip:** All targets accept `PKG=`, `TEST=`, and `TIMEOUT=` variables. For example:
337+
> `make test-systest PKG=systest/plugin TEST=TestPasswordReturn TIMEOUT=60m`
329338
330339
Run `make help` to see all available targets, variables, and dynamically discovered SUITE/TAGS
331340
values.
@@ -501,6 +510,19 @@ go test ./types/...
501510
go test -v ./types/... -run TestConvert
502511
```
503512

513+
**With make:**
514+
515+
```bash
516+
# Run all unit tests (no Docker, no build tags)
517+
make test-unit
518+
519+
# Run unit tests for a specific package
520+
make test-unit PKG=types
521+
522+
# Run a specific unit test
523+
make test-unit PKG=types TEST=TestConvert
524+
```
525+
504526
### Identifying a unit test
505527

506528
- No `//go:build` tag at the top of the file = unit test
@@ -586,6 +608,19 @@ cd t && go build .
586608
./t -r
587609
```
588610

611+
**With make:**
612+
613+
```bash
614+
# Run a suite
615+
make test SUITE=core
616+
617+
# Run specific package
618+
make test SUITE=integration PKG=systest/export
619+
620+
# Run single test
621+
make test TEST=TestExportAndLoadJson
622+
```
623+
589624
### Key Flags
590625

591626
| Flag | Description |
@@ -621,6 +656,16 @@ cd t && go build .
621656
./t --pkg=systest/export --keep
622657
```
623658

659+
**With make:**
660+
661+
```bash
662+
# Run all tests in a package (make builds the runner automatically)
663+
make test SUITE=integration PKG=systest/export
664+
665+
# Run single test
666+
make test TEST=TestExportAndLoadJson
667+
```
668+
624669
### Method 2: Manual Cluster + go test
625670

626671
For fine-grained control, manually start a cluster and run tests against it.
@@ -693,6 +738,19 @@ Using `t/` runner:
693738
./t --suite=systest
694739
```
695740

741+
**With make:**
742+
743+
```bash
744+
# Run all systest packages
745+
make test-systest
746+
747+
# Run specific systest package
748+
make test SUITE=systest PKG=systest/export
749+
750+
# Run specific test by name
751+
make test TEST=TestExportAndLoadJson
752+
```
753+
696754
### Key Environment Variables
697755

698756
| Variable | Purpose | Set by |
@@ -738,6 +796,19 @@ go test -v --tags=integration2 ./systest/integration2/
738796
go test -v --tags=integration2 --run '^TestName$' ./pkg/
739797
```
740798

799+
**With make:**
800+
801+
```bash
802+
# Run all integration2 tests
803+
make test-integration2
804+
805+
# Run integration2 tests for a specific package
806+
make test TAGS=integration2 PKG=systest/vector
807+
808+
# Run a specific integration2 test
809+
make test TAGS=integration2 PKG=systest/vector TEST=TestVectorSearch
810+
```
811+
741812
### Version & Binary Management
742813

743814
**Automatic version handling:**
@@ -948,6 +1019,19 @@ go test -v --tags=upgrade ./worker/
9481019
go test -v --tags=upgrade -run '^TestUpgradeName$' ./pkg/
9491020
```
9501021

1022+
**With make:**
1023+
1024+
```bash
1025+
# Run all upgrade tests
1026+
make test-upgrade
1027+
1028+
# Run upgrade tests for a specific package
1029+
make test TAGS=upgrade PKG=acl
1030+
1031+
# Run a specific upgrade test
1032+
make test TAGS=upgrade PKG=acl TEST=TestACL
1033+
```
1034+
9511035
### Where Upgrade Tests Live
9521036

9531037
| Package | Tests |
@@ -1222,6 +1306,19 @@ go test -v --tags=integration --run 'TestPluginTestSuite/TestPasswordReturn/subt
12221306
go test -v --tags=upgrade --run 'TestPluginTestSuite/TestPasswordReturn' ./systest/plugin/
12231307
```
12241308

1309+
**With make:**
1310+
1311+
```bash
1312+
# Run the plugin systest package via t/ runner
1313+
make test SUITE=systest PKG=systest/plugin
1314+
1315+
# Run a specific test
1316+
make test SUITE=systest PKG=systest/plugin TEST=TestPluginTestSuite/TestPasswordReturn
1317+
1318+
# Run in upgrade mode
1319+
make test TAGS=upgrade PKG=systest/plugin TEST=TestPluginTestSuite/TestPasswordReturn
1320+
```
1321+
12251322
**When NOT to use:**
12261323

12271324
- Simple one-off tests → use regular `func TestX(t *testing.T)`
@@ -1258,6 +1355,16 @@ go test -v ./dql -fuzz=Fuzz -fuzztime=5m
12581355
go test -v ./dql -fuzz=Fuzz -fuzztime=300s -fuzzminimizetime=120s
12591356
```
12601357

1358+
**With make:**
1359+
1360+
```bash
1361+
# Run all fuzz tests (default 300s per package)
1362+
make test-fuzz
1363+
1364+
# Fuzz a specific package with custom duration
1365+
make test FUZZ=1 PKG=dql FUZZTIME=5m
1366+
```
1367+
12611368
### CI Workflow
12621369

12631370
- `ci-dgraph-fuzz.yml` (runs on PRs)
@@ -1276,41 +1383,6 @@ go test -v ./dql -fuzz=Fuzz -fuzztime=300s -fuzzminimizetime=120s
12761383

12771384
## Future Improvement Ideas
12781385

1279-
### ✅ Completed Improvements
1280-
1281-
The following items from the original wishlist have been implemented:
1282-
1283-
- **✅ OS detection and automatic binary handling:** The Makefile now detects the host OS at runtime
1284-
and automatically builds the correct binaries. On macOS, `make install` builds both native and
1285-
Linux binaries without manual intervention.
1286-
1287-
- **✅ Automatic binary path management:** The `LINUX_GOBIN` environment variable is automatically
1288-
set based on OS. Docker Compose files use `${LINUX_GOBIN:-$GOPATH/bin}` to mount the correct
1289-
binary.
1290-
1291-
- **✅ No manual setup scripts required:** The `make test` target now depends on `dgraph-installed`
1292-
which automatically builds binaries if missing. Dependency checking scripts in `t/scripts/` can
1293-
auto-install missing tools with `AUTO_INSTALL=true`.
1294-
1295-
- **✅ Prerequisites handled automatically:** Running `make test` validates dependencies and builds
1296-
required binaries before running tests.
1297-
1298-
- **✅ Unified test interface:** A single `make test` entry point that accepts arguments to run any
1299-
test type (unit, integration, integration2, upgrade, fuzz) with environment variables for control.
1300-
The default (`make test` with no args) runs `integration` suite plus `integration2` for a fast
1301-
feedback loop (~30 min). Use `make test-all` to run every test.
1302-
1303-
- **✅ Example commands that "just work":** The following now work as expected:
1304-
1305-
```bash
1306-
make test SUITE=systest
1307-
make test FUZZ=1 PKG=dql
1308-
make test TAGS=upgrade PKG=acl
1309-
make test SUITE=systest PKG=systest/plugin
1310-
```
1311-
1312-
### Remaining Ideas
1313-
13141386
The following improvements could still enhance the developer experience:
13151387

13161388
- **Extend t/ runner:** Have the `t/` runner also handle unit and integration2 tests, providing a

0 commit comments

Comments
 (0)