Skip to content

Commit 974923a

Browse files
mlwellesmlwelles
authored
feat(test): simplify macOS testing with automatic cross-compilation (#9585)
## Summary This PR eliminates manual setup for running integration tests on macOS and introduces a unified test interface via Make. ### Key Features **1. Automatic Cross-Compilation (macOS)** Previously, macOS developers had to manually cross-compile a Linux binary and juggle environment variables. Now, `make install` handles everything automatically. **2. Unified Test Interface** A single `make test` entry point that routes to the appropriate test runner based on variables: ```bash # Run all tests (default) make test # Use variables to control what runs make test SUITE=systest # Run systest suite via t/ runner make test TAGS=integration2 # Run integration2 tests via go test make test FUZZ=1 PKG=dql # Fuzz test the dql package make test TAGS=upgrade PKG=acl # Run ACL upgrade tests ``` **3. Convenient Shortcuts** Every SUITE and TAGS value has a corresponding helper target: ```bash # SUITE-based targets make test-all # All test suites (i.e. 'make test SUITE=all') make test-unit # Unit tests, no Docker (i.e. 'make test SUITE=unit') make test-core # Core tests (i.e. 'make test SUITE=core') make test-systest # System integration tests (i.e. 'make test SUITE=systest') make test-vector # Vector search tests (i.e. 'make test SUITE=vector') make test-ldbc # LDBC benchmark tests (i.e. 'make test SUITE=ldbc') make test-load # Heavy load tests (i.e. 'make test SUITE=load') # TAGS-based targets make test-integration # Integration tests (i.e. 'make test TAGS=integration') make test-integration2 # Integration2 tests (i.e. 'make test TAGS=integration2') make test-upgrade # Upgrade tests (i.e. 'make test TAGS=upgrade') # Other make test-fuzz # Fuzz tests (i.e. 'make test FUZZ=1') make test-benchmark PKG=posting # Benchmark a specific package ``` **4. Dynamic Discovery** `make help` dynamically discovers available SUITE values from `t/t.go` and TAGS values from test file build directives. ### Changes - **Build automation**: `make install` auto-detects the OS and builds both native and Linux binaries on non-Linux systems - **Unified test target**: `make test` routes to t/ runner or `go test` based on `SUITE`, `TAGS`, `FUZZ` variables - **Complete helper targets**: Every SUITE and TAGS value has a corresponding `test-*` target - **Dynamic help**: `make help` shows available SUITE/TAGS values discovered from the codebase - **Environment variable**: Introduced `LINUX_GOBIN` to specify where Linux binaries are stored for Docker tests - **Docker compose**: All systest docker-compose files updated to use `${LINUX_GOBIN:-$GOPATH/bin}` for binary mounting - **Documentation**: Updated TESTING.md, CONTRIBUTING.md with comprehensive examples ### Before (macOS) ```bash make install mv $GOPATH/bin/dgraph $GOPATH/bin/dgraph_osx GOOS=linux make install mv $GOPATH/bin/linux_arm64/dgraph $GOPATH/bin/dgraph export DGRAPH_BINARY=$GOPATH/bin/dgraph_osx cd t && ./t --pkg=<package> ``` ### After (macOS or Linux) ```bash make install make test SUITE=systest PKG=systest/export ``` ### `make help` Output ``` Usage: make [target] [VAR=value ...] Targets: all Build all targets dgraph Build dgraph binary docker-image Build Docker image (dgraph/dgraph:$VERSION) help Show available targets and variables image-local Alias for local-image install Install dgraph binary local-image Build local Docker image (dgraph/dgraph:local) test-all All test suites via t/ runner (i.e. 'make test SUITE=all') test-benchmark Go benchmarks (i.e. 'go test -bench') test-core Core tests (i.e. 'make test SUITE=core') test-fuzz Fuzz tests, auto-discovers packages (i.e. 'make test FUZZ=1') test-integration Integration tests (i.e. 'make test TAGS=integration') test-integration2 Integration2 tests via dgraphtest (i.e. 'make test TAGS=integration2') test-ldbc LDBC benchmark tests (i.e. 'make test SUITE=ldbc') test-load Heavy load tests (i.e. 'make test SUITE=load') test-systest System integration tests (i.e. 'make test SUITE=systest') test-unit Unit tests, no Docker (i.e. 'make test SUITE=unit') test-upgrade Upgrade tests (i.e. 'make test TAGS=upgrade') test-vector Vector search tests (i.e. 'make test SUITE=vector') test Run tests (see 'make help' for options) uninstall Uninstall dgraph binary version Show build version info Variables that can be passed to 'test': SUITE Select t/ runner suite (e.g., make test SUITE=systest) TAGS Go build tags - bypasses t/ runner (e.g., make test TAGS=integration2) PKG Limit to specific package (e.g., make test PKG=systest/export) TEST Run specific test function (e.g., make test TEST=TestGQLSchema) FUZZ Enable fuzz testing (e.g., make test FUZZ=1) FUZZTIME Fuzz duration per package (e.g., make test FUZZ=1 FUZZTIME=60s) Available SUITE values: all ldbc load unit systest vector core Available TAGS values: integration integration2 upgrade Examples: make test TAGS=integration2 PKG=systest/vector # integration2 tests for vector make test TAGS=upgrade PKG=acl TEST=TestACL # specific upgrade test make test FUZZ=1 PKG=dql FUZZTIME=30s # fuzz dql package for 30s make test SUITE=systest PKG=systest/backup/filesystem # systest for backup pkg make test-benchmark PKG=posting # benchmark posting package ``` ## Test plan - [x] Run `make install` on macOS and verify both binaries are created - [x] Run `make check` in `t/` directory - [x] Verify `make test-unit` routes to t/ runner with `--suite=unit` - [x] Verify `make test TAGS=integration2 PKG=types` runs go test directly - [x] Verify `make test FUZZ=1 PKG=dql FUZZTIME=5s` runs fuzz tests - [x] Verify `make test-benchmark PKG=types` runs benchmarks - [x] Verify `make help` shows all targets, variables, and discovered SUITE/TAGS values - [x] Verify every SUITE value has a corresponding `test-*` target - [x] Verify every TAGS value has a corresponding `test-*` target - [x] Verify Linux CI still works as expected
1 parent 408e8f8 commit 974923a

44 files changed

Lines changed: 2420 additions & 213 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,3 +46,5 @@ systest/bulk_live/live/**/*.rdf
4646
systest/bulk_live/live/**/*.txt
4747
x/log_test/*.enc
4848
*.buf
49+
.osgrep
50+
.worktrees/

‎CONTRIBUTING.md‎

Lines changed: 22 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -137,23 +137,28 @@ then use this local image to test Dgraph in your local Docker setup.
137137

138138
### Testing
139139

140-
Dgraph employs a ~~complex~~ sophisticated testing framework that includes extensive test coverage.
141-
Due to the comprehensive nature of these tests, a complete test run can take several hours,
142-
depending on your hardware. To manage this complex testing process efficiently, we've developed a
143-
custom test framework implemented in Go, which resides in the [./t](/t) directory. This specialized
144-
framework provides enhanced control and flexibility beyond what's available through standard Go
145-
testing framework.
146-
147-
For dependencies, runner flags and instructions for running tests on non-Linux machines, see the
148-
[README](t/README.md) in the [_t_](t) folder.
149-
150-
Other integration tests do not use the testing framework located in the `t` folder. Consult the
151-
[github workflow definitions](.github/workflows) folder to discover the tests we run as part of our
152-
continuous integration process.
153-
154-
Non-integration unit tests exist for many core packages that can be exercised without invoking the
155-
testing framework. For instance, to unit test the core DQL parsing package:
156-
`go test github.com/dgraph-io/dgraph/v25/dql`.
140+
Dgraph employs a ~~complex~~ sophisticated testing framework with extensive test coverage. A full
141+
test run can take several hours. We've developed a custom test runner in Go in the [t/](t)
142+
directory, providing control and flexibility beyond the standard Go testing framework.
143+
144+
The simplest way to run tests is via Make:
145+
146+
```bash
147+
# Run all tests
148+
make test
149+
150+
# Run specific test types
151+
make test-unit # Unit tests only (no Docker)
152+
make test-integration2 # Integration2 tests via dgraphtest
153+
make test-upgrade # Upgrade tests
154+
155+
# Use variables for more control
156+
make test TAGS=integration2 PKG=systest/vector
157+
```
158+
159+
Run `make help` to see all available targets and variables.
160+
161+
For a comprehensive testing guide, see [TESTING.md](TESTING.md).
157162

158163
## Contributing
159164

‎Makefile‎

Lines changed: 146 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,21 @@ BUILD_DATE ?= $(shell git log -1 --format=%ci)
99
BUILD_BRANCH ?= $(shell git rev-parse --abbrev-ref HEAD)
1010
BUILD_VERSION ?= $(shell git describe --always --tags)
1111

12-
GOPATH ?= $(shell go env GOPATH)
12+
export GOPATH ?= $(shell go env GOPATH)
13+
GOHOSTOS := $(shell go env GOHOSTOS)
14+
GOHOSTARCH := $(shell go env GOHOSTARCH)
15+
16+
# Guard against empty GOPATH, which would resolve paths to root (e.g. /bin)
17+
ifeq ($(GOPATH),)
18+
$(error GOPATH is not set. Please set it explicitly, e.g. export GOPATH=$$HOME/go)
19+
endif
20+
21+
# On non-Linux systems, use a separate directory for Linux binaries
22+
ifeq ($(GOHOSTOS),linux)
23+
export LINUX_GOBIN ?= $(GOPATH)/bin
24+
else
25+
export LINUX_GOBIN ?= $(GOPATH)/linux_$(GOHOSTARCH)
26+
endif
1327

1428
######################
1529
# Build & Release Parameters
@@ -20,18 +34,18 @@ GOPATH ?= $(shell go env GOPATH)
2034
DGRAPH_VERSION ?= local
2135

2236
.PHONY: all
23-
all: dgraph
37+
all: dgraph ## Build all targets
2438

2539
.PHONY: dgraph
26-
dgraph:
40+
dgraph: ## Build dgraph binary
2741
$(MAKE) -w -C $@ all
2842

2943
.PHONY: dgraph-coverage
3044
dgraph-coverage:
3145
$(MAKE) -w -C dgraph test-coverage-binary
3246

3347
.PHONY: version
34-
version:
48+
version: ## Show build version info
3549
@echo Dgraph: ${BUILD_VERSION}
3650
@echo Build: ${BUILD}
3751
@echo Codename: ${BUILD_CODENAME}
@@ -40,30 +54,115 @@ version:
4054
@echo Go version: $(shell go version)
4155

4256
.PHONY: install
43-
install:
44-
@echo "Installing Dgraph..."; \
45-
$(MAKE) -C dgraph install; \
57+
install: ## Install dgraph binary
58+
@echo "Installing dgraph ($(GOHOSTOS)/$(GOHOSTARCH))..."
59+
@$(MAKE) -C dgraph install
60+
ifneq ($(GOHOSTOS),linux)
61+
@mkdir -p $(LINUX_GOBIN)
62+
@echo "Installing dgraph (linux/$(GOHOSTARCH))..."
63+
@GOOS=linux GOARCH=$(GOHOSTARCH) $(MAKE) -C dgraph dgraph
64+
@mv dgraph/dgraph $(LINUX_GOBIN)/dgraph
65+
@echo "Installed dgraph (linux/$(GOHOSTARCH)) to $(LINUX_GOBIN)/dgraph"
66+
endif
67+
4668

4769
.PHONY: uninstall
48-
uninstall:
70+
uninstall: ## Uninstall dgraph binary
4971
@echo "Uninstalling Dgraph ..."; \
5072
$(MAKE) -C dgraph uninstall; \
5173

52-
.PHONY: test
53-
test: docker-image
54-
@mv dgraph/dgraph ${GOPATH}/bin/dgraph
55-
@$(MAKE) -C t test
74+
.PHONY: dgraph-installed
75+
dgraph-installed:
76+
@if [ ! -f "$(GOPATH)/bin/dgraph" ] || [ ! -f "$(LINUX_GOBIN)/dgraph" ]; then \
77+
echo "Dgraph binary missing, running make install..."; \
78+
$(MAKE) install; \
79+
fi
5680

57-
.PHONY: image-local local-image
58-
image-local local-image:
81+
.PHONY: test
82+
test: dgraph-installed local-image ## Run tests (see 'make help' for options)
83+
ifdef TAGS
84+
@echo "Running tests with tags: $(TAGS)"
85+
go test -v --tags="$(TAGS)" \
86+
$(if $(TEST),--run="$(TEST)") \
87+
$(if $(PKG),./$(PKG)/...,./...)
88+
else ifdef FUZZ
89+
@echo "Discovering and running fuzz tests..."
90+
ifdef PKG
91+
go test -v -fuzz=Fuzz -fuzztime=$(or $(FUZZTIME),300s) ./$(PKG)/...
92+
else
93+
@grep -r "^func Fuzz" --include="*_test.go" -l . 2>/dev/null | \
94+
xargs -I{} dirname {} | sort -u | while read dir; do \
95+
echo "Fuzzing $$dir..."; \
96+
go test -v -fuzz=Fuzz -fuzztime=$(or $(FUZZTIME),300s) ./$$dir/...; \
97+
done
98+
endif
99+
else
100+
@echo "Running test suite: $(or $(SUITE),all)"
101+
$(MAKE) -C t test args="--suite=$(or $(SUITE),all) $(if $(PKG),--pkg=\"$(PKG)\") $(if $(TEST),--test=\"$(TEST)\")"
102+
endif
103+
104+
.PHONY: test-all
105+
test-all: ## All test suites via t/ runner (i.e. 'make test SUITE=all')
106+
@SUITE=all $(MAKE) test
107+
108+
.PHONY: test-unit
109+
test-unit: ## Unit tests, no Docker (i.e. 'make test SUITE=unit')
110+
@SUITE=unit $(MAKE) test
111+
112+
.PHONY: test-core
113+
test-core: ## Core tests (i.e. 'make test SUITE=core')
114+
@SUITE=core $(MAKE) test
115+
116+
.PHONY: test-integration
117+
test-integration: ## Integration tests (i.e. 'make test TAGS=integration')
118+
@TAGS=integration $(MAKE) test
119+
120+
.PHONY: test-integration2
121+
test-integration2: ## Integration2 tests via dgraphtest (i.e. 'make test TAGS=integration2')
122+
@TAGS=integration2 $(MAKE) test
123+
124+
.PHONY: test-upgrade
125+
test-upgrade: ## Upgrade tests (i.e. 'make test TAGS=upgrade')
126+
@TAGS=upgrade $(MAKE) test
127+
128+
.PHONY: test-systest
129+
test-systest: ## System integration tests (i.e. 'make test SUITE=systest')
130+
@SUITE=systest $(MAKE) test
131+
132+
.PHONY: test-vector
133+
test-vector: ## Vector search tests (i.e. 'make test SUITE=vector')
134+
@SUITE=vector $(MAKE) test
135+
136+
.PHONY: test-fuzz
137+
test-fuzz: ## Fuzz tests, auto-discovers packages (i.e. 'make test FUZZ=1')
138+
@FUZZ=1 $(MAKE) test
139+
140+
.PHONY: test-ldbc
141+
test-ldbc: ## LDBC benchmark tests (i.e. 'make test SUITE=ldbc')
142+
@SUITE=ldbc $(MAKE) test
143+
144+
.PHONY: test-load
145+
test-load: ## Heavy load tests (i.e. 'make test SUITE=load')
146+
@SUITE=load $(MAKE) test
147+
148+
.PHONY: test-benchmark
149+
test-benchmark: ## Go benchmarks (i.e. 'go test -bench')
150+
go test -bench=. -benchmem $(if $(PKG),./$(PKG)/...,./...)
151+
152+
.PHONY: local-image
153+
local-image: ## Build local Docker image (dgraph/dgraph:local)
154+
@echo building local docker image
59155
@GOOS=linux GOARCH=amd64 $(MAKE) dgraph
60156
@mkdir -p linux
61157
@mv ./dgraph/dgraph ./linux/dgraph
62158
@docker build -f contrib/Dockerfile -t dgraph/dgraph:local .
63159
@rm -r linux
64160

161+
.PHONY: image-local
162+
image-local: local-image ## Alias for local-image
163+
65164
.PHONY: docker-image
66-
docker-image: dgraph
165+
docker-image: dgraph ## Build Docker image (dgraph/dgraph:$VERSION)
67166
@mkdir -p linux
68167
@cp ./dgraph/dgraph ./linux/dgraph
69168
docker build -f contrib/Dockerfile -t dgraph/dgraph:$(DGRAPH_VERSION) .
@@ -93,14 +192,35 @@ linux-dependency:
93192
sudo apt-get -y install protobuf-compiler
94193

95194
.PHONY: help
96-
help:
97-
@echo
98-
@echo Build commands:
99-
@echo " make [all] - Build all targets [EE]"
100-
@echo " make dgraph - Build dgraph binary"
101-
@echo " make install - Install all targets"
102-
@echo " make uninstall - Uninstall known targets"
103-
@echo " make version - Show current build info"
104-
@echo " make help - This help"
105-
@echo " make test - Make local image and run t.go"
106-
@echo
195+
help: ## Show available targets and variables
196+
@echo "Usage: make [target] [VAR=value ...]"
197+
@echo ""
198+
@echo "Targets:"
199+
@grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(MAKEFILE_LIST) | \
200+
sort | \
201+
awk 'BEGIN {FS = ":.*?## "}; {printf " %-20s %s\n", $$1, $$2}'
202+
@echo ""
203+
@echo "Variables that can be passed to 'test':"
204+
@echo " SUITE Select t/ runner suite (e.g., make test SUITE=systest)"
205+
@echo " TAGS Go build tags - bypasses t/ runner (e.g., make test TAGS=integration2)"
206+
@echo " PKG Limit to specific package (e.g., make test PKG=systest/export)"
207+
@echo " TEST Run specific test function (e.g., make test TEST=TestGQLSchema)"
208+
@echo " FUZZ Enable fuzz testing (e.g., make test FUZZ=1)"
209+
@echo " FUZZTIME Fuzz duration per package (e.g., make test FUZZ=1 FUZZTIME=60s)"
210+
@echo ""
211+
@printf " Available SUITE values: "
212+
@grep -o 'allowed := \[\]string{[^}]*}' t/t.go 2>/dev/null | \
213+
sed 's/allowed := \[\]string{"\([^}]*\)"}/\1/' | \
214+
tr -d '"' | tr ',' ' ' || echo "all, unit, core, systest, vector, ldbc, load"
215+
@printf " Available TAGS values: "
216+
@grep -roh "//go:build [a-z0-9]*" --include="*_test.go" . 2>/dev/null | \
217+
awk '{print $$2}' | \
218+
grep -E '^(integration|integration2|upgrade)$$' | \
219+
sort -u | tr '\n' ' ' && echo ""
220+
@echo ""
221+
@echo "Examples:"
222+
@echo " make test TAGS=integration2 PKG=systest/vector # integration2 tests for vector"
223+
@echo " make test TAGS=upgrade PKG=acl TEST=TestACL # specific upgrade test"
224+
@echo " make test FUZZ=1 PKG=dql FUZZTIME=30s # fuzz dql package for 30s"
225+
@echo " make test SUITE=systest PKG=systest/backup/filesystem # systest for backup pkg"
226+
@echo " make test-benchmark PKG=posting # benchmark posting package"

0 commit comments

Comments
 (0)