Skip to content

Commit 4badb27

Browse files
ChaitanayaKulkarnikawasaki
authored andcommitted
Makefile: add check-parallel target for faster shellcheck
Add a new check-parallel target that utilizes all available CPUs to run shellcheck in parallel on test files, significantly reducing validation time from ~40s to ~11s on a 48-core system. The original check target remains unchanged for compatibility. blktests (master) # vim 0001-Makefile-add-check-parallel-target-for-faster-shellc.patch blktests (master) # time make check shellcheck -x -e SC2119 -f gcc check common/* \ tests/*/rc tests/*/[0-9]*[0-9] src/*.sh check:634:20: note: Double quote to prevent globbing and word splitting. [SC2086] check:667:20: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:447:5: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:453:6: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:553:42: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:554:28: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:556:28: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:557:28: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:558:23: note: Double quote to prevent globbing and word splitting. [SC2086] tests/pr.bk/001:9:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/001:11:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/001:12:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/002:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/002:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/002:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/003:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/003:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/003:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/004:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/004:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/004:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/005:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/005:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/005:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/006:9:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/006:11:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/006:12:1: warning: TIMED appears unused. Verify use (or export if used externally). [SC2034] make: *** [Makefile:22: check] Error 1 real 3m43.712s user 3m38.640s sys 0m3.321s Running shellcheck with 48 parallel jobs... tests/pr.bk/005:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/005:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/005:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/006:9:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/006:11:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/006:12:1: warning: TIMED appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/001:9:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/001:11:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/001:12:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/002:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/002:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/002:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/004:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/004:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/004:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/003:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/003:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/003:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] check:634:20: note: Double quote to prevent globbing and word splitting. [SC2086] check:667:20: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:447:5: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:453:6: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:553:42: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:554:28: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:556:28: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:557:28: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:558:23: note: Double quote to prevent globbing and word splitting. [SC2086] make: *** [Makefile:30: check-parallel] Error 1 real 0m22.464s user 7m24.518s sys 0m11.703s 10.0x faster than sequential Concrete numbers: - make check: 223.7s (3m43.7s) - make check-parallel: 22.5s Signed-off-by: Chaitanya Kulkarni <[email protected]> Signed-off-by: Shin'ichiro Kawasaki <[email protected]>
1 parent 1673a2b commit 4badb27

1 file changed

Lines changed: 13 additions & 1 deletion

File tree

Makefile

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ install:
1616
# SC2119: "Use foo "$@" if function's $1 should mean script's $1". False
1717
# positives on helpers like _init_scsi_debug.
1818
SHELLCHECK_EXCLUDE := SC2119
19+
NPROCS := $(shell nproc)
1920

2021
check:
2122
shellcheck -x -e $(SHELLCHECK_EXCLUDE) -f gcc check common/* \
@@ -24,4 +25,15 @@ check:
2425
! grep TODO tests/*/rc tests/*/[0-9]*[0-9]
2526
! find -L -name '*.out' -perm /u=x+g=x+o=x -printf '%p is executable\n' | grep .
2627

27-
.PHONY: all check install
28+
check-parallel:
29+
@echo "Running shellcheck with $(NPROCS) parallel jobs..."
30+
@ret=0; \
31+
find tests -type f -name '[0-9]*[0-9]' | \
32+
xargs -P $(NPROCS) -n 1 shellcheck -x -e $(SHELLCHECK_EXCLUDE) -f gcc || ret=1; \
33+
shellcheck -x -e $(SHELLCHECK_EXCLUDE) -f gcc check common/* tests/*/rc src/*.sh || ret=1; \
34+
shellcheck --exclude=$(SHELLCHECK_EXCLUDE),SC2154 --format=gcc new || ret=1; \
35+
grep TODO tests/*/rc tests/*/[0-9]*[0-9] && ret=1; \
36+
find -L -name '*.out' -perm /u=x+g=x+o=x -printf '%p is executable\n' | grep . && ret=1; \
37+
exit $$ret
38+
39+
.PHONY: all check check-parallel install

0 commit comments

Comments
 (0)