Introduce a Bazel build system

[BYVoid]

Describe the change

This PR adds a bzlmod-based Bazel build system alongside the existing CMake build. It covers the runtime and trainer libraries, the five spm_* command line tools, and a spm_test target that runs the full unit test suite (equivalent to the CMake spm_test built with -DSPM_BUILD_TEST=ON), plus a GitHub Actions workflow that builds and tests with Bazel on Linux (x86_64/arm64), macOS, and Windows. Usage is documented in doc/bazel.md.

External dependencies. All third-party code is resolved from the Bazel Central Registry; nothing is vendored into the Bazel build (the vendored copies under third_party/ and src/builtin_pb remain in the tree for CMake, whose behavior is unchanged):

• abseil-cpp, pinned to the same version as the GIT_TAG in CMakeLists.txt;
• protobuf — the .pb.cc/.pb.h files are regenerated from src/*.proto at build time via proto_library/cc_proto_library, equivalent to the SPM_PROTOBUF_PROVIDER=package CMake configuration and enabled through the existing _USE_EXTERNAL_PROTOBUF support;
• darts-clone 0.32h.bcr.1 — upstream v0.32h plus the compatibility helpers (copy_array(), validate()) that the copy vendored here carries;
• esaxx 20250106.1.bcr.1 with your fixes.

Why the third_party/ include prefix had to go. The sources now include third-party headers by the canonical paths the upstream projects export ("absl/...", "darts.h", "esa.hxx") instead of the google3-style "third_party/..." paths, for two reasons:

1. google3 layout vs. OSS convention. Inside google3, third_party/absl is the real repository path, but in the open-source Bazel ecosystem dependencies export canonical include paths, and the OSS trees of TensorFlow, protobuf, and gRPC all consume Abseil as "absl/...". Keeping the internal prefix forces every build system to maintain an adaptation layer — in this repository that is the third_party/absl symlink created at configure time plus the install-time header rewriting in cmake/modify_headers.cmake; a Bazel build would need an equivalent layer of generated forwarding headers.
2. Windows. Bazel builds without sandboxing on Windows, so quote-include resolution physically picks up the vendored in-tree files at the literal "third_party/..." paths. They shadow the module headers and fail Bazel's strict inclusion checks, which we hit in CI; only the canonical prefix-free paths behave identically across all platforms.

As a side effect, the CMake symlink machinery and cmake/modify_headers.cmake are no longer needed: the abseil-cpp checkout root and the vendored darts_clone/esaxx directories are put on the include path directly, and the public headers are installed as-is (content-equivalent to what the rewrite script used to generate).

The optional CMake features SPM_ENABLE_NFKC_COMPILE, SPM_ENABLE_TCMALLOC, SPM_ENABLE_BENCHMARK, and SPM_NLCODEC_BPE have no Bazel equivalent yet; this is noted in the docs.

Link to a related Issue

None. Opened as a draft for discussion.

Testing Information

• bazel build //... and bazel test //src:spm_test (all 164 test cases) pass on Linux x86_64, Linux arm64, macOS, and Windows via the included GitHub Actions workflow, which also smoke-tests spm_train/spm_encode end to end (latest green run).
• The full CMake pipeline (configure with the module provider, build, ctest, cmake --install) passes; the installed sentencepiece_processor.h/sentencepiece_trainer.h carry the same absl/... includes the deleted rewrite script used to generate.
• A model trained and encoded with the Bazel build produces output identical to the CMake/vendored build (spm_train on data/botchan.txt, vocab 1000, then spm_encode).

[BYVoid]

Hi @taku910,

This is Carbo.

I'd love to hear your thoughts on this PR.

I use SentencePiece from a Bazel-based project, and this PR upstreams the Bazel build I have been running there: a bzlmod MODULE.bazel with all dependencies (abseil-cpp, protobuf, darts-clone, esaxx) resolved from the Bazel Central Registry, the full unit test suite as a bazel test target, and a CI workflow covering Linux (x86_64/arm64), macOS, and Windows.

I'm happy to maintain the Bazel build long-term, keeping the dependency pins in sync with CMake, following Bazel/BCR releases, and fixing any breakage the CI surfaces. The PR is marked as draft for discussion; the include-path change it requires (dropping the google3-style third_party/ prefix in favor of the canonical absl/... paths, as TensorFlow/protobuf/gRPC do in their OSS trees) is explained in the description, and I'm glad to adjust the approach if you'd prefer something different.

[taku910]

Thank you.

Maintaining both Bazel and CMake is not feasible, especially given our current bandwidth constraints. Therefore, I would like to make the transition to Bazel an exclusive one. With today's AI coding tools, I believe the migration itself shouldn't be too difficult.

On the other hand, there is a potential issue where Bazel lacks sufficient support from package maintainers like those for Debian, and often isn't provided upstream. This is currently happening with Mozc, where the Ubuntu package is still relying on patches from the old GYP-based branch, leaving it about two years behind upstream.

Since there is no clear answer yet, It would be nice to create an issue and consolidate the feedback there.

[BYVoid]

Thank you. I agree that distro packaging is an important concern, and that is one reason I think Bazel support is useful as an additional build path alongside the existing CMake build.

This PR uses Bzlmod/BCR, which is convenient for Bazel consumers, while CMake remains the path that Linux distribution/package maintainers already understand and can patch. Distribution builds often need no-network builds, declared packaged build dependencies, system libraries, shared-library packaging, pkg-config/CMake metadata, etc., so keeping CMake available continues to serve that use case.

This dual-build approach also matches the direction of several Google C++ OSS libraries. Abseil documents Bazel and CMake as official build systems; GoogleTest has both Bazel and CMake quickstarts and follows the Google Foundational C++ Support Policy; gRPC uses Bazel as primary but still supports CMake for C++; RE2 and Highway also support both. Google’s own C++ support policy notes that CMake is dominant externally while Bazel is essential for Google, and suggests supporting Bazel and CMake as the practical minimum.

I’m happy to help maintain the Bazel build long-term: keep dependency pins aligned with CMake, follow Bazel/BCR changes, maintain the Bazel CI, investigate and fix Bazel-specific breakages, and review future Bazel-related PRs/issues when they come up.