Fix Decode() of empty input for non-str return types (proto crash, wrong type for bytes/serialized_proto)

[sohumt123]

Describe the change

SentencePieceProcessor._decode_raw() short-circuits empty/None input by returning the literal str '' for every return_type. The str case happens to be correct, but the other return types are broken:

• Decode([], return_type='proto') crashes: the str '' is passed to SentencePieceText.ParseFromString(), which raises TypeError: expected bytes, str found.
• Decode([], return_type=bytes) returns str '' instead of b'' (the non-empty path returns bytes).
• Decode([], return_type='serialized_proto') returns str '' instead of the serialized empty proto (bytes).

Root cause: the empty-input guard in _decode_raw() hardcodes return '' and ignores the return_type argument, bypassing the correctly-typed empty results the C++ layer already produces (_DecodeIdsAsBytes([]) returns b'', _DecodeIdsAsSerializedProto([]) returns a valid serialized empty SentencePieceText).

Fix: instead of returning a hardcoded '', normalize empty/None input to an empty id sequence and fall through to the regular dispatch, so the result comes from the C++ method and is typed according to return_type — exactly like non-empty input. The batch check now guards input[0] with len(input) > 0. No behavior change for any non-empty input, and the default return_type=str still returns '' for empty input.

Link to a related Issue

Fixes #1288

Testing Information

Added test_decode_empty_input to python/test/sentencepiece_test.py, covering Decode([]) and Decode(None) for return_type in {str, bytes, 'serialized_proto', 'proto', 'offset_mapping'}. Without the fix it fails with TypeError: expected bytes, str found on the 'proto' case (and the bytes type assertions); with the fix it passes.

Ran the full Python test suite with the extension built in place (python3.11 setup.py build_ext --inplace):

PYTHONPATH=src python3.11 -m pytest test/sentencepiece_test.py test/numpy_test.py -q

All tests pass (58 passed), including the numpy suite, confirming no regression on single/batch id/piece decoding or empty numpy arrays.

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.