[BUG] fix(encoding): toByteArray silently discards invalid hex characters causing data corruption

[faizahmad-khan]

Bug Description

toByteArray(str, 'hex') in lib/utils/encoding.ts silently strips all non-hex characters via a regex before parsing, instead of throwing a validation error. This causes silent data corruption — callers receive a wrong byte array with no indication that the input was invalid.

Steps to Reproduce

1. Call toByteArray('xyz', 'hex') — any non-hex string works.
2. Observe: no error is thrown, and a 0-byte array is returned.
3. Reproduce in the app: select any hex-input cipher (RC4, OTP, 3DES), enter xyz as ciphertext, and trigger decryption — wrong output, no error shown.

Expected Behavior

toByteArray(str, 'hex') should throw a CipherError with code INVALID_INPUT and a message like "Invalid hex string: contains non-hex characters." whenever the input contains any character outside [0-9a-fA-F].

Actual Behavior

The function runs str.replace(/[^0-9a-fA-F]/g, '') which silently removes all invalid characters. Because the stripped string can have even length (including length 0), the even-length check passes and no error is ever thrown:

• toByteArray('xyz', 'hex') → Uint8Array(0) — 0 bytes, no error
• toByteArray('0x1a', 'hex') → strips 0x, silently returns [0x1a]
• toByteArray('AES!key', 'hex') → strips !, returns wrong bytes

Root cause:

lib/utils/encoding.ts line 9. All cipher decrypt() functions are affected: rc4.ts, otp.ts, 3des.ts, chacha20.ts, aes.ts, des.ts.

Screenshots

N/A — reproducible via unit test or browser console.

Environment

• OS: Any
• Browser: Any
• Version: Latest (main branch)

Additional Context

Proposed fix (lib/utils/encoding.ts):

- const clean = str.replace(/[^0-9a-fA-F]/g, '')
- if (clean.length % 2 !== 0) {
-   throw new CipherError('INVALID_KEY', 'Hex string must have an even length.')
- }
- const arr = new Uint8Array(clean.length / 2)
+ if (str.length > 0 && /[^0-9a-fA-F]/.test(str)) {
+   throw new CipherError('INVALID_INPUT', 'Invalid hex string: contains non-hex characters.')
+ }
+ if (str.length % 2 !== 0) {
+   throw new CipherError('INVALID_INPUT', 'Hex string must have an even number of characters.')
+ }
+ const arr = new Uint8Array(str.length / 2)

[faizahmad-khan]

Hi @csxark 👋

I'd love to work on this! Could you please assign this issue to me? 🙏

I've already identified the root cause in lib/utils/encoding.ts (line 9) and have a working fix ready that:

• Replaces the silent-strip regex with strict validation
• Uses the correct INVALID_INPUT error code instead of INVALID_KEY
• Passes all 304 existing unit tests with zero regressions

Suggested labels: fix cipher good first issue ECSoC26

Thanks! 🚀

[csxark]

Hi! @faizahmad-khan, I'm assigning this initiative to you.

A warm welcome to the CryptoViz project! We're excited to have you on board and truly appreciate your willingness to contribute. Looking forward to your creativity and meaningful contributions, especially from the branding perspective. Wishing you the very best, and happy contributing! 🚀