CJK (Korean/Chinese/Japanese) character rendering and IME input support

[BikiBikiBik]

Summary

cmux-windows does not properly handle CJK (Chinese, Japanese, Korean) characters. There are two distinct issues:

1. Display: CJK characters rendered incorrectly (character width)

CJK characters occupy 2 terminal cells, but TerminalCell.Width is always set to 1 in WriteChar() and SetChar(). This causes overlapping and garbled output for all CJK text.

Root cause: No Unicode East Asian Width (UAX #11) implementation.

Affected files:

• TerminalBuffer.cs: WriteChar() and SetChar() hardcode Width = 1
• TerminalControl.cs: Rendering assumes x = col * _cellWidth without width compensation
• FlushTextRun(): runWidth = _textRunBuffer.Length * _cellWidth (assumes 1 cell per char)
• Mouse position, selection, URL detection all assume uniform 1-cell width

Fix needed: Implement wcwidth() equivalent, set Width = 2 for wide characters, mark next cell as continuation (Width = 0), and adjust rendering/mouse/selection accordingly.

2. Input: Korean IME composition fails — jamo appear decomposed

When typing Korean, individual jamo (ㅋ, ㅡ, ㄹ) appear instead of composed syllables (클). The final consonant (batchim) consistently separates: "전" appears as "저" + "ㄴ".

Root cause: TerminalControl extends FrameworkElement, which lacks TSF (Text Services Framework) integration. Without TSF, the Windows IME cannot maintain composition state — it falls back to committing each consonant+vowel pair immediately without waiting for a possible final consonant.

Approaches tried (all insufficient):

• Key.ImeProcessed check in OnKeyDown — partial improvement
• InputMethod.SetIsInputMethodEnabled — no effect
• Win32 IMM32 API hooks (ImmGetContext, WM_IME_COMPOSITION) — WPF uses TSF, not IMM32
• Hidden TextBox as IME proxy with TextChanged — fires during composition, not just on commit
• Hidden TextBox with TextCompositionManager.TextInput — focus routing breaks

Fix needed: Either rebase TerminalControl on a control with TSF support (e.g., wrap a TextBox), or implement custom TSF integration for proper IME composition handling.

Environment

• Windows 11 Pro
• cmux-windows v1.0.6
• Korean (두벌식) keyboard layout
• .NET 10 SDK

Screenshots

Korean output garbled (before any fix):

• Characters overlap and mix due to width=1 for all characters

Korean input decomposed:

• Typing "클로드코드" produces "클르ㄴㄷ—ㅋㄴㄷ" (batchim separated)

Related

This is a common challenge for Windows terminal emulators. References:

• microsoft/terminal #226 — CJK font fallback
• cmderdev/cmder #1066 — Korean character width
• Maximus5/ConEmu #1330 — Character width calculation

[BikiBikiBik]

Additional finding: Root cause identified from Microsoft Terminal's fix

Microsoft Terminal had the exact same issue and fixed it in PR #4796.

Root cause

Korean IME composes characters differently from Chinese/Japanese. When a CompositionCompleted event fires, the next syllable's initial consonant is already in the input buffer. If the buffer is cleared at this point, the initial consonant of the next character is lost — causing jamo separation, especially during fast typing.

How Microsoft Terminal solved it

They implemented a dedicated TSFInputControl that:

1. Does NOT clear the input buffer on CompositionCompleted
2. Only clears the buffer when the user presses Enter or Esc
3. This preserves the composition context across syllable boundaries

Reference implementation: TSFInputControl.cpp

Recommended approach for cmux-windows

The current TerminalControl (inheriting FrameworkElement) lacks TSF (Text Services Framework) integration, which is required for proper Korean/CJK IME support. The fix would involve:

1. Implementing a TSF-aware input control (similar to Windows Terminal's TSFInputControl)
2. Not clearing the composition buffer on CompositionCompleted
3. Rendering the composing text inline (underlined preview) during composition

This is an architecture-level change to the input handling system.

[BikiBikiBik]

Alternative approaches (without full TSF rewrite)

1. Overlay TextBox at cursor position (least invasive)

Place a small WPF TextBox at the terminal cursor position during IME composition. The TextBox handles IME natively. On composition complete, send text to ConPTY and hide the overlay. This is how Windows Terminal handled it before their full TSF integration.

2. Track _activeTextStart pointer (Windows Terminal pattern)

Instead of clearing the entire buffer on CompositionCompleted, track the composition start position. Only send the range [_activeTextStart, currentPosition] to the terminal. Don't clear the buffer — update _activeTextStart to the current position. Clear everything only on Enter/Esc.

3. AvalonEdit-style approach

Use WinAPI ImmGetContext/ImmSetCompositionWindow to position the IME composition window at the cursor location, letting Windows handle the composition display. Combined with approach #2 for buffer management.

Reference: AvalonEdit IME integration

[zaiyuan119]

I hit the same issue on Windows 11 with cmux-windows v1.0.6, mainly with Chinese output:

• CJK text looked like it had extra spaces
• mixed Chinese/English text overlapped
• colored mixed-width text was especially broken

I patched this locally and the display issue is now much better. The main fix was to stop treating every character as width 1.

What I changed locally:

• Added a Unicode width helper to distinguish:
  • 0 = continuation cell for a wide glyph
  • 1 = normal-width cell
  • 2 = wide-character start cell
• Updated TerminalBuffer so wide characters occupy two terminal cells:
  • write the real glyph into the first cell with Width = 2
  • mark the next cell as a continuation cell with Width = 0
• Updated rendering in TerminalControl to respect cell.Width for:
  • background rectangles
  • text drawing
  • underline / strikethrough
  • cursor width
  • mouse hit-testing
• Updated copy / export paths to skip continuation cells, so they do not introduce fake spaces

Files touched in my local patch:

• src/Cmux.Core/Terminal/TerminalBuffer.cs
• src/Cmux.Core/Terminal/TerminalAttribute.cs
• src/Cmux.Core/Terminal/TerminalSelection.cs
• src/Cmux/Controls/TerminalControl.cs
• added src/Cmux.Core/Terminal/UnicodeWidth.cs

This does not attempt a full TSF/IME architecture fix. It is specifically a wide-character cell-width/rendering fix, but for Chinese display it resolved the visible corruption for me.