feat(terminal): Chinese / IME input (P3)

Implement IME composition support and migrate regular character input to
a hidden textarea, matching the xterm.js approach.

Key changes:
- Add a hidden, absolutely-positioned `<textarea>` inside `.terminal-screen`
  that follows the terminal cursor cell. It receives focus on click, connect,
  and resize so the OS/browser IME candidate window appears at the cursor.
- `on:input` now sends regular printable characters; `on:keydown` only handles
  special keys (arrows, Enter, Esc, Backspace, F-keys, Ctrl combos).
- `on:compositionstart/end` track IME sessions and send the final composed
  string at the end, avoiding partial pinyin/jamo/kana leaks.
- Move `on:paste` and `on:keydown` from `.terminal-screen` to the textarea so
  focus stays on the IME target.
- Replace `.terminal-screen:focus` with `:focus-within` since the screen div
  no longer holds focus.
- Add `is_regular_text_input` helper and unit test.

Web-sys features extended with `InputEvent` and `CompositionEvent`.

Tests: 24/24 pass, clippy clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
zhangheng
2026-06-18 16:09:45 +08:00
parent 843e4e55ab
commit f76d341501
4 changed files with 252 additions and 45 deletions

View File

@@ -42,6 +42,7 @@ web-sys = { version = "0.3", default-features = false, features = [
"Clipboard",
"ClipboardEvent",
"CloseEvent",
"CompositionEvent",
"console",
"DataTransfer",
"Document",
@@ -52,6 +53,7 @@ web-sys = { version = "0.3", default-features = false, features = [
"Event",
"EventTarget",
"HtmlElement",
"InputEvent",
"KeyboardEvent",
"Location",
"MessageEvent",

View File

@@ -36,6 +36,7 @@
| 粘贴Ctrl-V / Ctrl-Shift-V / 右键菜单,含 bracketed paste | `component.rs::handle_paste` / `prepare_paste` |
| 文本选区 + 复制自建模型Ctrl-Shift-C / 选区存在时 Ctrl+C跨滚动 | `component.rs` 鼠标处理 / `core.rs::selection_text` |
| 鼠标上报vim/htop/tmux 鼠标SGR/Default/UTF-8 编码,滚轮) | `component.rs` 鼠标处理 / `encode_mouse_report` |
| 中文 / IME 输入(拼音/日文/韩文,隐藏 textarea + composition 事件) | `component.rs::handle_input` / `handle_composition_*` |
---
@@ -111,33 +112,45 @@
---
### P3 — 中文 / IME 输入
### P3 — 中文 / IME 输入 ✅ 已完成
**现状**`map_key_to_terminal_input` 只处理 `key.chars().count() == 1` 的单字符IME 组合输入(拼音候选)完全没接
**现状**已实现。拼音、日文、韩文等 IME 组合输入可以正常工作;普通英文/数字输入也迁到了 `textarea``input` 事件,确保和 IME 走同一条路径
**计划实现**
1. 改用 **`composition` 事件 + 隐藏 `<textarea>`** 的标准方案xterm.js 也是这套):
- 一个 1×1 透明、跟随光标的 `textarea` 承接 IME
- 监听 `compositionstart` / `compositionupdate` / `compositionend`
- 组合进行中不发字节;`compositionend` 时把最终文本一次性作为 Input 发送。
- 普通 `input` 事件(非组合)也走这个 textarea替代部分 `keydown` 逐字符逻辑
2. 光标定位:把隐藏 textarea 定位到终端光标处,让候选框出现在正确位置
**实现**
1. **隐藏 `<textarea>` IME 载体**`component.rs` view
- 绝对定位、透明、无边框、大小跟随终端光标单元格
- 通过 `tabindex="0"` 成为焦点目标;`terminal-screen` 改用 `:focus-within` 显示焦点环
2. **事件模型迁移**
- `on:input`:普通字符/数字/空格输入,读取 textarea value 后清空并发送
- `on:compositionstart` / `on:compositionend`IME 组合期间不发字节,`compositionend` 时发送最终文本
- `on:keydown`仍处理方向键、Enter、Esc、Backspace、F 键、Ctrl 组合等**非打印键**。
- `on:paste`:从 `terminal-screen` 移到 textarea确保 focus 在 textarea 时右键/菜单粘贴也能触发。
3. **焦点管理**
- 点击 terminal-screen、WebSocket 连接成功、窗口 resize 后都聚焦 IME textarea。
4. **测试**:新增 `is_regular_text_input_detects_printable_keys` 单测,确保打印字符被正确交给 textarea24 个 terminal 测试全过clippy 干净。
**风险**中高。事件模型要从「keydown 逐键」迁一部分到「textarea + composition」需保证不回归现有按键。建议**先把 P1 功能键做完再动这块**,避免两套输入逻辑互相打架。
**已知限制**
- 主屏幕 scrollback 滚动后光标可能位于可见区域下方IME 候选框位置会对应到光标逻辑位置可能偏下。IME 主要用于当前输入, scrolled-back 场景输入较少见。
- 目前不支持 textarea 自动滚动延伸选区(和 P0 同一限制)。
---
## 3. 其余缺口(暂不排期,记录备查
### P4 — Canvas / WebGL 渲染器(性能目标
xterm.js 还有这些,本项目暂列「按需再做」:
**现状**DOM 渲染。常规 80×24160×40 终端流畅但大终端≥200×100或高刷新率连续输出时DOM 节点多、重排重绘开销大。
- **链接检测**(点 URL 打开)、**搜索 / 查找**Ctrl-F 高亮)
- **光标样式**:块 / 竖线 / 下划线 + 闪烁(目前固定块状反色)
- **Canvas / WebGL 渲染器**:当前是 DOM 渲染,够用但不如 Canvas 快;超大终端 + 高刷新率场景才需要 → **已纳入长期性能目标P3 后推进**
- **sixel / 图片协议**、**字形连字控制**
- **Unicode 11/15 宽度表**:目前依赖 vt100 的宽度判断,极端 emoji / 新字符可能不准
- **无障碍**(屏幕阅读器 live region
- **公开 API / 插件系统 / 配置项**:当前是项目内定制组件,无对外 API
**计划实现**
1. **Canvas 2D 渲染器(先做)**
- 一屏单元格一次 `fillText` / `fillRect` 绘制,减少 DOM 节点。
- 保留现有 `TerminalCore` 数据模型,渲染层换成 Canvas。
- 文本选区用 Canvas 半透明矩形;光标用 Canvas 反色或额外 DOM overlay。
2. **WebGL 渲染器(后做)**
- 把字符集预渲染到 texture atlas用 shader 批量绘制。
- 适合超大终端 + 高刷新率场景,但实现复杂度高。
3. **可切换**
- 默认 DOM提供 prop/api 切到 Canvas/WebGL。
**风险**:高。需要重新实现渲染层、文本测量、选区高亮、滚动逻辑,同时保证和现有 `TerminalCore` 数据模型兼容。
---

View File

@@ -7,9 +7,9 @@ use wasm_bindgen::JsCast;
use wasm_bindgen::closure::Closure;
use web_sys::js_sys::{Array, Function};
use web_sys::{
BinaryType, ClipboardEvent, ErrorEvent, Event, HtmlElement, KeyboardEvent,
MessageEvent, MouseEvent, ResizeObserver, ResizeObserverEntry, WebSocket,
WheelEvent,
BinaryType, ClipboardEvent, CompositionEvent, ErrorEvent, Event, HtmlElement,
InputEvent, KeyboardEvent, MessageEvent, MouseEvent, ResizeObserver,
ResizeObserverEntry, WebSocket, WheelEvent,
};
use super::core::{
@@ -87,6 +87,7 @@ impl Default for TerminalViewport {
#[component]
pub fn TerminalPanel() -> impl IntoView {
let terminal_ref = NodeRef::<html::Div>::new();
let ime_ref = NodeRef::<html::Textarea>::new();
let measure_ref = NodeRef::<html::Span>::new();
let connection_status = RwSignal::new("Connecting".to_owned());
let ws_signal = RwSignal::new(None::<WebSocket>);
@@ -121,6 +122,10 @@ pub fn TerminalPanel() -> impl IntoView {
let selection = RwSignal::new(None::<Selection>);
let selecting = RwSignal::new(false);
// IME composition state: when true, regular `input` events are ignored and
// the final composed string is sent on `compositionend`.
let composing = RwSignal::new(false);
// Mouse-reporting bookkeeping (only used while an app has enabled mouse
// reporting). `mouse_held_button` remembers which button is down so we can
// emit motion events and SGR-style releases; `mouse_last_cell` throttles
@@ -130,6 +135,33 @@ pub fn TerminalPanel() -> impl IntoView {
let mouse_last_cell: Rc<std::cell::Cell<Option<(usize, usize)>>> =
Rc::new(std::cell::Cell::new(None));
// -- IME cursor follower -------------------------------------------------
//
// The hidden IME textarea's position is bound reactively to the terminal
// cursor cell so the OS/browser IME candidate window appears in the right
// place. See the `style` prop on the textarea in the view below.
let ime_style = move || {
let _ = render_tick.get();
let Some((row, col)) = core_signal
.try_with_untracked(|core| {
if core.hide_cursor() {
None
} else {
Some(core.cursor_position())
}
})
.flatten()
else {
return String::new();
};
let (cell_w, cell_h) = cell_size.get();
let left = TERMINAL_PADDING + f64::from(col) * cell_w;
let top = TERMINAL_PADDING + f64::from(row) * cell_h;
format!(
"left:{left}px;top:{top}px;width:{cell_w}px;height:{cell_h}px;"
)
};
// -- WebSocket lifecycle ------------------------------------------------
Effect::new(move |_| {
@@ -148,6 +180,7 @@ pub fn TerminalPanel() -> impl IntoView {
let open_ws = ws_signal;
let open_viewport = current_viewport;
let open_ref = terminal_ref;
let open_ime = ime_ref;
let open_measure = measure_ref;
let open_mode = view_mode;
let open_prog_scroll = programmatic_scroll_top;
@@ -168,7 +201,7 @@ pub fn TerminalPanel() -> impl IntoView {
send_resize_message(&open_socket, viewport);
sync_scroll_to_bottom(open_ref, open_prog_scroll);
focus_terminal(open_ref);
focus_ime_textarea(&open_ime);
});
socket.set_onopen(Some(on_open.as_ref().unchecked_ref()));
on_open.forget();
@@ -273,6 +306,7 @@ pub fn TerminalPanel() -> impl IntoView {
let resize_core = core_signal;
let resize_st = scroll_top;
let resize_tick = render_tick;
let resize_ime = ime_ref;
let callback = Closure::<dyn FnMut(Array, ResizeObserver)>::new(
move |entries: Array, _observer: ResizeObserver| {
let Some(entry) = entries
@@ -314,7 +348,7 @@ pub fn TerminalPanel() -> impl IntoView {
send_resize_message(&resize_socket, viewport);
}
focus_terminal(resize_ref);
focus_ime_textarea(&resize_ime);
},
);
@@ -393,6 +427,19 @@ pub fn TerminalPanel() -> impl IntoView {
return;
}
// Printable characters are handled by the IME textarea's `input` /
// `composition` events so that IME composition and dead keys work.
// Only special keys (arrows, Enter, Esc, Backspace, F-keys,
// Ctrl-combos, …) are processed here.
if is_regular_text_input(
&event.key(),
event.ctrl_key(),
event.alt_key(),
event.meta_key(),
) {
return;
}
// The terminal owns bare Ctrl+letter (no Shift/Alt/Meta) so the
// browser doesn't intercept common shortcuts (Ctrl-W close tab,
// Ctrl-T new tab, Ctrl-N new window, Ctrl-R reload, Ctrl-F find,
@@ -643,9 +690,6 @@ pub fn TerminalPanel() -> impl IntoView {
let paste_core = core_signal;
move |event: ClipboardEvent| {
let Some(socket) = ws_signal.get_untracked() else {
return;
};
let Some(raw) = event
.clipboard_data()
.and_then(|dt| dt.get_data("text").ok())
@@ -660,18 +704,78 @@ pub fn TerminalPanel() -> impl IntoView {
.try_with_untracked(|core| core.bracketed_paste())
== Some(true);
let data = prepare_paste(&raw, bracketed);
// Pasting is an interaction at the prompt → return to live view.
if !paste_mode.get_untracked().is_live() {
paste_mode.set(ViewMode::Live);
sync_scroll_to_bottom(terminal_ref, paste_prog_scroll);
}
event.prevent_default();
send_terminal_message(
&socket,
&ClientTerminalMessage::Input { data },
send_input_data(
data,
ws_signal,
paste_mode,
terminal_ref,
paste_prog_scroll,
);
event.prevent_default();
}
};
// -- IME input ------------------------------------------------------------
//
// Regular characters and IME composition are handled through a hidden
// textarea that sits on top of the terminal cursor. Special keys
// (arrows, Enter, Esc, F-keys, Ctrl-combos) still go through keydown.
let handle_input = {
let input_mode = view_mode;
let input_prog_scroll = programmatic_scroll_top;
let input_composing = composing;
let input_ime = ime_ref;
move |event: Event| {
let Ok(_event) = event.dyn_into::<InputEvent>() else {
return;
};
if input_composing.get_untracked() {
return;
}
let Some(text) = ime_ref_text(&input_ime) else {
return;
};
if text.is_empty() {
return;
}
clear_ime_textarea(&input_ime);
send_input_data(
text,
ws_signal,
input_mode,
terminal_ref,
input_prog_scroll,
);
}
};
let handle_composition_start = move |_: CompositionEvent| {
composing.set(true);
};
let handle_composition_end = {
let end_mode = view_mode;
let end_prog_scroll = programmatic_scroll_top;
let end_composing = composing;
let end_ime = ime_ref;
move |event: CompositionEvent| {
if !end_composing.get_untracked() {
clear_ime_textarea(&end_ime);
return;
}
end_composing.set(false);
if let Some(text) = event.data()
&& !text.is_empty()
{
send_input_data(
text,
ws_signal,
end_mode,
terminal_ref,
end_prog_scroll,
);
}
clear_ime_textarea(&end_ime);
}
};
@@ -909,7 +1013,7 @@ pub fn TerminalPanel() -> impl IntoView {
};
let handle_click = move |_| {
focus_terminal(terminal_ref);
focus_ime_textarea(&ime_ref);
};
// -- View ---------------------------------------------------------------
@@ -954,16 +1058,27 @@ pub fn TerminalPanel() -> impl IntoView {
<div
node_ref=terminal_ref
class="terminal-screen"
tabindex="0"
on:click=handle_click
on:keydown=handle_keydown
on:scroll=handle_scroll
on:wheel=handle_wheel
on:paste=handle_paste
on:mousedown=handle_mousedown
on:mousemove=handle_mousemove
on:mouseup=handle_mouseup
>
<textarea
node_ref=ime_ref
class="terminal-ime"
tabindex="0"
autocomplete="off"
autocapitalize="off"
spellcheck="false"
style=ime_style
on:keydown=handle_keydown
on:input=handle_input
on:compositionstart=handle_composition_start
on:compositionend=handle_composition_end
on:paste=handle_paste
></textarea>
<span node_ref=measure_ref class="terminal-measure" aria-hidden="true">
{MEASURE_SAMPLE_TEXT}
</span>
@@ -1402,8 +1517,9 @@ fn point_to_cell(
)
}
fn focus_terminal(terminal_ref: NodeRef<html::Div>) {
if let Some(element) = terminal_ref.get_untracked() {
/// Focus the hidden IME textarea so keyboard input and composition go there.
fn focus_ime_textarea(ime_ref: &NodeRef<html::Textarea>) {
if let Some(element) = ime_ref.get_untracked() {
let _ = element.focus();
}
}
@@ -1430,6 +1546,42 @@ fn copy_current_selection(selection: RwSignal<Option<Selection>>, core: RwSignal
}
}
/// Send a string as terminal input, scrolling to the bottom if needed.
fn send_input_data(
data: String,
socket_signal: RwSignal<Option<WebSocket>>,
mode_signal: RwSignal<ViewMode>,
terminal_ref: NodeRef<html::Div>,
prog_scroll: RwSignal<i32>,
) {
if data.is_empty() {
return;
}
let Some(socket) = socket_signal.get_untracked() else {
return;
};
if !mode_signal.get_untracked().is_live() {
mode_signal.set(ViewMode::Live);
sync_scroll_to_bottom(terminal_ref, prog_scroll);
}
send_terminal_message(
&socket,
&ClientTerminalMessage::Input { data },
);
}
/// Read the current value of the hidden IME textarea.
fn ime_ref_text(ime_ref: &NodeRef<html::Textarea>) -> Option<String> {
ime_ref.get_untracked().map(|t| t.value())
}
/// Clear the hidden IME textarea after sending composed/typed text.
fn clear_ime_textarea(ime_ref: &NodeRef<html::Textarea>) {
if let Some(t) = ime_ref.get_untracked() {
t.set_value("");
}
}
// ---------------------------------------------------------------------------
// Key mapping
// ---------------------------------------------------------------------------
@@ -1668,6 +1820,14 @@ fn map_key_to_terminal_input(
)
}
/// Whether a keydown event represents a regular printable character that
/// should be handled by the IME textarea's `input` event, rather than sent
/// directly from `keydown`. This lets the browser manage IME composition,
/// dead keys, and upper/lower case before we see the final text.
fn is_regular_text_input(key: &str, ctrl: bool, alt: bool, meta: bool) -> bool {
key.chars().count() == 1 && !ctrl && !alt && !meta
}
#[cfg(test)]
mod tests {
use super::*;
@@ -1854,6 +2014,19 @@ mod tests {
assert!(selection_rects(None, 0, 30, cols, cw).is_empty());
}
#[test]
fn is_regular_text_input_detects_printable_keys() {
assert!(is_regular_text_input("a", false, false, false));
assert!(is_regular_text_input("A", false, false, false)); // Shift handled by browser
assert!(is_regular_text_input(" ", false, false, false));
assert!(!is_regular_text_input("Enter", false, false, false));
assert!(!is_regular_text_input("ArrowUp", false, false, false));
assert!(!is_regular_text_input("a", true, false, false)); // Ctrl+a
assert!(!is_regular_text_input("a", false, true, false)); // Alt+a
assert!(!is_regular_text_input("a", false, false, true)); // Meta+a
assert!(!is_regular_text_input("Dead", false, false, false));
}
#[test]
fn key_to_bytes_table() {
// --- basic printable / control keys ---

View File

@@ -610,13 +610,32 @@
scrollbar-color: rgb(148 163 184 / 60%) transparent;
}
.terminal-screen:focus {
.terminal-screen:focus-within {
border-color: #38bdf8;
box-shadow:
inset 0 1px 0 rgb(255 255 255 / 4%),
0 0 0 3px rgb(56 189 248 / 15%);
}
.terminal-ime {
position: absolute;
z-index: 100;
opacity: 0;
background: transparent;
color: transparent;
caret-color: transparent;
border: none;
outline: none;
resize: none;
padding: 0;
margin: 0;
overflow: hidden;
pointer-events: auto;
font-family: inherit;
font-size: 1px;
line-height: 1px;
}
.terminal-text {
margin: 0;
min-height: 100%;