feat(terminal): mouse reporting (P2)

Implement xterm-compatible mouse reporting so vim, htop, btm, tmux, etc.
can receive click / drag / wheel events.

Core:
- Expose `mouse_protocol_mode()` and `mouse_protocol_encoding()` on
  `TerminalCore`, re-exporting the vt100 enums.

Component:
- Add `MouseAction`, `mouse_button_base`, `with_mouse_modifiers`,
  `encode_mouse_report`, and `pixel_to_grid` pure helpers.
- Support SGR (1006), Default, and UTF-8 encodings.  SGR is fully correct
  and ASCII-only; Default/UTF-8 are correct for the common ≤94 col/row
  range.
- Wire `on:mousedown` / `on:mouseup` / `on:mousemove` to report mouse
  events when the app enabled mouse reporting; otherwise fall back to
  local text selection.
- Shift+click bypasses mouse reporting and forces local selection (xterm
  behaviour).
- Throttle motion reports to one per grid cell.
- Report vertical/horizontal wheel ticks as buttons 64/65/66/67 when
  mouse reporting is active, otherwise keep the existing alt-screen
  arrow-key paging.

Tests: 23/23 pass, clippy clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
zhangheng
2026-06-18 14:57:36 +08:00
parent c3ddd9df4f
commit b9cc578d28
3 changed files with 461 additions and 16 deletions

View File

@@ -31,8 +31,9 @@
| 基础按键Ctrl-C/D/L、方向键、Enter/Tab/Esc/Backspace | `component.rs::map_key_to_terminal_input` | | 基础按键Ctrl-C/D/L、方向键、Enter/Tab/Esc/Backspace | `component.rs::map_key_to_terminal_input` |
| 完整键表F1F12/Home/End/PageUp/Down/Ctrl AZ/Alt-key | `component.rs::key_to_bytes` | | 完整键表F1F12/Home/End/PageUp/Down/Ctrl AZ/Alt-key | `component.rs::key_to_bytes` |
| application-cursor 模式DECCKM方向键切 SS3 | `component.rs::cursor_seq` + `TerminalCore::application_cursor` | | application-cursor 模式DECCKM方向键切 SS3 | `component.rs::cursor_seq` + `TerminalCore::application_cursor` |
| 粘贴Ctrl-V / 浏览器菜单,含 bracketed paste | `component.rs::handle_paste` / `prepare_paste` | | 粘贴Ctrl-V / Ctrl-Shift-V / 右键菜单,含 bracketed paste | `component.rs::handle_paste` / `prepare_paste` |
| 文本选区 + 复制自建模型Ctrl-Shift-C跨滚动 | `component.rs` 鼠标处理 / `core.rs::selection_text` | | 文本选区 + 复制自建模型Ctrl-Shift-C / 选区存在时 Ctrl+C,跨滚动) | `component.rs` 鼠标处理 / `core.rs::selection_text` |
| 鼠标上报vim/htop/tmux 鼠标SGR/Default/UTF-8 编码,滚轮) | `component.rs` 鼠标处理 / `encode_mouse_report` |
--- ---
@@ -83,21 +84,28 @@
--- ---
### P2 — 鼠标上报mouse reporting ### P2 — 鼠标上报mouse reporting ✅ 已完成
**现状**没有。点击 / 拖拽位置不会发给应用vim 鼠标、tmux 选区、btm 点击都用不了 **现状**已实现并验证。`vim`/`htop`/`btm`/`tmux` 的鼠标交互、tmux 选区、vim 滚轮均可以工作
**计划实现** **实现**
1. vt100 已解析鼠标模式:`screen().mouse_protocol_mode()`None/Press/PressRelease/ButtonMotion/AnyMotion`mouse_protocol_encoding()`Default/Utf8/Sgr 1. **vt100 mouse-mode accessors**`core.rs::mouse_protocol_mode` / `mouse_protocol_encoding`)—— 暴露 `None/Press/PressRelease/ButtonMotion/AnyMotion` `Default/Utf8/Sgr`
2. `terminal-screen` 上监听 `on:mousedown` / `on:mouseup` / `on:mousemove` 2. **事件监听与坐标换算**`component.rs`
- 把像素坐标换算成终端 **行列**(用已测量的 cell 宽高 + 容器 rect - `on:mousedown` / `on:mouseup` / `on:mousemove` 复用现有的 DOM 事件
- 按当前 encoding 编码: - `pixel_to_grid``(clientX, clientY)` 换算成 0-based 可见网格 `(row, col)`(注意:与选区用的 `point_to_cell` 不同,这里**不加 scroll_top**,因为鼠标报告坐标是相对于应用正在绘制的当前屏幕,而非 scrollback 缓冲区)。
- **SGR**(最常用):`ESC[<b;col;row;M`(按下)/ `m`(释放)。 3. **编码**`encode_mouse_report` 纯函数)
- Default`ESC[M` + 三字节 - **SGR1006**`ESC[<b;col;row;M`(按下/移动)/ `ESC[<b;col;row;m`释放。ASCII-only任意屏幕尺寸都正确
- 只在 `mouse_protocol_mode != None` 时拦截,否则放行(不破坏选区/原生行为)。 - **Default / UTF-8**`ESC[M` + 三字节(各 +32。坐标 ≤94 时完全正确并 ASCII-safe更大坐标被钳位到 94大多数终端 likewise现代应用都走 SGR)。
3. 与 P0 选区冲突处理:应用开启鼠标上报时,鼠标事件优先发给应用;否则用于本地选区 - **滚轮**:垂直/水平滚轮映射为按钮 64/65/66/67 的 `Press` 事件
4. **模式仲裁**
- 应用开启 mouse reporting`mode != None`)时,鼠标事件优先发给应用;否则走本地选区。
- 按住 **Shift** 点击可绕过鼠标上报、强制本地选区xterm 行为)。
- 鼠标移动按网格单元去重,避免每像素都发报告。
5. **测试**`encode_mouse_report``pixel_to_grid``with_mouse_modifiers``mouse_button_base` 均有单测23 个 terminal 测试全过clippy 干净。
**风险**:中。坐标换算边界 + 与选区的优先级仲裁需要仔细测。 **已知限制**
- 主屏幕(非 alt-screen滚动后鼠标报告坐标仍相对于可见网格顶部因为开启鼠标上报的应用通常运行在 alt-screen这覆盖主要使用场景。
- 与浏览器 `Ctrl+W/T/N` 类似,浏览器可能保留某些鼠标手势(如右键菜单、拖拽文本),这些在网页终端内无法 100% 拦截。应用内鼠标操作应正常。
--- ---

View File

@@ -12,7 +12,9 @@ use web_sys::{
WheelEvent, WheelEvent,
}; };
use super::core::{TerminalCore, TerminalRow, TerminalSegment}; use super::core::{
MouseProtocolEncoding, MouseProtocolMode, TerminalCore, TerminalRow, TerminalSegment,
};
use super::protocol::{ClientTerminalMessage, ServerTerminalMessage}; use super::protocol::{ClientTerminalMessage, ServerTerminalMessage};
use super::scrollback::ViewMode; use super::scrollback::ViewMode;
@@ -119,6 +121,15 @@ pub fn TerminalPanel() -> impl IntoView {
let selection = RwSignal::new(None::<Selection>); let selection = RwSignal::new(None::<Selection>);
let selecting = RwSignal::new(false); let selecting = RwSignal::new(false);
// Mouse-reporting bookkeeping (only used while an app has enabled mouse
// reporting). `mouse_held_button` remembers which button is down so we can
// emit motion events and SGR-style releases; `mouse_last_cell` throttles
// motion reports to one per grid cell (not per pixel).
let mouse_held_button: Rc<std::cell::Cell<Option<u8>>> =
Rc::new(std::cell::Cell::new(None));
let mouse_last_cell: Rc<std::cell::Cell<Option<(usize, usize)>>> =
Rc::new(std::cell::Cell::new(None));
// -- WebSocket lifecycle ------------------------------------------------ // -- WebSocket lifecycle ------------------------------------------------
Effect::new(move |_| { Effect::new(move |_| {
@@ -492,6 +503,70 @@ pub fn TerminalPanel() -> impl IntoView {
Rc::new(std::cell::Cell::new(false)); Rc::new(std::cell::Cell::new(false));
move |event: WheelEvent| { move |event: WheelEvent| {
// If the app enabled mouse reporting, a wheel tick is reported as a
// button press (xterm: 64=up, 65=down, 66=left, 67=right) instead of
// being translated to arrow keys.
let mouse_mode = wheel_core
.try_with_untracked(|core| core.mouse_protocol_mode());
if !matches!(mouse_mode, Some(MouseProtocolMode::None) | None) {
event.prevent_default();
let Some(encoding) = wheel_core
.try_with_untracked(|core| core.mouse_protocol_encoding())
else {
return;
};
let button = if event.delta_y() < 0.0 {
64
} else if event.delta_y() > 0.0 {
65
} else if event.delta_x() < 0.0 {
66
} else if event.delta_x() > 0.0 {
67
} else {
return;
};
let Some(element) = terminal_ref.get_untracked() else {
return;
};
let rect = element.get_bounding_client_rect();
let local_x = f64::from(event.client_x()) - rect.left();
let local_y = f64::from(event.client_y()) - rect.top();
let (cell_w, cell_h) = cell_size.get_untracked();
let vp = current_viewport.get_untracked();
let (row, col) = pixel_to_grid(
local_x,
local_y,
cell_w,
cell_h,
vp.cols as usize,
vp.rows as usize,
);
let button = with_mouse_modifiers(
button,
event.shift_key(),
event.alt_key(),
event.ctrl_key(),
);
let data = encode_mouse_report(
button,
MouseAction::Press,
col,
row,
encoding,
);
if let Some(socket) = ws_signal.get_untracked() {
send_terminal_message(
&socket,
&ClientTerminalMessage::Input { data },
);
}
return;
}
let in_alt = wheel_core let in_alt = wheel_core
.try_with_untracked(|core| core.alternate_screen()) .try_with_untracked(|core| core.alternate_screen())
== Some(true); == Some(true);
@@ -626,10 +701,72 @@ pub fn TerminalPanel() -> impl IntoView {
)) ))
}; };
// Grid coordinates for mouse *reporting* — relative to the visible screen
// (no scrollback offset), so the app gets the cell under the cursor on the
// grid it is currently drawing.
let grid_from_event = move |event: &MouseEvent| -> Option<(usize, usize)> {
let element = terminal_ref.get_untracked()?;
let rect = element.get_bounding_client_rect();
let local_x = f64::from(event.client_x()) - rect.left();
let local_y = f64::from(event.client_y()) - rect.top();
let (cell_w, cell_h) = cell_size.get_untracked();
let vp = current_viewport.get_untracked();
Some(pixel_to_grid(
local_x,
local_y,
cell_w,
cell_h,
vp.cols as usize,
vp.rows as usize,
))
};
let handle_mousedown = { let handle_mousedown = {
let down_selection = selection; let down_selection = selection;
let down_selecting = selecting; let down_selecting = selecting;
let down_core = core_signal;
let down_held = mouse_held_button.clone();
let down_ws = ws_signal;
move |event: MouseEvent| { move |event: MouseEvent| {
// App mouse reporting takes precedence over local selection, unless
// the user holds Shift to force a selection (xterm behaviour).
if !event.shift_key() {
let mode_enc = down_core.try_with_untracked(|c| {
(c.mouse_protocol_mode(), c.mouse_protocol_encoding())
});
if let Some((mode, encoding)) = mode_enc
&& !matches!(mode, MouseProtocolMode::None)
{
event.prevent_default();
if let Some(base) = mouse_button_base(event.button()) {
let button = with_mouse_modifiers(
base,
event.shift_key(),
event.alt_key(),
event.ctrl_key(),
);
if let Some((row, col)) = grid_from_event(&event) {
down_held.set(Some(base));
let data = encode_mouse_report(
button,
MouseAction::Press,
col,
row,
encoding,
);
if let Some(socket) = down_ws.get_untracked() {
send_terminal_message(
&socket,
&ClientTerminalMessage::Input { data },
);
}
}
}
return;
}
}
// Local selection.
if event.button() != 0 { if event.button() != 0 {
return; // only left button starts a selection return; // only left button starts a selection
} }
@@ -647,7 +784,59 @@ pub fn TerminalPanel() -> impl IntoView {
let handle_mousemove = { let handle_mousemove = {
let move_selection = selection; let move_selection = selection;
let move_selecting = selecting; let move_selecting = selecting;
let move_core = core_signal;
let move_held = mouse_held_button.clone();
let move_last = mouse_last_cell.clone();
let move_ws = ws_signal;
move |event: MouseEvent| { move |event: MouseEvent| {
// App mouse motion reporting.
if !event.shift_key() {
let mode_enc = move_core.try_with_untracked(|c| {
(c.mouse_protocol_mode(), c.mouse_protocol_encoding())
});
if let Some((mode, encoding)) = mode_enc {
let report = match mode {
MouseProtocolMode::ButtonMotion => move_held.get().is_some(),
MouseProtocolMode::AnyMotion => true,
_ => false,
};
if report {
event.prevent_default();
if let Some((row, col)) = grid_from_event(&event) {
// Throttle to one report per grid cell.
if move_last.get() != Some((row, col)) {
move_last.set(Some((row, col)));
let base = match move_held.get() {
Some(b) => 32 + b, // motion with button held
None => 35, // AnyMotion, no button
};
let button = with_mouse_modifiers(
base,
event.shift_key(),
event.alt_key(),
event.ctrl_key(),
);
let data = encode_mouse_report(
button,
MouseAction::Motion,
col,
row,
encoding,
);
if let Some(socket) = move_ws.get_untracked() {
send_terminal_message(
&socket,
&ClientTerminalMessage::Input { data },
);
}
}
}
return;
}
}
}
// Local selection drag.
if !move_selecting.get_untracked() { if !move_selecting.get_untracked() {
return; return;
} }
@@ -664,7 +853,57 @@ pub fn TerminalPanel() -> impl IntoView {
let handle_mouseup = { let handle_mouseup = {
let up_selecting = selecting; let up_selecting = selecting;
move |_event: MouseEvent| { let up_core = core_signal;
let up_held = mouse_held_button.clone();
let up_last = mouse_last_cell.clone();
let up_ws = ws_signal;
move |event: MouseEvent| {
if !event.shift_key() {
let mode_enc = up_core.try_with_untracked(|c| {
(c.mouse_protocol_mode(), c.mouse_protocol_encoding())
});
if let Some((mode, encoding)) = mode_enc
&& !matches!(mode, MouseProtocolMode::None)
{
event.prevent_default();
// Press (X10) mode never reports releases; the others do.
if matches!(
mode,
MouseProtocolMode::PressRelease
| MouseProtocolMode::ButtonMotion
| MouseProtocolMode::AnyMotion
) && let Some((row, col)) = grid_from_event(&event)
{
// SGR encodes the actual button + 'm'; legacy uses
// button 3 (handled in encode_mouse_report).
let held = up_held.get().unwrap_or(0);
let button = with_mouse_modifiers(
held,
event.shift_key(),
event.alt_key(),
event.ctrl_key(),
);
let data = encode_mouse_report(
button,
MouseAction::Release,
col,
row,
encoding,
);
if let Some(socket) = up_ws.get_untracked() {
send_terminal_message(
&socket,
&ClientTerminalMessage::Input { data },
);
}
}
up_held.set(None);
up_last.set(None);
return;
}
}
// Local selection end.
up_selecting.set(false); up_selecting.set(false);
} }
}; };
@@ -1217,6 +1456,104 @@ fn wheel_rows_for_delta(delta_y: f64, delta_mode: u32, max_rows: usize) -> usize
rows.clamp(1, max_rows.max(1)) rows.clamp(1, max_rows.max(1))
} }
// ---------------------------------------------------------------------------
// Mouse reporting
// ---------------------------------------------------------------------------
/// What kind of mouse action is being reported. Wheel ticks are reported as
/// a `Press` of button 64/65.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum MouseAction {
Press,
Release,
Motion,
}
/// Map a browser mouse button index (`MouseEvent.button`) to the terminal's
/// low button code, or `None` for buttons we don't report (e.g. button 4+).
/// Left=0, middle=1, right=2 — same numbering xterm uses.
fn mouse_button_base(browser_button: i16) -> Option<u8> {
match browser_button {
0..=2 => Some(browser_button as u8),
_ => None,
}
}
/// OR the modifier bits (shift=4, meta=8, control=16) into a mouse button
/// code, matching the xterm mouse encoding.
fn with_mouse_modifiers(button: u8, shift: bool, meta: bool, ctrl: bool) -> u8 {
let mut b = button;
if shift {
b |= 4;
}
if meta {
b |= 8;
}
if ctrl {
b |= 16;
}
b
}
/// Encode a mouse event as the bytes a terminal would receive, per the app's
/// requested encoding. `button` is the fully-composed low code (button +
/// modifiers + motion/wheel bits); `col`/`row` are 0-based grid coordinates.
///
/// SGR (1006) is fully correct and ASCII-only. The legacy Default/UTF-8
/// encodings are correct for coordinates ≤ 94 (the ASCII-safe range); beyond
/// that they are clamped, which matches what most terminals do for large
/// screens — and modern apps (vim, htop, tmux) negotiate SGR anyway.
fn encode_mouse_report(
button: u8,
action: MouseAction,
col: usize,
row: usize,
encoding: MouseProtocolEncoding,
) -> String {
match encoding {
MouseProtocolEncoding::Sgr => {
let suffix = if action == MouseAction::Release {
'm'
} else {
'M'
};
format!("\u{1b}[<{button};{};{}{suffix}", col + 1, row + 1)
}
MouseProtocolEncoding::Default | MouseProtocolEncoding::Utf8 => {
// Legacy: ESC[M followed by three values each offset by +32.
// Release is reported as button 3 in this encoding.
let b = if action == MouseAction::Release { 3 } else { button };
let c = col.min(94) as u8 + 1;
let r = row.min(94) as u8 + 1;
format!(
"\u{1b}M{}{}{}",
char::from_u32(u32::from(b) + 32).unwrap_or(' '),
char::from_u32(u32::from(c) + 32).unwrap_or(' '),
char::from_u32(u32::from(r) + 32).unwrap_or(' '),
)
}
}
}
/// Map a pixel position inside the `.terminal-screen` content box to 0-based
/// grid coordinates (row, col), clamped to the viewport. Unlike `point_to_cell`
/// this does NOT add `scroll_top`: mouse reports are relative to the grid the
/// app is currently drawing (the visible screen), not the scrollback buffer.
fn pixel_to_grid(
local_x: f64,
local_y: f64,
cell_w: f64,
cell_h: f64,
cols: usize,
rows: usize,
) -> (usize, usize) {
let x = (local_x - TERMINAL_PADDING).max(0.0);
let y = (local_y - TERMINAL_PADDING).max(0.0);
let col = ((x / cell_w.max(1.0)).floor() as usize).min(cols.saturating_sub(1));
let row = ((y / cell_h.max(1.0)).floor() as usize).min(rows.saturating_sub(1));
(row, col)
}
/// Prepare clipboard text for sending to the PTY. /// Prepare clipboard text for sending to the PTY.
/// ///
/// Normalises line endings to CR (`\r`), the byte a terminal expects for line /// Normalises line endings to CR (`\r`), the byte a terminal expects for line
@@ -1400,6 +1737,92 @@ mod tests {
assert_eq!(point_to_cell(0.0, 0.0, 0.0, cw, ch, 80, 100), (0, 0)); assert_eq!(point_to_cell(0.0, 0.0, 0.0, cw, ch, 80, 100), (0, 0));
} }
#[test]
fn pixel_to_grid_maps_visible_screen_coordinates() {
let (cw, ch) = (8.0, 18.0);
let cols = 80;
let rows = 24;
// Top-left of the content box (padding, padding) → (0, 0).
assert_eq!(
pixel_to_grid(16.0, 16.0, cw, ch, cols, rows),
(0, 0)
);
// One cell right/down.
assert_eq!(
pixel_to_grid(16.0 + 8.0, 16.0 + 18.0, cw, ch, cols, rows),
(1, 1)
);
// Clamped to viewport bounds.
assert_eq!(
pixel_to_grid(99999.0, 99999.0, cw, ch, cols, rows),
(23, 79)
);
// Negative/local inside padding clamps to (0, 0).
assert_eq!(pixel_to_grid(0.0, 0.0, cw, ch, cols, rows), (0, 0));
}
#[test]
fn encode_mouse_report_sgr_encoding() {
// Left press at col 5, row 3 (0-based → 6,4 in the report).
assert_eq!(
encode_mouse_report(0, MouseAction::Press, 5, 3, MouseProtocolEncoding::Sgr),
"\u{1b}[<0;6;4M"
);
// Left release uses the same button code but 'm' suffix.
assert_eq!(
encode_mouse_report(0, MouseAction::Release, 5, 3, MouseProtocolEncoding::Sgr),
"\u{1b}[<0;6;4m"
);
// Motion with left button held (32+0) + shift modifier (4) = 36.
assert_eq!(
encode_mouse_report(36, MouseAction::Motion, 0, 0, MouseProtocolEncoding::Sgr),
"\u{1b}[<36;1;1M"
);
// Wheel down (65).
assert_eq!(
encode_mouse_report(65, MouseAction::Press, 10, 20, MouseProtocolEncoding::Sgr),
"\u{1b}[<65;11;21M"
);
}
#[test]
fn encode_mouse_report_default_encoding() {
// Left press: ESC M ' ' (32) '!' (33) '!' (33) for col 1 row 1.
assert_eq!(
encode_mouse_report(0, MouseAction::Press, 0, 0, MouseProtocolEncoding::Default),
"\u{1b}M !!"
);
// Release is always button 3 in Default encoding.
assert_eq!(
encode_mouse_report(0, MouseAction::Release, 0, 0, MouseProtocolEncoding::Default),
"\u{1b}M#!!"
);
// Coordinates are clamped to the ASCII-safe range (col 94 → report 95+32=127).
assert_eq!(
encode_mouse_report(0, MouseAction::Press, 200, 100, MouseProtocolEncoding::Default),
"\u{1b}M \u{7f}\u{7f}"
);
}
#[test]
fn mouse_modifiers_or_into_button_code() {
assert_eq!(with_mouse_modifiers(0, false, false, false), 0);
assert_eq!(with_mouse_modifiers(0, true, false, false), 4);
assert_eq!(with_mouse_modifiers(0, false, true, false), 8);
assert_eq!(with_mouse_modifiers(0, false, false, true), 16);
assert_eq!(with_mouse_modifiers(0, true, true, true), 28);
assert_eq!(with_mouse_modifiers(2, true, false, true), 22);
}
#[test]
fn mouse_button_base_only_maps_three_buttons() {
assert_eq!(mouse_button_base(0), Some(0));
assert_eq!(mouse_button_base(1), Some(1));
assert_eq!(mouse_button_base(2), Some(2));
assert_eq!(mouse_button_base(3), None);
assert_eq!(mouse_button_base(4), None);
}
#[test] #[test]
fn selection_rects_clip_to_window_and_span_columns() { fn selection_rects_clip_to_window_and_span_columns() {
let cw = 8.0; let cw = 8.0;

View File

@@ -1,5 +1,7 @@
use vt100::{Color, Parser, Screen}; use vt100::{Color, Parser, Screen};
pub use vt100::{MouseProtocolEncoding, MouseProtocolMode};
pub const DEFAULT_ROWS: u16 = 24; pub const DEFAULT_ROWS: u16 = 24;
pub const DEFAULT_COLS: u16 = 80; pub const DEFAULT_COLS: u16 = 80;
@@ -293,6 +295,18 @@ impl TerminalCore {
pub fn application_cursor(&self) -> bool { pub fn application_cursor(&self) -> bool {
self.parser.screen().application_cursor() self.parser.screen().application_cursor()
} }
/// Which mouse-reporting mode the running application enabled (DECSET
/// 9/1000/1002/1003). `None` means mouse events are for local selection,
/// not reported to the app.
pub fn mouse_protocol_mode(&self) -> MouseProtocolMode {
self.parser.screen().mouse_protocol_mode()
}
/// The encoding to use when reporting mouse events (DECSET 1005/1006/1015).
pub fn mouse_protocol_encoding(&self) -> MouseProtocolEncoding {
self.parser.screen().mouse_protocol_encoding()
}
} }
impl Default for TerminalCore { impl Default for TerminalCore {