feat: add voice transcription feedback workflow

This commit is contained in:
2026-07-20 10:49:49 +08:00
parent 0a07538dce
commit bed80aa807
17 changed files with 1691 additions and 19 deletions

2
.gitignore vendored
View File

@@ -3,6 +3,8 @@ graphify-out/
# Dependencies and build output # Dependencies and build output
node_modules/ node_modules/
.venv/
.models/
miniprogram_npm/ miniprogram_npm/
dist/ dist/
coverage/ coverage/

310
DEBUG_GUIDE.md Normal file
View File

@@ -0,0 +1,310 @@
# 本地调试指南
本文用于在本机启动 PostgreSQL 连接、中文 FunASR、Rust API 和微信小程序,并验证短录音、长录音及多课节汇总。
## 1. 服务关系
```text
微信小程序
-> Rust API127.0.0.1:8080
-> PostgreSQL
-> FunASR127.0.0.1:10095
```
本地录音转写的核心是 FunASR当前组合为
- Paraformer中文语音识别。
- FSMN-VAD检测并切分有效语音。
- CT-Transformer恢复中文标点。
- PyTorchCPU 推理运行时。
Rust API 负责音频保存、持久化任务队列、重试、课节状态和反馈汇总。小程序每 8 分钟自动切片,并每 3 秒无感刷新转写状态。
## 2. 当前主机资源实测
测试主机为 Apple Silicon、10 核 CPU、16 GB 统一内存,配置为 CPU 单并发:
| 项目 | 实测值 |
| --- | --- |
| FunASR 模型缓存 | 约 2.1 GB |
| Python 虚拟环境 | 约 871 MB |
| 模型加载后的物理内存 | 约 2.8 GB |
| 进程历史内存峰值 | 约 4.9 GB |
| 8 分钟、16 kHz、单声道、48 kbps MP3 | 约 44.8 秒完成 |
| 推理期间 CPU | 约 90% 至 132%,即约 1 至 1.3 个 CPU 核 |
8 分钟数据使用重复的清晰中文样本测得,真实课堂中的噪声、停顿和说话人数会影响耗时和识别质量。
容量建议:
- 当前单教师或低并发调试:不需要 GPU当前 16 GB 主机足够。
- CPU 部署最低建议4 vCPU、8 GB 内存8 GB 可能发生交换16 GB 更稳妥。
- 多教师并发:先保持单并发观察队列;若片段持续积压,再考虑增加 CPU 实例或 NVIDIA GPU。
- GPU 是可选优化不是运行前提。CUDA 部署需要单独的 CUDA/PyTorch 镜像,当前 CPU 镜像不能只修改 `ASR_DEVICE=cuda`
- GPU 部署建议从 8 GB 显存起步并用真实课堂录音压测。当前 Apple Silicon 配置没有使用 Apple GPU。
- `ASR_WORKER_CONCURRENCY``ASR_MAX_CONCURRENCY` 应保持一致;每提高一个并发都要重新观察内存和吞吐。
## 3. 首次准备
在项目根目录执行:
```bash
cd /Users/zhangheng/project/ai/miniprogram/teaching-feedback-assistant
```
确认 `server/.env` 存在,并填写可访问的 `DATABASE_URL`。不存在时再复制示例,避免覆盖现有配置:
```bash
test -f server/.env || cp server/.env.example server/.env
```
本地 FunASR 默认配置如下,无需腾讯云密钥:
```env
ASR_PROVIDER=local
LOCAL_ASR_URL=http://127.0.0.1:10095
ASR_REQUEST_TIMEOUT_SECONDS=1800
ASR_WORKER_CONCURRENCY=1
ASR_JOB_LEASE_SECONDS=3600
```
首次准备 Python 环境:
```bash
cd asr-service
python3 -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/python -m pip install -r requirements.txt
cd ..
```
模型首次启动会下载约 2.1 GB 到 `asr-service/.models``.venv``.models` 均已被 Git 和微信小程序打包忽略。
## 4. 启动全部服务
### 4.1 执行数据库迁移
每次拉取到新迁移后执行一次:
```bash
cd server
cargo run --bin migrate
cd ..
```
看到 `PostgreSQL migrations completed` 表示成功。
### 4.2 启动 FunASR
终端一:
```bash
cd asr-service
MODELSCOPE_CACHE=.models .venv/bin/uvicorn app:app \
--host 127.0.0.1 \
--port 10095 \
--no-access-log
```
另一个终端检查:
```bash
curl http://127.0.0.1:10095/health
```
预期结果:
```json
{"status":"ok","model":"paraformer-zh","device":"cpu"}
```
首次下载或缓存加载期间端口可能暂不可用,应等待模型完全加载。
Docker 方式可替代本机 Python
```bash
docker compose -f compose.asr.yml up --build -d
docker compose -f compose.asr.yml logs -f funasr
```
### 4.3 启动 Rust API
终端二:
```bash
cd server
cargo run
```
检查完整链路:
```bash
curl http://127.0.0.1:8080/health | jq .
```
正常状态应包含:
```json
{
"status": "ok",
"database_configured": true,
"speech_configured": true,
"speech_available": true,
"speech_provider": "local-funasr"
}
```
接口文档:<http://127.0.0.1:8080/scalar>
### 4.4 启动微信开发者工具
终端三,在项目根目录执行:
```bash
cli open \
--project /Users/zhangheng/project/ai/miniprogram/teaching-feedback-assistant \
--port 63097 \
--lang zh
```
自动预览编译检查:
```bash
cli auto-preview \
--project /Users/zhangheng/project/ai/miniprogram/teaching-feedback-assistant \
--port 63097 \
--lang zh \
--info-output /tmp/teaching-feedback-preview.json
```
开发者工具模拟器默认访问 `http://127.0.0.1:8080`。进入小程序“我的”页面,应看到“服务、数据库和语音转录正常”。
## 5. 录音测试流程
### 短录音
1. 打开“反馈生成”。
2. 点击“语音录入”,录制 10 至 30 秒中文。
3. 点击“结束录音”。
4. 页面短暂显示“正在处理”。
5. 转写完成后文字应自动加入反馈内容。
短录音条件为不超过 60 秒,且转写文本未超过后端短文本限制。
### 长录音
1. 录制超过 60 秒的中文。
2. 结束后等待状态变成“已就绪”。
3. 主按钮变为“生成反馈”。
4. 点击一次后,转写内容会汇总进反馈草稿。
### 多课节汇总
1. 录制并结束第一节课。
2. 再次点击“语音录入”录制下一节。
3. 重复完成多节录音。
4. 所有课节就绪后点击一次“生成反馈”。
单节录音超过 8 分钟时,小程序会自动结束当前片段并立即开始下一片段,用户无需操作。
## 6. 故障恢复测试
### FunASR 未启动
停止 FunASR 后录音Rust API 仍会保存音频,页面保持“正在处理”。重新启动 FunASR 后,后台队列会自动继续。
### Rust 在推理中退出
任务使用租约避免多个工作线程重复处理。默认租约为 3600 秒。调试崩溃恢复时,可在 `server/.env` 临时设置:
```env
ASR_JOB_LEASE_SECONDS=60
```
重新启动 Rust API 后,超出租约的任务会再次被领取。生产环境应使用足够覆盖最长片段推理时间的租约。
### 转写失败
点击页面中的语音状态,可查看课节状态并重试失败片段。音频保存在 `server/data/audio`,转写失败不会立即删除原文件。
## 7. 常见问题
### `speech_configured=true` 但 `speech_available=false`
- FunASR 尚未启动或模型仍在加载。
- `10095` 端口被占用。
- `LOCAL_ASR_URL` 配置错误。
检查:
```bash
lsof -nP -iTCP:10095 -sTCP:LISTEN
curl http://127.0.0.1:10095/health
```
### `database_configured=false` 或数据接口返回 503
检查 `server/.env` 中的 `DATABASE_URL`,然后重新执行迁移并启动 API。
### 微信模拟器无法连接 API
```bash
lsof -nP -iTCP:8080 -sTCP:LISTEN
curl http://127.0.0.1:8080/health
```
再到小程序“我的”页面确认 API 地址为 `http://127.0.0.1:8080`
### 真机无法访问 `127.0.0.1`
真机中的 `127.0.0.1` 指手机自身。局域网调试时:
1.`server/.env``HOST` 改为 `0.0.0.0`
2. 确认手机和电脑连接同一网络。
3. 在“我的”页面将 API 地址改为 `http://<电脑局域网IP>:8080`
4. 检查系统防火墙和微信开发者工具的域名校验设置。
正式环境必须使用微信允许的 HTTPS 合法域名。
## 8. 代码检查
小程序:
```bash
npm run typecheck
```
Rust
```bash
cd server
cargo fmt --check
cargo test
cargo clippy --all-targets -- -D warnings
```
FunASR 服务:
```bash
python3 -m py_compile asr-service/app.py
asr-service/.venv/bin/python -m pip check
```
## 9. 停止服务
前台运行时,在 FunASR 和 Rust API 各自终端按 `Ctrl+C`
Docker 方式:
```bash
docker compose -f compose.asr.yml down
```
确认端口已经释放:
```bash
lsof -nP -iTCP:8080 -sTCP:LISTEN
lsof -nP -iTCP:10095 -sTCP:LISTEN
```
正常停止服务不会删除 `asr-service/.models` 模型缓存。除非希望重新下载模型,否则不要删除该目录。

View File

@@ -11,6 +11,8 @@ cli open --project /Users/zhangheng/project/ai/miniprogram/teaching-feedback-ass
3. 在微信开发者工具内编译、预览和调试。 3. 在微信开发者工具内编译、预览和调试。
反馈生成页使用小程序原生录音能力。短录音会直接追加到反馈内容,长录音每 8 分钟自动切片,多节录音可以汇总为一次反馈。真机首次使用时需要允许麦克风权限;中文转录默认使用项目自带的本地 FunASR 服务,不要求腾讯云密钥。
小程序默认连接 `http://127.0.0.1:8080`。在开发者工具模拟器中,可进入“我的”页面保存其他 API 地址并检测服务状态。 小程序默认连接 `http://127.0.0.1:8080`。在开发者工具模拟器中,可进入“我的”页面保存其他 API 地址并检测服务状态。
## 验证 ## 验证
@@ -33,6 +35,15 @@ cargo run --bin migrate
cargo run cargo run
``` ```
另开一个终端启动中文转录服务。首次启动会下载 Paraformer、VAD 和标点模型,下载完成后健康检查才会显示语音服务可用:
```bash
docker compose -f compose.asr.yml up --build -d
curl http://127.0.0.1:10095/health
```
没有 Docker 时可按 [asr-service/README.md](./asr-service/README.md) 使用 Python 3.10 启动。Rust API 默认连接 `http://127.0.0.1:10095`;模型未就绪时已上传录音会保留在队列中,不会阻塞小程序上传。
未设置 `DATABASE_URL` 时,仅健康检查可用,数据接口会返回 `503`。数据库配置和迁移说明参见 [server/API.md](./server/API.md)。 未设置 `DATABASE_URL` 时,仅健康检查可用,数据接口会返回 `503`。数据库配置和迁移说明参见 [server/API.md](./server/API.md)。
服务启动后可打开 `http://127.0.0.1:8080/scalar` 查看并直接调用 Scalar 交互文档,原始规范位于 `http://127.0.0.1:8080/openapi.json`。完整测试顺序参见 [server/API_GUIDE.md](./server/API_GUIDE.md)。 服务启动后可打开 `http://127.0.0.1:8080/scalar` 查看并直接调用 Scalar 交互文档,原始规范位于 `http://127.0.0.1:8080/openapi.json`。完整测试顺序参见 [server/API_GUIDE.md](./server/API_GUIDE.md)。
@@ -40,6 +51,7 @@ cargo run
## 小程序与后端 ## 小程序与后端
- 学生档案、档案预设和反馈记录均通过 `utils/api.ts` 访问 Rust API。 - 学生档案、档案预设和反馈记录均通过 `utils/api.ts` 访问 Rust API。
- 反馈页只暴露一个语音入口;反馈批次、课节、录音片段和转录状态由后端自动维护。
- 开发者工具模拟器使用默认地址 `http://127.0.0.1:8080` - 开发者工具模拟器使用默认地址 `http://127.0.0.1:8080`
- 真机和正式版本不能把 `127.0.0.1` 当作电脑地址;应部署可公网访问的 HTTPS API并在微信公众平台配置 request 合法域名,然后到小程序“我的”页面修改服务地址。 - 真机和正式版本不能把 `127.0.0.1` 当作电脑地址;应部署可公网访问的 HTTPS API并在微信公众平台配置 request 合法域名,然后到小程序“我的”页面修改服务地址。
- 微信登录尚未接入。业务请求暂时使用固定开发用户 UUID生产发布前必须改为由后端根据微信登录凭据解析用户身份。 - 微信登录尚未接入。业务请求暂时使用固定开发用户 UUID生产发布前必须改为由后端根据微信登录凭据解析用户身份。

View File

@@ -13,6 +13,11 @@
"backgroundColor": "#F4F8FE", "backgroundColor": "#F4F8FE",
"backgroundTextStyle": "light" "backgroundTextStyle": "light"
}, },
"permission": {
"scope.record": {
"desc": "用于将课堂反馈录音转成文字"
}
},
"tabBar": { "tabBar": {
"custom": true, "custom": true,
"color": "#7B8796", "color": "#7B8796",

View File

@@ -0,0 +1,4 @@
__pycache__/
*.pyc
.venv/
.pytest_cache/

20
asr-service/Dockerfile Normal file
View File

@@ -0,0 +1,20 @@
FROM python:3.10-slim
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1
RUN apt-get update \
&& apt-get install -y --no-install-recommends ffmpeg libsndfile1 \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt ./
RUN pip install --upgrade pip \
&& pip install torch==2.8.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cpu \
&& pip install -r requirements.txt
COPY app.py ./
EXPOSE 10095
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "10095", "--workers", "1", "--no-access-log"]

38
asr-service/README.md Normal file
View File

@@ -0,0 +1,38 @@
# 中文录音转写服务
该服务使用 FunASR 的中文 Paraformer、FSMN-VAD 和标点模型,为 Rust API 提供本地录音转写。模型首次启动时会下载约 2.1 GB 到持久化缓存,完成后 `/health` 才会可用。
## Docker 启动
在项目根目录执行:
```bash
docker compose -f compose.asr.yml up --build -d
curl http://127.0.0.1:10095/health
```
默认使用 CPU 且只允许一个并发推理任务。可通过 `ASR_DEVICE``ASR_MAX_CONCURRENCY``ASR_MODEL``ASR_VAD_MODEL``ASR_PUNC_MODEL` 调整。GPU 部署需要使用带 CUDA 的 PyTorch 基础环境,不能只把 CPU 镜像中的 `ASR_DEVICE` 改为 `cuda`
## 本机 Python 启动
支持 Python 3.9 及以上版本,推荐 Python 3.10
```bash
cd asr-service
python3.10 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
MODELSCOPE_CACHE=.models uvicorn app:app --host 127.0.0.1 --port 10095 --no-access-log
```
## 接口
Rust API 调用 `POST /v1/transcriptions`,以 multipart 上传 `file``language``duration_ms`,响应格式为:
```json
{
"text": "识别后的中文课堂内容。",
"duration_ms": 480000,
"request_id": "b2f6fc1c-62c1-4a99-8ab4-6f933f1bd252"
}
```

173
asr-service/app.py Normal file
View File

@@ -0,0 +1,173 @@
from __future__ import annotations
import asyncio
import logging
import os
import tempfile
import uuid
from contextlib import asynccontextmanager
from pathlib import Path
from typing import Any
from fastapi import FastAPI, File, Form, HTTPException, UploadFile
from funasr import AutoModel
from pydantic import BaseModel
LOGGER = logging.getLogger("funasr-service")
MAX_AUDIO_BYTES = 15 * 1024 * 1024
CHUNK_BYTES = 1024 * 1024
SUPPORTED_FORMATS = {"mp3", "aac", "wav", "m4a", "amr"}
class Settings:
def __init__(self) -> None:
self.model = os.getenv("ASR_MODEL", "paraformer-zh")
self.vad_model = os.getenv("ASR_VAD_MODEL", "fsmn-vad")
self.punc_model = os.getenv("ASR_PUNC_MODEL", "ct-punc")
self.device = os.getenv("ASR_DEVICE", "cpu")
self.max_concurrency = positive_int("ASR_MAX_CONCURRENCY", 1)
self.batch_size_seconds = positive_int("ASR_BATCH_SIZE_SECONDS", 300)
class HealthResponse(BaseModel):
status: str
model: str
device: str
class TranscriptionResponse(BaseModel):
text: str
duration_ms: int
request_id: str
def positive_int(name: str, default: int) -> int:
raw = os.getenv(name, str(default))
try:
value = int(raw)
except ValueError as error:
raise RuntimeError(f"{name} must be an integer") from error
if value <= 0:
raise RuntimeError(f"{name} must be greater than 0")
return value
def load_model(settings: Settings) -> Any:
LOGGER.info(
"loading FunASR model model=%s vad=%s punc=%s device=%s",
settings.model,
settings.vad_model,
settings.punc_model,
settings.device,
)
model = AutoModel(
model=settings.model,
vad_model=settings.vad_model,
punc_model=settings.punc_model,
device=settings.device,
disable_update=True,
)
LOGGER.info("FunASR model is ready")
return model
@asynccontextmanager
async def lifespan(app: FastAPI):
settings = Settings()
app.state.settings = settings
app.state.model = await asyncio.to_thread(load_model, settings)
app.state.inference_slots = asyncio.Semaphore(settings.max_concurrency)
yield
app = FastAPI(
title="Teaching Feedback FunASR Service",
version="1.0.0",
lifespan=lifespan,
)
@app.get("/health", response_model=HealthResponse)
async def health() -> HealthResponse:
settings: Settings = app.state.settings
return HealthResponse(status="ok", model=settings.model, device=settings.device)
@app.post("/v1/transcriptions", response_model=TranscriptionResponse)
async def create_transcription(
file: UploadFile = File(...),
language: str = Form("zh"),
duration_ms: int = Form(0),
hotword: str = Form(""),
) -> TranscriptionResponse:
if language not in {"zh", "zh-cn", "cmn"}:
raise HTTPException(status_code=422, detail="this service is configured for Chinese audio")
audio_format = file_extension(file.filename)
temporary_path = await save_upload(file, audio_format)
request_id = str(uuid.uuid4())
try:
async with app.state.inference_slots:
text = await asyncio.to_thread(
run_inference,
app.state.model,
temporary_path,
app.state.settings.batch_size_seconds,
hotword.strip(),
)
except Exception as error:
LOGGER.exception("FunASR inference failed request_id=%s", request_id)
raise HTTPException(status_code=500, detail=f"speech recognition failed: {error}") from error
finally:
temporary_path.unlink(missing_ok=True)
if not text:
raise HTTPException(status_code=422, detail="no clear speech was recognized")
return TranscriptionResponse(
text=text,
duration_ms=max(0, duration_ms),
request_id=request_id,
)
def file_extension(filename: str | None) -> str:
suffix = Path(filename or "segment.mp3").suffix.lower().lstrip(".")
if suffix not in SUPPORTED_FORMATS:
raise HTTPException(status_code=415, detail="unsupported audio format")
return suffix
async def save_upload(file: UploadFile, audio_format: str) -> Path:
handle = tempfile.NamedTemporaryFile(suffix=f".{audio_format}", delete=False)
path = Path(handle.name)
total_bytes = 0
try:
with handle:
while chunk := await file.read(CHUNK_BYTES):
total_bytes += len(chunk)
if total_bytes > MAX_AUDIO_BYTES:
raise HTTPException(status_code=413, detail="audio file exceeds 15 MB")
handle.write(chunk)
except Exception:
path.unlink(missing_ok=True)
raise
finally:
await file.close()
if total_bytes == 0:
path.unlink(missing_ok=True)
raise HTTPException(status_code=422, detail="audio file is empty")
return path
def run_inference(model: Any, path: Path, batch_size_seconds: int, hotword: str) -> str:
options: dict[str, Any] = {
"input": str(path),
"batch_size_s": batch_size_seconds,
"use_itn": True,
}
if hotword:
options["hotword"] = hotword
result = model.generate(**options)
if not isinstance(result, list):
return ""
texts = [str(item.get("text", "")).strip() for item in result if isinstance(item, dict)]
return "\n".join(text for text in texts if text).strip()

View File

@@ -0,0 +1,7 @@
fastapi==0.128.8
funasr==1.3.14
modelscope==1.37.1
python-multipart==0.0.20
torch==2.8.0
torchaudio==2.8.0
uvicorn[standard]==0.39.0

31
compose.asr.yml Normal file
View File

@@ -0,0 +1,31 @@
services:
funasr:
build:
context: ./asr-service
ports:
- "127.0.0.1:10095:10095"
environment:
ASR_MODEL: paraformer-zh
ASR_VAD_MODEL: fsmn-vad
ASR_PUNC_MODEL: ct-punc
ASR_DEVICE: cpu
ASR_MAX_CONCURRENCY: "1"
ASR_BATCH_SIZE_SECONDS: "300"
MODELSCOPE_CACHE: /models
volumes:
- funasr-model-cache:/models
restart: unless-stopped
healthcheck:
test:
- CMD
- python
- -c
- "import urllib.request; urllib.request.urlopen('http://127.0.0.1:10095/health', timeout=3)"
interval: 15s
timeout: 5s
retries: 20
start_period: 5m
shm_size: 1gb
volumes:
funasr-model-cache:

View File

@@ -1,13 +1,54 @@
import { import {
createFeedbackRecord, createFeedbackRecord,
createVoiceLesson,
ensureFeedbackSession,
finishVoiceLesson,
generateFeedbackFromVoice,
getActiveFeedbackSession,
getApiErrorMessage, getApiErrorMessage,
listProfiles listProfiles,
markVoiceLessonApplied,
retryVoiceSegment,
updateFeedbackSession,
uploadVoiceSegment
} from '../../utils/api' } from '../../utils/api'
import type { FeedbackDraft, StudentProfile } from '../../utils/types' import type {
FeedbackDraft,
FeedbackSession,
StudentProfile,
VoiceLessonFinished
} from '../../utils/types'
type PickerEvent = { detail: { value: string | number } } type PickerEvent = { detail: { value: string | number } }
type InputEvent = { detail: { value: string } } type InputEvent = { detail: { value: string } }
interface PendingVoiceUpload {
lessonId: string
sequence: number
durationMs: number
filePath: string
}
const MAX_CONTENT_LENGTH = 2000
const RECORDING_SEGMENT_DURATION_MS = 8 * 60 * 1000
const SHORT_RECORDING_DURATION_MS = 60 * 1000
const VOICE_POLL_INTERVAL_MS = 3000
const PENDING_UPLOADS_KEY = 'teaching-feedback-pending-voice-uploads-v1'
const recorderManager = wx.getRecorderManager()
let recordingTimer: number | null = null
let draftSaveTimer: number | null = null
let voicePollTimer: number | null = null
let voicePollBusy = false
let pageVisible = false
let lessonStartedAt = 0
let segmentStartedAt = 0
let currentLessonId: string | null = null
let currentSegmentSequence = 0
let finishRequested = false
let pendingUploads: Promise<void>[] = []
let failedLocalUploads: PendingVoiceUpload[] = []
function today(): string { function today(): string {
const now = new Date() const now = new Date()
const year = now.getFullYear() const year = now.getFullYear()
@@ -20,39 +61,147 @@ function buildDraft(): FeedbackDraft {
return { return {
profileId: null, profileId: null,
feedbackDate: today(), feedbackDate: today(),
content: '' content: '',
feedbackSessionId: null
} }
} }
function appendTranscript(content: string, transcript: string): { content: string; truncated: boolean } {
const normalizedTranscript = transcript.trim()
const separator = content && !/\s$/.test(content) ? '\n' : ''
const combined = `${content}${separator}${normalizedTranscript}`
return {
content: combined.slice(0, MAX_CONTENT_LENGTH),
truncated: combined.length > MAX_CONTENT_LENGTH
}
}
function formatRecordingTime(milliseconds: number): string {
const totalSeconds = Math.max(0, Math.floor(milliseconds / 1000))
const hours = Math.floor(totalSeconds / 3600)
const minutes = Math.floor((totalSeconds % 3600) / 60)
const seconds = totalSeconds % 60
const minuteText = String(minutes).padStart(2, '0')
const secondText = String(seconds).padStart(2, '0')
return hours > 0 ? `${String(hours).padStart(2, '0')}:${minuteText}:${secondText}` : `${minuteText}:${secondText}`
}
function formatVoiceDuration(milliseconds: number): string {
const totalMinutes = Math.floor(milliseconds / 60000)
if (totalMinutes < 1) return `${Math.max(1, Math.round(milliseconds / 1000))}`
return `${totalMinutes}分钟`
}
function wait(milliseconds: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, milliseconds))
}
Page({ Page({
data: { data: {
profiles: [] as StudentProfile[], profiles: [] as StudentProfile[],
profileOptions: ['不关联学生'], profileOptions: ['不关联学生'],
selectedProfileIndex: 0, selectedProfileIndex: 0,
draft: buildDraft(), draft: buildDraft(),
session: null as FeedbackSession | null,
contentLength: 0, contentLength: 0,
loading: false, loading: false,
saving: false, saving: false,
generating: false,
voiceBusy: false,
recording: false,
recordingPaused: false,
recordingTime: '00:00',
voiceStatus: '语音录入',
voiceActionable: false,
primaryActionText: '保存反馈',
errorMessage: '' errorMessage: ''
}, },
onLoad() {
failedLocalUploads = (wx.getStorageSync(PENDING_UPLOADS_KEY) as PendingVoiceUpload[]) || []
this.initRecorder()
if (failedLocalUploads.length > 0) {
currentLessonId = failedLocalUploads[0].lessonId
this.setData({ voiceStatus: '上传失败,点击重试', voiceActionable: true })
}
},
onShow() { onShow() {
pageVisible = true
this.getTabBar()?.setData({ selected: 0 }) this.getTabBar()?.setData({ selected: 0 })
void this.loadProfiles() void this.loadProfiles()
}, },
onHide() {
pageVisible = false
this.clearVoicePollTimer()
if (this.data.recording) this.stopRecording()
},
onUnload() {
pageVisible = false
this.clearRecordingTimer()
this.clearDraftSaveTimer()
this.clearVoicePollTimer()
if (this.data.recording) recorderManager.stop()
},
initRecorder() {
recorderManager.onStart(() => {
segmentStartedAt = Date.now()
this.setData({
voiceBusy: false,
recording: true,
recordingPaused: false,
voiceStatus: '正在录音'
})
this.startRecordingTimer()
})
recorderManager.onStop((result) => {
void this.handleRecorderStop(result)
})
recorderManager.onPause(() => {
this.setData({ recordingPaused: true, voiceStatus: '录音已暂停' })
})
recorderManager.onResume(() => {
this.setData({ recordingPaused: false, voiceStatus: '正在录音' })
})
recorderManager.onInterruptionEnd(() => {
if (this.data.recordingPaused) recorderManager.resume()
})
recorderManager.onError((error) => {
console.warn('RecorderManager error', error)
this.clearRecordingTimer()
finishRequested = true
this.setData({
recording: false,
recordingPaused: false,
voiceBusy: true,
voiceStatus: '正在保留已录内容'
})
void this.finalizeCurrentLesson()
})
},
async loadProfiles() { async loadProfiles() {
this.setData({ loading: true, errorMessage: '' }) this.setData({ loading: true, errorMessage: '' })
try { try {
const profiles = await listProfiles() const profiles = await listProfiles()
const matchedProfileIndex = profiles.findIndex((profile) => profile.id === this.data.draft.profileId) const matchedProfileIndex = profiles.findIndex((profile) => profile.id === this.data.draft.profileId)
const selectedProfileIndex = matchedProfileIndex + 1 const selectedProfileIndex = matchedProfileIndex + 1
const profileId = matchedProfileIndex >= 0 ? profiles[matchedProfileIndex].id : null
this.setData({ this.setData({
profiles, profiles,
profileOptions: ['不关联学生', ...profiles.map((profile) => `${profile.name} · ${profile.subject}`)], profileOptions: ['不关联学生', ...profiles.map((profile) => `${profile.name} · ${profile.subject}`)],
selectedProfileIndex, selectedProfileIndex,
'draft.profileId': matchedProfileIndex >= 0 ? profiles[matchedProfileIndex].id : null 'draft.profileId': profileId
}) })
await this.loadActiveSession(profileId)
} catch (error) { } catch (error) {
this.setData({ errorMessage: getApiErrorMessage(error) }) this.setData({ errorMessage: getApiErrorMessage(error) })
} finally { } finally {
@@ -60,29 +209,483 @@ Page({
} }
}, },
async loadActiveSession(profileId?: string | null) {
try {
const targetProfileId = profileId === undefined ? this.data.draft.profileId : profileId
const session = await getActiveFeedbackSession(targetProfileId)
if (targetProfileId !== this.data.draft.profileId) return
const nextData: Record<string, unknown> = {
session,
'draft.feedbackSessionId': session?.id || null
}
if (session) {
nextData['draft.feedbackDate'] = session.feedbackDate
nextData['draft.content'] = session.content
nextData.contentLength = session.content.length
}
this.setData(nextData)
this.syncVoicePresentation(session)
this.scheduleVoicePolling(session)
} catch (error) {
this.setData({ errorMessage: getApiErrorMessage(error) })
}
},
syncVoicePresentation(session: FeedbackSession | null) {
if (this.data.recording || this.data.voiceBusy) return
if (failedLocalUploads.length > 0) {
this.setData({ voiceStatus: '上传失败,点击重试', voiceActionable: true })
return
}
if (!session || session.lessonCount === 0) {
this.setData({ voiceStatus: '语音录入', voiceActionable: false, primaryActionText: '保存反馈' })
return
}
const prefix = `${session.lessonCount}节 · ${formatVoiceDuration(session.totalDurationMs)}`
let suffix = '已就绪'
if (session.failedSegmentCount > 0) suffix = '转录失败,点击重试'
else if (session.processingSegmentCount > 0) suffix = '正在处理'
let primaryActionText = session.needsGeneration ? '生成反馈' : '保存反馈'
if (session.processingSegmentCount > 0) primaryActionText = '转录中'
this.setData({
voiceStatus: `${prefix} · ${suffix}`,
voiceActionable: true,
primaryActionText
})
},
scheduleVoicePolling(session: FeedbackSession | null) {
this.clearVoicePollTimer()
if (!pageVisible || !session || session.processingSegmentCount === 0) return
voicePollTimer = setTimeout(() => {
voicePollTimer = null
void this.pollVoiceSession()
}, VOICE_POLL_INTERVAL_MS) as unknown as number
},
clearVoicePollTimer() {
if (voicePollTimer === null) return
clearTimeout(voicePollTimer)
voicePollTimer = null
},
async pollVoiceSession() {
if (!pageVisible) return
if (voicePollBusy || this.data.recording || this.data.voiceBusy) {
this.scheduleVoicePolling(this.data.session)
return
}
voicePollBusy = true
const profileId = this.data.draft.profileId
try {
const session = await getActiveFeedbackSession(profileId)
if (!pageVisible || profileId !== this.data.draft.profileId) return
this.setData({ session, 'draft.feedbackSessionId': session?.id || null })
this.syncVoicePresentation(session)
if (!session) return
const readyShortLessons = session.lessons.filter(
(lesson) =>
lesson.status === 'ready' &&
lesson.durationMs <= SHORT_RECORDING_DURATION_MS &&
!lesson.appliedDirectly &&
!lesson.includedInGeneration
)
for (const lesson of readyShortLessons) {
const finished = await finishVoiceLesson(lesson.id)
await this.applyFinishedLesson(finished)
}
if (readyShortLessons.length > 0) await this.loadActiveSession(profileId)
} catch (error) {
console.warn('Failed to refresh voice transcription status', error)
} finally {
voicePollBusy = false
this.scheduleVoicePolling(this.data.session)
}
},
retryLoad() { retryLoad() {
void this.loadProfiles() void this.loadProfiles()
}, },
onProfileChange(event: PickerEvent) { onProfileChange(event: PickerEvent) {
this.clearDraftSaveTimer()
const selectedProfileIndex = Number(event.detail.value) const selectedProfileIndex = Number(event.detail.value)
const profile = this.data.profiles[selectedProfileIndex - 1] const profile = this.data.profiles[selectedProfileIndex - 1]
const profileId = profile?.id || null
this.setData({ this.setData({
selectedProfileIndex, selectedProfileIndex,
'draft.profileId': profile?.id || null 'draft.profileId': profileId,
session: null,
'draft.feedbackSessionId': null,
'draft.content': '',
contentLength: 0,
voiceActionable: false,
primaryActionText: '保存反馈'
}) })
void this.loadActiveSession(profileId)
}, },
onDateChange(event: InputEvent) { onDateChange(event: InputEvent) {
this.setData({ 'draft.feedbackDate': event.detail.value }) this.setData({ 'draft.feedbackDate': event.detail.value })
this.scheduleDraftSave()
}, },
onContentInput(event: InputEvent) { onContentInput(event: InputEvent) {
const content = event.detail.value const content = event.detail.value
this.setData({ 'draft.content': content, contentLength: content.length }) this.setData({ 'draft.content': content, contentLength: content.length })
this.scheduleDraftSave()
},
scheduleDraftSave() {
this.clearDraftSaveTimer()
if (!this.data.session) return
draftSaveTimer = setTimeout(() => {
void this.persistSessionDraft()
}, 800) as unknown as number
},
clearDraftSaveTimer() {
if (draftSaveTimer === null) return
clearTimeout(draftSaveTimer)
draftSaveTimer = null
},
async persistSessionDraft() {
const session = this.data.session
if (!session) return
const sessionId = session.id
try {
const updated = await updateFeedbackSession(
sessionId,
this.data.draft.feedbackDate,
this.data.draft.content
)
if (this.data.session?.id !== sessionId) return
this.setData({ session: updated })
this.syncVoicePresentation(updated)
} catch (error) {
console.warn('Failed to auto-save feedback session', error)
}
},
ensureRecordPermission(): Promise<boolean> {
return new Promise((resolve) => {
wx.authorize({
scope: 'scope.record',
success: () => resolve(true),
fail: () => {
wx.showModal({
title: '需要麦克风权限',
content: '请在设置中允许使用麦克风,才能录音转录。',
confirmText: '去设置',
success: (modalResult) => {
if (!modalResult.confirm) {
resolve(false)
return
}
wx.openSetting({
success: (settingResult) => resolve(settingResult.authSetting['scope.record'] === true),
fail: () => resolve(false)
})
},
fail: () => resolve(false)
})
}
})
})
},
async toggleRecording() {
if (failedLocalUploads.length > 0) {
await this.retryPendingUploads()
return
}
if (this.data.recording) {
this.stopRecording()
return
}
this.setData({ voiceBusy: true, voiceStatus: '正在开启麦克风' })
const authorized = await this.ensureRecordPermission()
if (!authorized) {
this.setData({ voiceBusy: false })
this.syncVoicePresentation(this.data.session)
return
}
try {
const session = await ensureFeedbackSession(
this.data.draft.profileId,
this.data.draft.feedbackDate,
this.data.draft.content
)
const lesson = await createVoiceLesson(session.id)
currentLessonId = lesson.id
currentSegmentSequence = 0
finishRequested = false
pendingUploads = []
lessonStartedAt = Date.now()
this.setData({
session,
'draft.feedbackSessionId': session.id,
recordingTime: '00:00'
})
this.startNextRecordingSegment()
} catch (error) {
this.setData({ voiceBusy: false })
this.syncVoicePresentation(this.data.session)
wx.showToast({ title: getApiErrorMessage(error), icon: 'none' })
}
},
startNextRecordingSegment() {
currentSegmentSequence += 1
recorderManager.start({
duration: RECORDING_SEGMENT_DURATION_MS,
sampleRate: 16000,
numberOfChannels: 1,
encodeBitRate: 48000,
format: 'mp3',
audioSource: 'auto'
})
},
stopRecording() {
if (!this.data.recording) return
finishRequested = true
this.clearRecordingTimer()
this.setData({
recording: false,
recordingPaused: false,
voiceBusy: true,
voiceStatus: '正在整理录音'
})
recorderManager.stop()
},
async handleRecorderStop(result: WechatMiniprogram.OnStopListenerResult) {
const lessonId = currentLessonId
if (!lessonId) return
const durationMs = Math.max(1, result.duration || Date.now() - segmentStartedAt)
const upload = this.queueSegmentUpload({
lessonId,
sequence: currentSegmentSequence,
durationMs,
filePath: result.tempFilePath
})
pendingUploads.push(upload)
if (!finishRequested) {
this.startNextRecordingSegment()
return
}
await this.finalizeCurrentLesson()
},
async queueSegmentUpload(segment: PendingVoiceUpload): Promise<void> {
try {
await this.uploadWithRetry(segment)
} catch (error) {
console.warn('Voice segment upload failed', error)
const saved = await this.persistRecordingFile(segment)
failedLocalUploads.push(saved)
wx.setStorageSync(PENDING_UPLOADS_KEY, failedLocalUploads)
throw error
}
},
async uploadWithRetry(segment: PendingVoiceUpload): Promise<void> {
let lastError: unknown
for (let attempt = 0; attempt < 2; attempt += 1) {
try {
await uploadVoiceSegment(
segment.lessonId,
segment.filePath,
segment.sequence,
segment.durationMs
)
return
} catch (error) {
lastError = error
if (attempt === 0) await wait(800)
}
}
throw lastError
},
persistRecordingFile(segment: PendingVoiceUpload): Promise<PendingVoiceUpload> {
return new Promise((resolve) => {
wx.getFileSystemManager().saveFile({
tempFilePath: segment.filePath,
success: (result) => resolve({ ...segment, filePath: result.savedFilePath }),
fail: () => resolve(segment)
})
})
},
async retryPendingUploads() {
if (failedLocalUploads.length === 0) return
this.setData({ voiceBusy: true, voiceStatus: '正在重试上传' })
const remaining: PendingVoiceUpload[] = []
for (const segment of failedLocalUploads) {
try {
await this.uploadWithRetry(segment)
this.removeSavedFile(segment.filePath)
} catch {
remaining.push(segment)
}
}
failedLocalUploads = remaining
wx.setStorageSync(PENDING_UPLOADS_KEY, failedLocalUploads)
if (remaining.length > 0) {
this.setData({ voiceBusy: false, voiceStatus: '上传失败,点击重试', voiceActionable: true })
wx.showToast({ title: '录音上传失败,请检查网络', icon: 'none' })
return
}
await this.finalizeCurrentLesson()
},
removeSavedFile(filePath: string) {
wx.getFileSystemManager().unlink({ filePath, fail: () => undefined })
},
async finalizeCurrentLesson() {
const lessonId = currentLessonId
if (!lessonId) {
this.setData({ voiceBusy: false, recording: false })
return
}
const results = await Promise.allSettled(pendingUploads)
pendingUploads = []
if (results.some((result) => result.status === 'rejected') || failedLocalUploads.length > 0) {
this.setData({
recording: false,
voiceBusy: false,
voiceStatus: '上传失败,点击重试',
voiceActionable: true
})
return
}
try {
const finished = await finishVoiceLesson(lessonId)
await this.applyFinishedLesson(finished)
currentLessonId = null
currentSegmentSequence = 0
finishRequested = false
await this.loadActiveSession()
} catch (error) {
wx.showToast({ title: getApiErrorMessage(error), icon: 'none' })
} finally {
this.clearRecordingTimer()
this.setData({ recording: false, recordingPaused: false, voiceBusy: false, recordingTime: '00:00' })
this.syncVoicePresentation(this.data.session)
}
},
async applyFinishedLesson(finished: VoiceLessonFinished) {
if (finished.status === 'processing') {
wx.showToast({ title: '录音已上传,正在后台转写', icon: 'none' })
return
}
if (finished.status === 'failed') {
wx.showToast({ title: '部分录音转录失败,点击状态可重试', icon: 'none' })
return
}
if (!finished.canApplyDirectly) {
wx.showToast({ title: '录音已就绪,可生成反馈', icon: 'none' })
return
}
const merged = appendTranscript(this.data.draft.content, finished.transcript)
this.setData({ 'draft.content': merged.content, contentLength: merged.content.length })
if (!merged.truncated) {
await markVoiceLessonApplied(finished.id, merged.content)
wx.showToast({ title: '语音已转为文字', icon: 'none' })
} else {
wx.showToast({ title: '内容已达上限,请生成汇总反馈', icon: 'none' })
}
},
startRecordingTimer() {
this.clearRecordingTimer()
recordingTimer = setInterval(() => {
this.setData({ recordingTime: formatRecordingTime(Date.now() - lessonStartedAt) })
}, 1000) as unknown as number
},
clearRecordingTimer() {
if (recordingTimer === null) return
clearInterval(recordingTimer)
recordingTimer = null
},
async openVoiceDetails() {
if (this.data.recording || this.data.voiceBusy) return
if (failedLocalUploads.length > 0) {
await this.retryPendingUploads()
return
}
const session = this.data.session
if (!session || session.lessonCount === 0) return
const lines = session.lessons.map((lesson) => {
const status = lesson.status === 'ready' ? '已就绪' : lesson.status === 'failed' ? '转录失败' : '处理中'
return `${lesson.sequence}节 · ${formatVoiceDuration(lesson.durationMs)} · ${status}`
})
const hasFailures = session.failedSegmentCount > 0
wx.showModal({
title: '语音记录',
content: lines.join('\n'),
showCancel: hasFailures,
cancelText: '关闭',
confirmText: hasFailures ? '重试失败项' : '知道了',
success: (result) => {
if (hasFailures && result.confirm) void this.retryFailedSegments()
}
})
},
async retryFailedSegments() {
const session = this.data.session
if (!session) return
const failedSegments = session.lessons.flatMap((lesson) =>
lesson.segments.filter((segment) => segment.status === 'failed')
)
if (failedSegments.length === 0) return
this.setData({ voiceBusy: true, voiceStatus: '正在重试转录' })
try {
for (const segment of failedSegments) await retryVoiceSegment(segment.id)
const failedLessonIds = session.lessons
.filter((lesson) => lesson.segments.some((segment) => segment.status === 'failed'))
.map((lesson) => lesson.id)
for (const lessonId of failedLessonIds) {
const finished = await finishVoiceLesson(lessonId)
await this.applyFinishedLesson(finished)
}
await this.loadActiveSession()
} catch (error) {
wx.showToast({ title: getApiErrorMessage(error), icon: 'none' })
} finally {
this.setData({ voiceBusy: false })
this.syncVoicePresentation(this.data.session)
}
}, },
async saveFeedback() { async saveFeedback() {
this.clearDraftSaveTimer()
const session = this.data.session
if (session && session.processingSegmentCount > 0) {
wx.showToast({ title: '录音仍在转写,请稍候', icon: 'none' })
this.scheduleVoicePolling(session)
return
}
if (session?.needsGeneration) {
await this.generateFeedback()
return
}
const content = this.data.draft.content.trim() const content = this.data.draft.content.trim()
if (!content) { if (!content) {
wx.showToast({ title: '请填写反馈内容', icon: 'none' }) wx.showToast({ title: '请填写反馈内容', icon: 'none' })
@@ -91,8 +694,20 @@ Page({
this.setData({ saving: true }) this.setData({ saving: true })
try { try {
await createFeedbackRecord({ ...this.data.draft, content }) await createFeedbackRecord({
this.setData({ draft: buildDraft(), selectedProfileIndex: 0, contentLength: 0 }) ...this.data.draft,
content,
feedbackSessionId: session?.id || null
})
this.setData({
draft: buildDraft(),
session: null,
selectedProfileIndex: 0,
contentLength: 0,
voiceStatus: '语音录入',
voiceActionable: false,
primaryActionText: '保存反馈'
})
wx.showToast({ title: '反馈已保存', icon: 'success' }) wx.showToast({ title: '反馈已保存', icon: 'success' })
setTimeout(() => wx.switchTab({ url: '/pages/records/index' }), 450) setTimeout(() => wx.switchTab({ url: '/pages/records/index' }), 450)
} catch (error) { } catch (error) {
@@ -102,6 +717,26 @@ Page({
} }
}, },
async generateFeedback() {
const session = this.data.session
if (!session) return
this.setData({ generating: true, voiceBusy: true, voiceStatus: '正在生成反馈' })
try {
const result = await generateFeedbackFromVoice(session.id, this.data.draft.content)
this.setData({
'draft.content': result.content,
contentLength: result.content.length
})
await this.loadActiveSession()
wx.showToast({ title: '反馈已生成,可继续修改', icon: 'none' })
} catch (error) {
wx.showToast({ title: getApiErrorMessage(error), icon: 'none' })
} finally {
this.setData({ generating: false, voiceBusy: false })
this.syncVoicePresentation(this.data.session)
}
},
goStudents() { goStudents() {
wx.switchTab({ url: '/pages/students/index' }) wx.switchTab({ url: '/pages/students/index' })
} }

View File

@@ -12,12 +12,12 @@
</view> </view>
<view class="feedback-form surface"> <view class="feedback-form surface">
<picker class="feedback-field" mode="selector" range="{{profileOptions}}" value="{{selectedProfileIndex}}" disabled="{{loading}}" bindchange="onProfileChange"> <picker class="feedback-field" mode="selector" range="{{profileOptions}}" value="{{selectedProfileIndex}}" disabled="{{loading || recording || voiceBusy}}" bindchange="onProfileChange">
<text class="form-label">学生档案</text> <text class="form-label">学生档案</text>
<text class="form-value">{{loading ? '正在加载...' : profileOptions[selectedProfileIndex]}}</text> <text class="form-value">{{loading ? '正在加载...' : profileOptions[selectedProfileIndex]}}</text>
</picker> </picker>
<picker class="feedback-field" mode="date" value="{{draft.feedbackDate}}" bindchange="onDateChange"> <picker class="feedback-field" mode="date" value="{{draft.feedbackDate}}" disabled="{{recording || voiceBusy}}" bindchange="onDateChange">
<text class="form-label">反馈日期</text> <text class="form-label">反馈日期</text>
<text class="form-value">{{draft.feedbackDate}}</text> <text class="form-value">{{draft.feedbackDate}}</text>
</picker> </picker>
@@ -31,12 +31,30 @@
class="feedback-textarea" class="feedback-textarea"
value="{{draft.content}}" value="{{draft.content}}"
maxlength="2000" maxlength="2000"
disabled="{{recording || voiceBusy}}"
placeholder="记录课堂完成情况、进步与后续建议" placeholder="记录课堂完成情况、进步与后续建议"
bindinput="onContentInput" bindinput="onContentInput"
/> />
<view class="voice-input-row">
<button
class="voice-button {{recording ? 'voice-button-recording' : ''}}"
disabled="{{saving || voiceBusy}}"
bindtap="toggleRecording"
aria-label="{{recording ? '结束录音' : '开始录音'}}"
>
<text class="voice-button-icon">{{recording ? '■' : '●'}}</text>
<text>{{recording ? '结束录音' : '语音录入'}}</text>
</button>
<view class="voice-state {{voiceActionable ? 'voice-state-actionable' : ''}}" bindtap="openVoiceDetails">
<text class="voice-status">{{voiceStatus}}</text>
<text wx:if="{{recording}}" class="recording-time">{{recordingTime}}</text>
<text wx:elif="{{voiceActionable}}" class="voice-state-arrow"></text>
</view>
</view>
</view> </view>
<button class="save-feedback primary-button" loading="{{saving}}" disabled="{{saving}}" bindtap="saveFeedback">保存反馈</button> <button class="save-feedback primary-button" loading="{{saving || generating}}" disabled="{{saving || generating || recording || voiceBusy || (session && session.processingSegmentCount > 0)}}" bindtap="saveFeedback">{{primaryActionText}}</button>
</view> </view>
<view wx:if="{{!loading && profiles.length === 0}}" class="profiles-hint"> <view wx:if="{{!loading && profiles.length === 0}}" class="profiles-hint">

View File

@@ -42,6 +42,91 @@
line-height: 1.6; line-height: 1.6;
} }
.voice-input-row {
display: flex;
align-items: center;
gap: 20rpx;
margin-top: 18rpx;
padding-top: 18rpx;
border-top: 2rpx solid #e7eef6;
}
.voice-button {
display: flex;
align-items: center;
justify-content: center;
gap: 12rpx;
width: 220rpx;
height: 72rpx;
margin: 0;
padding: 0 20rpx;
border: 2rpx solid #b8cbe1;
border-radius: 12rpx;
background: #ffffff;
color: #2b67e8;
font-size: 26rpx;
font-weight: 600;
line-height: 1;
}
.voice-button::after {
border: 0;
}
.voice-button[disabled] {
background: #f4f7fa;
color: #99a5b3;
opacity: 1;
}
.voice-button-recording,
.voice-button-recording[disabled] {
border-color: #e46b65;
background: #fff4f3;
color: #c83f39;
}
.voice-button-icon {
font-size: 20rpx;
line-height: 1;
}
.voice-state {
display: flex;
flex: 1;
align-items: center;
justify-content: space-between;
min-width: 0;
color: #65758a;
font-size: 24rpx;
}
.voice-status {
flex: 1;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.voice-state-actionable {
min-height: 64rpx;
}
.voice-state-arrow {
flex: none;
margin-left: 10rpx;
color: #7f8da0;
font-size: 38rpx;
line-height: 1;
}
.recording-time {
flex: none;
margin-left: 12rpx;
color: #c83f39;
font-variant-numeric: tabular-nums;
}
.save-feedback { .save-feedback {
margin-top: 24rpx; margin-top: 24rpx;
} }

View File

@@ -41,9 +41,21 @@ Page({
this.setData({ testing: true, connectionText: '正在检测服务...', connectionTone: 'idle' }) this.setData({ testing: true, connectionText: '正在检测服务...', connectionTone: 'idle' })
try { try {
const health = await getHealth() const health = await getHealth()
let connectionText = '服务、数据库和语音转录正常'
let connectionTone = 'success'
if (!health.databaseConfigured) {
connectionText = '服务正常,数据库尚未配置'
connectionTone = 'warning'
} else if (!health.speechConfigured) {
connectionText = '服务和数据库正常,语音转录尚未配置'
connectionTone = 'warning'
} else if (!health.speechAvailable) {
connectionText = '服务和数据库正常,本地语音模型尚未就绪'
connectionTone = 'warning'
}
this.setData({ this.setData({
connectionText: health.databaseConfigured ? '服务和数据库连接正常' : '服务正常,数据库尚未配置', connectionText,
connectionTone: health.databaseConfigured ? 'success' : 'warning' connectionTone
}) })
} catch (error) { } catch (error) {
this.setData({ connectionText: getApiErrorMessage(error), connectionTone: 'error' }) this.setData({ connectionText: getApiErrorMessage(error), connectionTone: 'error' })

View File

@@ -37,6 +37,10 @@
"value": "server", "value": "server",
"type": "folder" "type": "folder"
}, },
{
"value": "asr-service",
"type": "folder"
},
{ {
"value": "node_modules", "value": "node_modules",
"type": "folder" "type": "folder"
@@ -44,6 +48,14 @@
{ {
"value": "graphify-out", "value": "graphify-out",
"type": "folder" "type": "folder"
},
{
"value": ".git",
"type": "folder"
},
{
"value": "compose.asr.yml",
"type": "file"
} }
], ],
"include": [] "include": []

View File

@@ -1,11 +1,18 @@
import type { import type {
FeedbackDraft, FeedbackDraft,
FeedbackRecord, FeedbackRecord,
FeedbackSession,
GeneratedFeedback,
HealthStatus, HealthStatus,
StudentDraft, StudentDraft,
StudentProfile, StudentProfile,
StudentProfileDefaults, StudentProfileDefaults,
Term Term,
VoiceAudioSegment,
VoiceLesson,
VoiceLessonCreated,
VoiceLessonFinished,
VoiceProcessingStatus
} from './types' } from './types'
const DEFAULT_API_BASE_URL = 'http://127.0.0.1:8080' const DEFAULT_API_BASE_URL = 'http://127.0.0.1:8080'
@@ -50,6 +57,66 @@ interface RawFeedbackRecord {
interface RawHealthStatus { interface RawHealthStatus {
status: string status: string
database_configured: boolean database_configured: boolean
speech_configured: boolean
speech_available: boolean
speech_provider: string | null
summary_configured: boolean
}
interface RawVoiceAudioSegment {
id: string
sequence: number
duration_ms: number
status: 'processing' | 'ready' | 'failed'
error_message: string | null
}
interface RawVoiceLesson {
id: string
sequence: number
status: VoiceProcessingStatus
duration_ms: number
applied_directly: boolean
included_in_generation: boolean
segments: RawVoiceAudioSegment[]
started_at: string
ended_at: string | null
}
interface RawFeedbackSession {
id: string
profile_id: string | null
feedback_date: string
content: string
status: 'active' | 'finalized'
lesson_count: number
total_duration_ms: number
processing_segment_count: number
failed_segment_count: number
needs_generation: boolean
lessons: RawVoiceLesson[]
created_at: string
updated_at: string
}
interface RawVoiceLessonCreated {
id: string
sequence: number
status: VoiceProcessingStatus
started_at: string
}
interface RawVoiceLessonFinished {
id: string
status: VoiceProcessingStatus
duration_ms: number
transcript: string
can_apply_directly: boolean
}
interface RawGeneratedFeedback {
content: string
method: 'llm' | 'extractive'
} }
type HttpMethod = 'GET' | 'POST' | 'PUT' | 'DELETE' type HttpMethod = 'GET' | 'POST' | 'PUT' | 'DELETE'
@@ -81,17 +148,17 @@ export function getApiErrorMessage(error: unknown): string {
if (!(error instanceof ApiRequestError)) return '请求失败,请稍后重试' if (!(error instanceof ApiRequestError)) return '请求失败,请稍后重试'
if (error.statusCode === 401) return '开发身份未通过后端校验' if (error.statusCode === 401) return '开发身份未通过后端校验'
if (error.statusCode === 404) return '请求的数据不存在或无权访问' if (error.statusCode === 404) return '请求的数据不存在或无权访问'
if (error.statusCode === 503) return '后端尚未配置数据库连接' if (error.statusCode === 503 && error.message.includes('database')) return '后端尚未配置数据库连接'
return error.message return error.message
} }
function request<T>(path: string, method: HttpMethod = 'GET', data?: object): Promise<T> { function request<T>(path: string, method: HttpMethod = 'GET', data?: object, timeout = 10000): Promise<T> {
return new Promise((resolve, reject) => { return new Promise((resolve, reject) => {
wx.request({ wx.request({
url: `${getApiBaseUrl()}${path}`, url: `${getApiBaseUrl()}${path}`,
method, method,
data, data,
timeout: 10000, timeout,
header: { header: {
'Content-Type': 'application/json', 'Content-Type': 'application/json',
'X-User-Id': DEVELOPMENT_USER_ID 'X-User-Id': DEVELOPMENT_USER_ID
@@ -165,11 +232,57 @@ function mapFeedbackRecord(record: RawFeedbackRecord): FeedbackRecord {
} }
} }
function mapVoiceSegment(segment: RawVoiceAudioSegment): VoiceAudioSegment {
return {
id: segment.id,
sequence: segment.sequence,
durationMs: segment.duration_ms,
status: segment.status,
errorMessage: segment.error_message
}
}
function mapVoiceLesson(lesson: RawVoiceLesson): VoiceLesson {
return {
id: lesson.id,
sequence: lesson.sequence,
status: lesson.status,
durationMs: lesson.duration_ms,
appliedDirectly: lesson.applied_directly,
includedInGeneration: lesson.included_in_generation,
segments: lesson.segments.map(mapVoiceSegment),
startedAt: parseTimestamp(lesson.started_at),
endedAt: lesson.ended_at ? parseTimestamp(lesson.ended_at) : null
}
}
function mapFeedbackSession(session: RawFeedbackSession): FeedbackSession {
return {
id: session.id,
profileId: session.profile_id,
feedbackDate: session.feedback_date,
content: session.content,
status: session.status,
lessonCount: session.lesson_count,
totalDurationMs: session.total_duration_ms,
processingSegmentCount: session.processing_segment_count,
failedSegmentCount: session.failed_segment_count,
needsGeneration: session.needs_generation,
lessons: session.lessons.map(mapVoiceLesson),
createdAt: parseTimestamp(session.created_at),
updatedAt: parseTimestamp(session.updated_at)
}
}
export async function getHealth(): Promise<HealthStatus> { export async function getHealth(): Promise<HealthStatus> {
const result = await request<RawHealthStatus>('/health') const result = await request<RawHealthStatus>('/health')
return { return {
status: result.status, status: result.status,
databaseConfigured: result.database_configured databaseConfigured: result.database_configured,
speechConfigured: result.speech_configured,
speechAvailable: result.speech_available,
speechProvider: result.speech_provider,
summaryConfigured: result.summary_configured
} }
} }
@@ -209,7 +322,8 @@ export async function createFeedbackRecord(draft: FeedbackDraft): Promise<Feedba
const result = await request<RawFeedbackRecord>('/api/v1/feedback-records', 'POST', { const result = await request<RawFeedbackRecord>('/api/v1/feedback-records', 'POST', {
profile_id: draft.profileId, profile_id: draft.profileId,
feedback_date: draft.feedbackDate, feedback_date: draft.feedbackDate,
content: draft.content content: draft.content,
feedback_session_id: draft.feedbackSessionId || null
}) })
return mapFeedbackRecord(result) return mapFeedbackRecord(result)
} }
@@ -217,3 +331,134 @@ export async function createFeedbackRecord(draft: FeedbackDraft): Promise<Feedba
export async function deleteFeedbackRecord(recordId: string): Promise<void> { export async function deleteFeedbackRecord(recordId: string): Promise<void> {
await request<unknown>(`/api/v1/feedback-records/${encodeURIComponent(recordId)}`, 'DELETE') await request<unknown>(`/api/v1/feedback-records/${encodeURIComponent(recordId)}`, 'DELETE')
} }
export async function getActiveFeedbackSession(profileId: string | null): Promise<FeedbackSession | null> {
const query = profileId ? `?profile_id=${encodeURIComponent(profileId)}` : ''
const result = await request<RawFeedbackSession | null>(`/api/v1/feedback-sessions/active${query}`)
return result ? mapFeedbackSession(result) : null
}
export async function ensureFeedbackSession(
profileId: string | null,
feedbackDate: string,
content: string
): Promise<FeedbackSession> {
const result = await request<RawFeedbackSession>('/api/v1/feedback-sessions', 'POST', {
profile_id: profileId,
feedback_date: feedbackDate,
content
})
return mapFeedbackSession(result)
}
export async function updateFeedbackSession(
sessionId: string,
feedbackDate: string,
content: string
): Promise<FeedbackSession> {
const result = await request<RawFeedbackSession>(
`/api/v1/feedback-sessions/${encodeURIComponent(sessionId)}`,
'PUT',
{ feedback_date: feedbackDate, content }
)
return mapFeedbackSession(result)
}
export async function createVoiceLesson(sessionId: string): Promise<VoiceLessonCreated> {
const result = await request<RawVoiceLessonCreated>(
`/api/v1/feedback-sessions/${encodeURIComponent(sessionId)}/lessons`,
'POST'
)
return {
id: result.id,
sequence: result.sequence,
status: result.status,
startedAt: parseTimestamp(result.started_at)
}
}
export function uploadVoiceSegment(
lessonId: string,
filePath: string,
sequence: number,
durationMs: number
): Promise<VoiceAudioSegment> {
return new Promise((resolve, reject) => {
wx.uploadFile({
url: `${getApiBaseUrl()}/api/v1/feedback-lessons/${encodeURIComponent(lessonId)}/segments`,
filePath,
name: 'audio',
formData: {
sequence: String(sequence),
duration_ms: String(Math.max(1, Math.round(durationMs))),
audio_format: 'mp3'
},
timeout: 120000,
header: { 'X-User-Id': DEVELOPMENT_USER_ID },
success(response) {
let body: RawVoiceAudioSegment | ApiErrorBody
try {
body = JSON.parse(response.data) as RawVoiceAudioSegment | ApiErrorBody
} catch {
reject(new ApiRequestError('服务返回了无效的录音响应', response.statusCode))
return
}
if (response.statusCode >= 200 && response.statusCode < 300) {
resolve(mapVoiceSegment(body as RawVoiceAudioSegment))
return
}
reject(new ApiRequestError((body as ApiErrorBody).error || `服务返回 HTTP ${response.statusCode}`, response.statusCode))
},
fail(error) {
reject(new ApiRequestError(`录音上传失败:${error.errMsg}`))
}
})
})
}
export async function finishVoiceLesson(lessonId: string): Promise<VoiceLessonFinished> {
const result = await request<RawVoiceLessonFinished>(
`/api/v1/feedback-lessons/${encodeURIComponent(lessonId)}/finish`,
'POST',
undefined,
120000
)
return {
id: result.id,
status: result.status,
durationMs: result.duration_ms,
transcript: result.transcript,
canApplyDirectly: result.can_apply_directly
}
}
export async function markVoiceLessonApplied(lessonId: string, content: string): Promise<void> {
await request<unknown>(
`/api/v1/feedback-lessons/${encodeURIComponent(lessonId)}/mark-applied`,
'POST',
{ content }
)
}
export async function retryVoiceSegment(segmentId: string): Promise<VoiceAudioSegment> {
const result = await request<RawVoiceAudioSegment>(
`/api/v1/audio-segments/${encodeURIComponent(segmentId)}/retry`,
'POST',
undefined,
120000
)
return mapVoiceSegment(result)
}
export async function generateFeedbackFromVoice(
sessionId: string,
existingContent: string
): Promise<GeneratedFeedback> {
const result = await request<RawGeneratedFeedback>(
`/api/v1/feedback-sessions/${encodeURIComponent(sessionId)}/generate`,
'POST',
{ existing_content: existingContent },
120000
)
return result
}

View File

@@ -44,9 +44,72 @@ export interface FeedbackDraft {
profileId: string | null profileId: string | null
feedbackDate: string feedbackDate: string
content: string content: string
feedbackSessionId?: string | null
} }
export interface HealthStatus { export interface HealthStatus {
status: string status: string
databaseConfigured: boolean databaseConfigured: boolean
speechConfigured: boolean
speechAvailable: boolean
speechProvider: string | null
summaryConfigured: boolean
}
export type VoiceProcessingStatus = 'recording' | 'processing' | 'ready' | 'failed'
export interface VoiceAudioSegment {
id: string
sequence: number
durationMs: number
status: 'processing' | 'ready' | 'failed'
errorMessage: string | null
}
export interface VoiceLesson {
id: string
sequence: number
status: VoiceProcessingStatus
durationMs: number
appliedDirectly: boolean
includedInGeneration: boolean
segments: VoiceAudioSegment[]
startedAt: number
endedAt: number | null
}
export interface FeedbackSession {
id: string
profileId: string | null
feedbackDate: string
content: string
status: 'active' | 'finalized'
lessonCount: number
totalDurationMs: number
processingSegmentCount: number
failedSegmentCount: number
needsGeneration: boolean
lessons: VoiceLesson[]
createdAt: number
updatedAt: number
}
export interface VoiceLessonCreated {
id: string
sequence: number
status: VoiceProcessingStatus
startedAt: number
}
export interface VoiceLessonFinished {
id: string
status: VoiceProcessingStatus
durationMs: number
transcript: string
canApplyDirectly: boolean
}
export interface GeneratedFeedback {
content: string
method: 'llm' | 'extractive'
} }