feat: add voice transcription feedback workflow
This commit is contained in:
2
.gitignore
vendored
2
.gitignore
vendored
@@ -3,6 +3,8 @@ graphify-out/
|
||||
|
||||
# Dependencies and build output
|
||||
node_modules/
|
||||
.venv/
|
||||
.models/
|
||||
miniprogram_npm/
|
||||
dist/
|
||||
coverage/
|
||||
|
||||
310
DEBUG_GUIDE.md
Normal file
310
DEBUG_GUIDE.md
Normal file
@@ -0,0 +1,310 @@
|
||||
# 本地调试指南
|
||||
|
||||
本文用于在本机启动 PostgreSQL 连接、中文 FunASR、Rust API 和微信小程序,并验证短录音、长录音及多课节汇总。
|
||||
|
||||
## 1. 服务关系
|
||||
|
||||
```text
|
||||
微信小程序
|
||||
-> Rust API(127.0.0.1:8080)
|
||||
-> PostgreSQL
|
||||
-> FunASR(127.0.0.1:10095)
|
||||
```
|
||||
|
||||
本地录音转写的核心是 FunASR,当前组合为:
|
||||
|
||||
- Paraformer:中文语音识别。
|
||||
- FSMN-VAD:检测并切分有效语音。
|
||||
- CT-Transformer:恢复中文标点。
|
||||
- PyTorch:CPU 推理运行时。
|
||||
|
||||
Rust API 负责音频保存、持久化任务队列、重试、课节状态和反馈汇总。小程序每 8 分钟自动切片,并每 3 秒无感刷新转写状态。
|
||||
|
||||
## 2. 当前主机资源实测
|
||||
|
||||
测试主机为 Apple Silicon、10 核 CPU、16 GB 统一内存,配置为 CPU 单并发:
|
||||
|
||||
| 项目 | 实测值 |
|
||||
| --- | --- |
|
||||
| FunASR 模型缓存 | 约 2.1 GB |
|
||||
| Python 虚拟环境 | 约 871 MB |
|
||||
| 模型加载后的物理内存 | 约 2.8 GB |
|
||||
| 进程历史内存峰值 | 约 4.9 GB |
|
||||
| 8 分钟、16 kHz、单声道、48 kbps MP3 | 约 44.8 秒完成 |
|
||||
| 推理期间 CPU | 约 90% 至 132%,即约 1 至 1.3 个 CPU 核 |
|
||||
|
||||
8 分钟数据使用重复的清晰中文样本测得,真实课堂中的噪声、停顿和说话人数会影响耗时和识别质量。
|
||||
|
||||
容量建议:
|
||||
|
||||
- 当前单教师或低并发调试:不需要 GPU,当前 16 GB 主机足够。
|
||||
- CPU 部署最低建议:4 vCPU、8 GB 内存;8 GB 可能发生交换,16 GB 更稳妥。
|
||||
- 多教师并发:先保持单并发观察队列;若片段持续积压,再考虑增加 CPU 实例或 NVIDIA GPU。
|
||||
- GPU 是可选优化,不是运行前提。CUDA 部署需要单独的 CUDA/PyTorch 镜像,当前 CPU 镜像不能只修改 `ASR_DEVICE=cuda`。
|
||||
- GPU 部署建议从 8 GB 显存起步并用真实课堂录音压测。当前 Apple Silicon 配置没有使用 Apple GPU。
|
||||
- `ASR_WORKER_CONCURRENCY` 和 `ASR_MAX_CONCURRENCY` 应保持一致;每提高一个并发都要重新观察内存和吞吐。
|
||||
|
||||
## 3. 首次准备
|
||||
|
||||
在项目根目录执行:
|
||||
|
||||
```bash
|
||||
cd /Users/zhangheng/project/ai/miniprogram/teaching-feedback-assistant
|
||||
```
|
||||
|
||||
确认 `server/.env` 存在,并填写可访问的 `DATABASE_URL`。不存在时再复制示例,避免覆盖现有配置:
|
||||
|
||||
```bash
|
||||
test -f server/.env || cp server/.env.example server/.env
|
||||
```
|
||||
|
||||
本地 FunASR 默认配置如下,无需腾讯云密钥:
|
||||
|
||||
```env
|
||||
ASR_PROVIDER=local
|
||||
LOCAL_ASR_URL=http://127.0.0.1:10095
|
||||
ASR_REQUEST_TIMEOUT_SECONDS=1800
|
||||
ASR_WORKER_CONCURRENCY=1
|
||||
ASR_JOB_LEASE_SECONDS=3600
|
||||
```
|
||||
|
||||
首次准备 Python 环境:
|
||||
|
||||
```bash
|
||||
cd asr-service
|
||||
python3 -m venv .venv
|
||||
.venv/bin/python -m pip install --upgrade pip
|
||||
.venv/bin/python -m pip install -r requirements.txt
|
||||
cd ..
|
||||
```
|
||||
|
||||
模型首次启动会下载约 2.1 GB 到 `asr-service/.models`。`.venv` 和 `.models` 均已被 Git 和微信小程序打包忽略。
|
||||
|
||||
## 4. 启动全部服务
|
||||
|
||||
### 4.1 执行数据库迁移
|
||||
|
||||
每次拉取到新迁移后执行一次:
|
||||
|
||||
```bash
|
||||
cd server
|
||||
cargo run --bin migrate
|
||||
cd ..
|
||||
```
|
||||
|
||||
看到 `PostgreSQL migrations completed` 表示成功。
|
||||
|
||||
### 4.2 启动 FunASR
|
||||
|
||||
终端一:
|
||||
|
||||
```bash
|
||||
cd asr-service
|
||||
MODELSCOPE_CACHE=.models .venv/bin/uvicorn app:app \
|
||||
--host 127.0.0.1 \
|
||||
--port 10095 \
|
||||
--no-access-log
|
||||
```
|
||||
|
||||
另一个终端检查:
|
||||
|
||||
```bash
|
||||
curl http://127.0.0.1:10095/health
|
||||
```
|
||||
|
||||
预期结果:
|
||||
|
||||
```json
|
||||
{"status":"ok","model":"paraformer-zh","device":"cpu"}
|
||||
```
|
||||
|
||||
首次下载或缓存加载期间端口可能暂不可用,应等待模型完全加载。
|
||||
|
||||
Docker 方式可替代本机 Python:
|
||||
|
||||
```bash
|
||||
docker compose -f compose.asr.yml up --build -d
|
||||
docker compose -f compose.asr.yml logs -f funasr
|
||||
```
|
||||
|
||||
### 4.3 启动 Rust API
|
||||
|
||||
终端二:
|
||||
|
||||
```bash
|
||||
cd server
|
||||
cargo run
|
||||
```
|
||||
|
||||
检查完整链路:
|
||||
|
||||
```bash
|
||||
curl http://127.0.0.1:8080/health | jq .
|
||||
```
|
||||
|
||||
正常状态应包含:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"database_configured": true,
|
||||
"speech_configured": true,
|
||||
"speech_available": true,
|
||||
"speech_provider": "local-funasr"
|
||||
}
|
||||
```
|
||||
|
||||
接口文档:<http://127.0.0.1:8080/scalar>
|
||||
|
||||
### 4.4 启动微信开发者工具
|
||||
|
||||
终端三,在项目根目录执行:
|
||||
|
||||
```bash
|
||||
cli open \
|
||||
--project /Users/zhangheng/project/ai/miniprogram/teaching-feedback-assistant \
|
||||
--port 63097 \
|
||||
--lang zh
|
||||
```
|
||||
|
||||
自动预览编译检查:
|
||||
|
||||
```bash
|
||||
cli auto-preview \
|
||||
--project /Users/zhangheng/project/ai/miniprogram/teaching-feedback-assistant \
|
||||
--port 63097 \
|
||||
--lang zh \
|
||||
--info-output /tmp/teaching-feedback-preview.json
|
||||
```
|
||||
|
||||
开发者工具模拟器默认访问 `http://127.0.0.1:8080`。进入小程序“我的”页面,应看到“服务、数据库和语音转录正常”。
|
||||
|
||||
## 5. 录音测试流程
|
||||
|
||||
### 短录音
|
||||
|
||||
1. 打开“反馈生成”。
|
||||
2. 点击“语音录入”,录制 10 至 30 秒中文。
|
||||
3. 点击“结束录音”。
|
||||
4. 页面短暂显示“正在处理”。
|
||||
5. 转写完成后文字应自动加入反馈内容。
|
||||
|
||||
短录音条件为不超过 60 秒,且转写文本未超过后端短文本限制。
|
||||
|
||||
### 长录音
|
||||
|
||||
1. 录制超过 60 秒的中文。
|
||||
2. 结束后等待状态变成“已就绪”。
|
||||
3. 主按钮变为“生成反馈”。
|
||||
4. 点击一次后,转写内容会汇总进反馈草稿。
|
||||
|
||||
### 多课节汇总
|
||||
|
||||
1. 录制并结束第一节课。
|
||||
2. 再次点击“语音录入”录制下一节。
|
||||
3. 重复完成多节录音。
|
||||
4. 所有课节就绪后点击一次“生成反馈”。
|
||||
|
||||
单节录音超过 8 分钟时,小程序会自动结束当前片段并立即开始下一片段,用户无需操作。
|
||||
|
||||
## 6. 故障恢复测试
|
||||
|
||||
### FunASR 未启动
|
||||
|
||||
停止 FunASR 后录音,Rust API 仍会保存音频,页面保持“正在处理”。重新启动 FunASR 后,后台队列会自动继续。
|
||||
|
||||
### Rust 在推理中退出
|
||||
|
||||
任务使用租约避免多个工作线程重复处理。默认租约为 3600 秒。调试崩溃恢复时,可在 `server/.env` 临时设置:
|
||||
|
||||
```env
|
||||
ASR_JOB_LEASE_SECONDS=60
|
||||
```
|
||||
|
||||
重新启动 Rust API 后,超出租约的任务会再次被领取。生产环境应使用足够覆盖最长片段推理时间的租约。
|
||||
|
||||
### 转写失败
|
||||
|
||||
点击页面中的语音状态,可查看课节状态并重试失败片段。音频保存在 `server/data/audio`,转写失败不会立即删除原文件。
|
||||
|
||||
## 7. 常见问题
|
||||
|
||||
### `speech_configured=true` 但 `speech_available=false`
|
||||
|
||||
- FunASR 尚未启动或模型仍在加载。
|
||||
- `10095` 端口被占用。
|
||||
- `LOCAL_ASR_URL` 配置错误。
|
||||
|
||||
检查:
|
||||
|
||||
```bash
|
||||
lsof -nP -iTCP:10095 -sTCP:LISTEN
|
||||
curl http://127.0.0.1:10095/health
|
||||
```
|
||||
|
||||
### `database_configured=false` 或数据接口返回 503
|
||||
|
||||
检查 `server/.env` 中的 `DATABASE_URL`,然后重新执行迁移并启动 API。
|
||||
|
||||
### 微信模拟器无法连接 API
|
||||
|
||||
```bash
|
||||
lsof -nP -iTCP:8080 -sTCP:LISTEN
|
||||
curl http://127.0.0.1:8080/health
|
||||
```
|
||||
|
||||
再到小程序“我的”页面确认 API 地址为 `http://127.0.0.1:8080`。
|
||||
|
||||
### 真机无法访问 `127.0.0.1`
|
||||
|
||||
真机中的 `127.0.0.1` 指手机自身。局域网调试时:
|
||||
|
||||
1. 将 `server/.env` 的 `HOST` 改为 `0.0.0.0`。
|
||||
2. 确认手机和电脑连接同一网络。
|
||||
3. 在“我的”页面将 API 地址改为 `http://<电脑局域网IP>:8080`。
|
||||
4. 检查系统防火墙和微信开发者工具的域名校验设置。
|
||||
|
||||
正式环境必须使用微信允许的 HTTPS 合法域名。
|
||||
|
||||
## 8. 代码检查
|
||||
|
||||
小程序:
|
||||
|
||||
```bash
|
||||
npm run typecheck
|
||||
```
|
||||
|
||||
Rust:
|
||||
|
||||
```bash
|
||||
cd server
|
||||
cargo fmt --check
|
||||
cargo test
|
||||
cargo clippy --all-targets -- -D warnings
|
||||
```
|
||||
|
||||
FunASR 服务:
|
||||
|
||||
```bash
|
||||
python3 -m py_compile asr-service/app.py
|
||||
asr-service/.venv/bin/python -m pip check
|
||||
```
|
||||
|
||||
## 9. 停止服务
|
||||
|
||||
前台运行时,在 FunASR 和 Rust API 各自终端按 `Ctrl+C`。
|
||||
|
||||
Docker 方式:
|
||||
|
||||
```bash
|
||||
docker compose -f compose.asr.yml down
|
||||
```
|
||||
|
||||
确认端口已经释放:
|
||||
|
||||
```bash
|
||||
lsof -nP -iTCP:8080 -sTCP:LISTEN
|
||||
lsof -nP -iTCP:10095 -sTCP:LISTEN
|
||||
```
|
||||
|
||||
正常停止服务不会删除 `asr-service/.models` 模型缓存。除非希望重新下载模型,否则不要删除该目录。
|
||||
12
README.md
12
README.md
@@ -11,6 +11,8 @@ cli open --project /Users/zhangheng/project/ai/miniprogram/teaching-feedback-ass
|
||||
|
||||
3. 在微信开发者工具内编译、预览和调试。
|
||||
|
||||
反馈生成页使用小程序原生录音能力。短录音会直接追加到反馈内容,长录音每 8 分钟自动切片,多节录音可以汇总为一次反馈。真机首次使用时需要允许麦克风权限;中文转录默认使用项目自带的本地 FunASR 服务,不要求腾讯云密钥。
|
||||
|
||||
小程序默认连接 `http://127.0.0.1:8080`。在开发者工具模拟器中,可进入“我的”页面保存其他 API 地址并检测服务状态。
|
||||
|
||||
## 验证
|
||||
@@ -33,6 +35,15 @@ cargo run --bin migrate
|
||||
cargo run
|
||||
```
|
||||
|
||||
另开一个终端启动中文转录服务。首次启动会下载 Paraformer、VAD 和标点模型,下载完成后健康检查才会显示语音服务可用:
|
||||
|
||||
```bash
|
||||
docker compose -f compose.asr.yml up --build -d
|
||||
curl http://127.0.0.1:10095/health
|
||||
```
|
||||
|
||||
没有 Docker 时可按 [asr-service/README.md](./asr-service/README.md) 使用 Python 3.10 启动。Rust API 默认连接 `http://127.0.0.1:10095`;模型未就绪时已上传录音会保留在队列中,不会阻塞小程序上传。
|
||||
|
||||
未设置 `DATABASE_URL` 时,仅健康检查可用,数据接口会返回 `503`。数据库配置和迁移说明参见 [server/API.md](./server/API.md)。
|
||||
|
||||
服务启动后可打开 `http://127.0.0.1:8080/scalar` 查看并直接调用 Scalar 交互文档,原始规范位于 `http://127.0.0.1:8080/openapi.json`。完整测试顺序参见 [server/API_GUIDE.md](./server/API_GUIDE.md)。
|
||||
@@ -40,6 +51,7 @@ cargo run
|
||||
## 小程序与后端
|
||||
|
||||
- 学生档案、档案预设和反馈记录均通过 `utils/api.ts` 访问 Rust API。
|
||||
- 反馈页只暴露一个语音入口;反馈批次、课节、录音片段和转录状态由后端自动维护。
|
||||
- 开发者工具模拟器使用默认地址 `http://127.0.0.1:8080`。
|
||||
- 真机和正式版本不能把 `127.0.0.1` 当作电脑地址;应部署可公网访问的 HTTPS API,并在微信公众平台配置 request 合法域名,然后到小程序“我的”页面修改服务地址。
|
||||
- 微信登录尚未接入。业务请求暂时使用固定开发用户 UUID;生产发布前必须改为由后端根据微信登录凭据解析用户身份。
|
||||
|
||||
5
app.json
5
app.json
@@ -13,6 +13,11 @@
|
||||
"backgroundColor": "#F4F8FE",
|
||||
"backgroundTextStyle": "light"
|
||||
},
|
||||
"permission": {
|
||||
"scope.record": {
|
||||
"desc": "用于将课堂反馈录音转成文字"
|
||||
}
|
||||
},
|
||||
"tabBar": {
|
||||
"custom": true,
|
||||
"color": "#7B8796",
|
||||
|
||||
4
asr-service/.dockerignore
Normal file
4
asr-service/.dockerignore
Normal file
@@ -0,0 +1,4 @@
|
||||
__pycache__/
|
||||
*.pyc
|
||||
.venv/
|
||||
.pytest_cache/
|
||||
20
asr-service/Dockerfile
Normal file
20
asr-service/Dockerfile
Normal file
@@ -0,0 +1,20 @@
|
||||
FROM python:3.10-slim
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
PIP_NO_CACHE_DIR=1
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends ffmpeg libsndfile1 \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /app
|
||||
COPY requirements.txt ./
|
||||
RUN pip install --upgrade pip \
|
||||
&& pip install torch==2.8.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cpu \
|
||||
&& pip install -r requirements.txt
|
||||
|
||||
COPY app.py ./
|
||||
|
||||
EXPOSE 10095
|
||||
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "10095", "--workers", "1", "--no-access-log"]
|
||||
38
asr-service/README.md
Normal file
38
asr-service/README.md
Normal file
@@ -0,0 +1,38 @@
|
||||
# 中文录音转写服务
|
||||
|
||||
该服务使用 FunASR 的中文 Paraformer、FSMN-VAD 和标点模型,为 Rust API 提供本地录音转写。模型首次启动时会下载约 2.1 GB 到持久化缓存,完成后 `/health` 才会可用。
|
||||
|
||||
## Docker 启动
|
||||
|
||||
在项目根目录执行:
|
||||
|
||||
```bash
|
||||
docker compose -f compose.asr.yml up --build -d
|
||||
curl http://127.0.0.1:10095/health
|
||||
```
|
||||
|
||||
默认使用 CPU 且只允许一个并发推理任务。可通过 `ASR_DEVICE`、`ASR_MAX_CONCURRENCY`、`ASR_MODEL`、`ASR_VAD_MODEL` 和 `ASR_PUNC_MODEL` 调整。GPU 部署需要使用带 CUDA 的 PyTorch 基础环境,不能只把 CPU 镜像中的 `ASR_DEVICE` 改为 `cuda`。
|
||||
|
||||
## 本机 Python 启动
|
||||
|
||||
支持 Python 3.9 及以上版本,推荐 Python 3.10:
|
||||
|
||||
```bash
|
||||
cd asr-service
|
||||
python3.10 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
MODELSCOPE_CACHE=.models uvicorn app:app --host 127.0.0.1 --port 10095 --no-access-log
|
||||
```
|
||||
|
||||
## 接口
|
||||
|
||||
Rust API 调用 `POST /v1/transcriptions`,以 multipart 上传 `file`、`language` 和 `duration_ms`,响应格式为:
|
||||
|
||||
```json
|
||||
{
|
||||
"text": "识别后的中文课堂内容。",
|
||||
"duration_ms": 480000,
|
||||
"request_id": "b2f6fc1c-62c1-4a99-8ab4-6f933f1bd252"
|
||||
}
|
||||
```
|
||||
173
asr-service/app.py
Normal file
173
asr-service/app.py
Normal file
@@ -0,0 +1,173 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
import tempfile
|
||||
import uuid
|
||||
from contextlib import asynccontextmanager
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from fastapi import FastAPI, File, Form, HTTPException, UploadFile
|
||||
from funasr import AutoModel
|
||||
from pydantic import BaseModel
|
||||
|
||||
LOGGER = logging.getLogger("funasr-service")
|
||||
MAX_AUDIO_BYTES = 15 * 1024 * 1024
|
||||
CHUNK_BYTES = 1024 * 1024
|
||||
SUPPORTED_FORMATS = {"mp3", "aac", "wav", "m4a", "amr"}
|
||||
|
||||
|
||||
class Settings:
|
||||
def __init__(self) -> None:
|
||||
self.model = os.getenv("ASR_MODEL", "paraformer-zh")
|
||||
self.vad_model = os.getenv("ASR_VAD_MODEL", "fsmn-vad")
|
||||
self.punc_model = os.getenv("ASR_PUNC_MODEL", "ct-punc")
|
||||
self.device = os.getenv("ASR_DEVICE", "cpu")
|
||||
self.max_concurrency = positive_int("ASR_MAX_CONCURRENCY", 1)
|
||||
self.batch_size_seconds = positive_int("ASR_BATCH_SIZE_SECONDS", 300)
|
||||
|
||||
|
||||
class HealthResponse(BaseModel):
|
||||
status: str
|
||||
model: str
|
||||
device: str
|
||||
|
||||
|
||||
class TranscriptionResponse(BaseModel):
|
||||
text: str
|
||||
duration_ms: int
|
||||
request_id: str
|
||||
|
||||
|
||||
def positive_int(name: str, default: int) -> int:
|
||||
raw = os.getenv(name, str(default))
|
||||
try:
|
||||
value = int(raw)
|
||||
except ValueError as error:
|
||||
raise RuntimeError(f"{name} must be an integer") from error
|
||||
if value <= 0:
|
||||
raise RuntimeError(f"{name} must be greater than 0")
|
||||
return value
|
||||
|
||||
|
||||
def load_model(settings: Settings) -> Any:
|
||||
LOGGER.info(
|
||||
"loading FunASR model model=%s vad=%s punc=%s device=%s",
|
||||
settings.model,
|
||||
settings.vad_model,
|
||||
settings.punc_model,
|
||||
settings.device,
|
||||
)
|
||||
model = AutoModel(
|
||||
model=settings.model,
|
||||
vad_model=settings.vad_model,
|
||||
punc_model=settings.punc_model,
|
||||
device=settings.device,
|
||||
disable_update=True,
|
||||
)
|
||||
LOGGER.info("FunASR model is ready")
|
||||
return model
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
settings = Settings()
|
||||
app.state.settings = settings
|
||||
app.state.model = await asyncio.to_thread(load_model, settings)
|
||||
app.state.inference_slots = asyncio.Semaphore(settings.max_concurrency)
|
||||
yield
|
||||
|
||||
|
||||
app = FastAPI(
|
||||
title="Teaching Feedback FunASR Service",
|
||||
version="1.0.0",
|
||||
lifespan=lifespan,
|
||||
)
|
||||
|
||||
|
||||
@app.get("/health", response_model=HealthResponse)
|
||||
async def health() -> HealthResponse:
|
||||
settings: Settings = app.state.settings
|
||||
return HealthResponse(status="ok", model=settings.model, device=settings.device)
|
||||
|
||||
|
||||
@app.post("/v1/transcriptions", response_model=TranscriptionResponse)
|
||||
async def create_transcription(
|
||||
file: UploadFile = File(...),
|
||||
language: str = Form("zh"),
|
||||
duration_ms: int = Form(0),
|
||||
hotword: str = Form(""),
|
||||
) -> TranscriptionResponse:
|
||||
if language not in {"zh", "zh-cn", "cmn"}:
|
||||
raise HTTPException(status_code=422, detail="this service is configured for Chinese audio")
|
||||
audio_format = file_extension(file.filename)
|
||||
temporary_path = await save_upload(file, audio_format)
|
||||
request_id = str(uuid.uuid4())
|
||||
try:
|
||||
async with app.state.inference_slots:
|
||||
text = await asyncio.to_thread(
|
||||
run_inference,
|
||||
app.state.model,
|
||||
temporary_path,
|
||||
app.state.settings.batch_size_seconds,
|
||||
hotword.strip(),
|
||||
)
|
||||
except Exception as error:
|
||||
LOGGER.exception("FunASR inference failed request_id=%s", request_id)
|
||||
raise HTTPException(status_code=500, detail=f"speech recognition failed: {error}") from error
|
||||
finally:
|
||||
temporary_path.unlink(missing_ok=True)
|
||||
|
||||
if not text:
|
||||
raise HTTPException(status_code=422, detail="no clear speech was recognized")
|
||||
return TranscriptionResponse(
|
||||
text=text,
|
||||
duration_ms=max(0, duration_ms),
|
||||
request_id=request_id,
|
||||
)
|
||||
|
||||
|
||||
def file_extension(filename: str | None) -> str:
|
||||
suffix = Path(filename or "segment.mp3").suffix.lower().lstrip(".")
|
||||
if suffix not in SUPPORTED_FORMATS:
|
||||
raise HTTPException(status_code=415, detail="unsupported audio format")
|
||||
return suffix
|
||||
|
||||
|
||||
async def save_upload(file: UploadFile, audio_format: str) -> Path:
|
||||
handle = tempfile.NamedTemporaryFile(suffix=f".{audio_format}", delete=False)
|
||||
path = Path(handle.name)
|
||||
total_bytes = 0
|
||||
try:
|
||||
with handle:
|
||||
while chunk := await file.read(CHUNK_BYTES):
|
||||
total_bytes += len(chunk)
|
||||
if total_bytes > MAX_AUDIO_BYTES:
|
||||
raise HTTPException(status_code=413, detail="audio file exceeds 15 MB")
|
||||
handle.write(chunk)
|
||||
except Exception:
|
||||
path.unlink(missing_ok=True)
|
||||
raise
|
||||
finally:
|
||||
await file.close()
|
||||
if total_bytes == 0:
|
||||
path.unlink(missing_ok=True)
|
||||
raise HTTPException(status_code=422, detail="audio file is empty")
|
||||
return path
|
||||
|
||||
|
||||
def run_inference(model: Any, path: Path, batch_size_seconds: int, hotword: str) -> str:
|
||||
options: dict[str, Any] = {
|
||||
"input": str(path),
|
||||
"batch_size_s": batch_size_seconds,
|
||||
"use_itn": True,
|
||||
}
|
||||
if hotword:
|
||||
options["hotword"] = hotword
|
||||
result = model.generate(**options)
|
||||
if not isinstance(result, list):
|
||||
return ""
|
||||
texts = [str(item.get("text", "")).strip() for item in result if isinstance(item, dict)]
|
||||
return "\n".join(text for text in texts if text).strip()
|
||||
7
asr-service/requirements.txt
Normal file
7
asr-service/requirements.txt
Normal file
@@ -0,0 +1,7 @@
|
||||
fastapi==0.128.8
|
||||
funasr==1.3.14
|
||||
modelscope==1.37.1
|
||||
python-multipart==0.0.20
|
||||
torch==2.8.0
|
||||
torchaudio==2.8.0
|
||||
uvicorn[standard]==0.39.0
|
||||
31
compose.asr.yml
Normal file
31
compose.asr.yml
Normal file
@@ -0,0 +1,31 @@
|
||||
services:
|
||||
funasr:
|
||||
build:
|
||||
context: ./asr-service
|
||||
ports:
|
||||
- "127.0.0.1:10095:10095"
|
||||
environment:
|
||||
ASR_MODEL: paraformer-zh
|
||||
ASR_VAD_MODEL: fsmn-vad
|
||||
ASR_PUNC_MODEL: ct-punc
|
||||
ASR_DEVICE: cpu
|
||||
ASR_MAX_CONCURRENCY: "1"
|
||||
ASR_BATCH_SIZE_SECONDS: "300"
|
||||
MODELSCOPE_CACHE: /models
|
||||
volumes:
|
||||
- funasr-model-cache:/models
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test:
|
||||
- CMD
|
||||
- python
|
||||
- -c
|
||||
- "import urllib.request; urllib.request.urlopen('http://127.0.0.1:10095/health', timeout=3)"
|
||||
interval: 15s
|
||||
timeout: 5s
|
||||
retries: 20
|
||||
start_period: 5m
|
||||
shm_size: 1gb
|
||||
|
||||
volumes:
|
||||
funasr-model-cache:
|
||||
@@ -1,13 +1,54 @@
|
||||
import {
|
||||
createFeedbackRecord,
|
||||
createVoiceLesson,
|
||||
ensureFeedbackSession,
|
||||
finishVoiceLesson,
|
||||
generateFeedbackFromVoice,
|
||||
getActiveFeedbackSession,
|
||||
getApiErrorMessage,
|
||||
listProfiles
|
||||
listProfiles,
|
||||
markVoiceLessonApplied,
|
||||
retryVoiceSegment,
|
||||
updateFeedbackSession,
|
||||
uploadVoiceSegment
|
||||
} from '../../utils/api'
|
||||
import type { FeedbackDraft, StudentProfile } from '../../utils/types'
|
||||
import type {
|
||||
FeedbackDraft,
|
||||
FeedbackSession,
|
||||
StudentProfile,
|
||||
VoiceLessonFinished
|
||||
} from '../../utils/types'
|
||||
|
||||
type PickerEvent = { detail: { value: string | number } }
|
||||
type InputEvent = { detail: { value: string } }
|
||||
|
||||
interface PendingVoiceUpload {
|
||||
lessonId: string
|
||||
sequence: number
|
||||
durationMs: number
|
||||
filePath: string
|
||||
}
|
||||
|
||||
const MAX_CONTENT_LENGTH = 2000
|
||||
const RECORDING_SEGMENT_DURATION_MS = 8 * 60 * 1000
|
||||
const SHORT_RECORDING_DURATION_MS = 60 * 1000
|
||||
const VOICE_POLL_INTERVAL_MS = 3000
|
||||
const PENDING_UPLOADS_KEY = 'teaching-feedback-pending-voice-uploads-v1'
|
||||
|
||||
const recorderManager = wx.getRecorderManager()
|
||||
let recordingTimer: number | null = null
|
||||
let draftSaveTimer: number | null = null
|
||||
let voicePollTimer: number | null = null
|
||||
let voicePollBusy = false
|
||||
let pageVisible = false
|
||||
let lessonStartedAt = 0
|
||||
let segmentStartedAt = 0
|
||||
let currentLessonId: string | null = null
|
||||
let currentSegmentSequence = 0
|
||||
let finishRequested = false
|
||||
let pendingUploads: Promise<void>[] = []
|
||||
let failedLocalUploads: PendingVoiceUpload[] = []
|
||||
|
||||
function today(): string {
|
||||
const now = new Date()
|
||||
const year = now.getFullYear()
|
||||
@@ -20,39 +61,147 @@ function buildDraft(): FeedbackDraft {
|
||||
return {
|
||||
profileId: null,
|
||||
feedbackDate: today(),
|
||||
content: ''
|
||||
content: '',
|
||||
feedbackSessionId: null
|
||||
}
|
||||
}
|
||||
|
||||
function appendTranscript(content: string, transcript: string): { content: string; truncated: boolean } {
|
||||
const normalizedTranscript = transcript.trim()
|
||||
const separator = content && !/\s$/.test(content) ? '\n' : ''
|
||||
const combined = `${content}${separator}${normalizedTranscript}`
|
||||
return {
|
||||
content: combined.slice(0, MAX_CONTENT_LENGTH),
|
||||
truncated: combined.length > MAX_CONTENT_LENGTH
|
||||
}
|
||||
}
|
||||
|
||||
function formatRecordingTime(milliseconds: number): string {
|
||||
const totalSeconds = Math.max(0, Math.floor(milliseconds / 1000))
|
||||
const hours = Math.floor(totalSeconds / 3600)
|
||||
const minutes = Math.floor((totalSeconds % 3600) / 60)
|
||||
const seconds = totalSeconds % 60
|
||||
const minuteText = String(minutes).padStart(2, '0')
|
||||
const secondText = String(seconds).padStart(2, '0')
|
||||
return hours > 0 ? `${String(hours).padStart(2, '0')}:${minuteText}:${secondText}` : `${minuteText}:${secondText}`
|
||||
}
|
||||
|
||||
function formatVoiceDuration(milliseconds: number): string {
|
||||
const totalMinutes = Math.floor(milliseconds / 60000)
|
||||
if (totalMinutes < 1) return `${Math.max(1, Math.round(milliseconds / 1000))}秒`
|
||||
return `${totalMinutes}分钟`
|
||||
}
|
||||
|
||||
function wait(milliseconds: number): Promise<void> {
|
||||
return new Promise((resolve) => setTimeout(resolve, milliseconds))
|
||||
}
|
||||
|
||||
Page({
|
||||
data: {
|
||||
profiles: [] as StudentProfile[],
|
||||
profileOptions: ['不关联学生'],
|
||||
selectedProfileIndex: 0,
|
||||
draft: buildDraft(),
|
||||
session: null as FeedbackSession | null,
|
||||
contentLength: 0,
|
||||
loading: false,
|
||||
saving: false,
|
||||
generating: false,
|
||||
voiceBusy: false,
|
||||
recording: false,
|
||||
recordingPaused: false,
|
||||
recordingTime: '00:00',
|
||||
voiceStatus: '语音录入',
|
||||
voiceActionable: false,
|
||||
primaryActionText: '保存反馈',
|
||||
errorMessage: ''
|
||||
},
|
||||
|
||||
onLoad() {
|
||||
failedLocalUploads = (wx.getStorageSync(PENDING_UPLOADS_KEY) as PendingVoiceUpload[]) || []
|
||||
this.initRecorder()
|
||||
if (failedLocalUploads.length > 0) {
|
||||
currentLessonId = failedLocalUploads[0].lessonId
|
||||
this.setData({ voiceStatus: '上传失败,点击重试', voiceActionable: true })
|
||||
}
|
||||
},
|
||||
|
||||
onShow() {
|
||||
pageVisible = true
|
||||
this.getTabBar()?.setData({ selected: 0 })
|
||||
void this.loadProfiles()
|
||||
},
|
||||
|
||||
onHide() {
|
||||
pageVisible = false
|
||||
this.clearVoicePollTimer()
|
||||
if (this.data.recording) this.stopRecording()
|
||||
},
|
||||
|
||||
onUnload() {
|
||||
pageVisible = false
|
||||
this.clearRecordingTimer()
|
||||
this.clearDraftSaveTimer()
|
||||
this.clearVoicePollTimer()
|
||||
if (this.data.recording) recorderManager.stop()
|
||||
},
|
||||
|
||||
initRecorder() {
|
||||
recorderManager.onStart(() => {
|
||||
segmentStartedAt = Date.now()
|
||||
this.setData({
|
||||
voiceBusy: false,
|
||||
recording: true,
|
||||
recordingPaused: false,
|
||||
voiceStatus: '正在录音'
|
||||
})
|
||||
this.startRecordingTimer()
|
||||
})
|
||||
|
||||
recorderManager.onStop((result) => {
|
||||
void this.handleRecorderStop(result)
|
||||
})
|
||||
|
||||
recorderManager.onPause(() => {
|
||||
this.setData({ recordingPaused: true, voiceStatus: '录音已暂停' })
|
||||
})
|
||||
|
||||
recorderManager.onResume(() => {
|
||||
this.setData({ recordingPaused: false, voiceStatus: '正在录音' })
|
||||
})
|
||||
|
||||
recorderManager.onInterruptionEnd(() => {
|
||||
if (this.data.recordingPaused) recorderManager.resume()
|
||||
})
|
||||
|
||||
recorderManager.onError((error) => {
|
||||
console.warn('RecorderManager error', error)
|
||||
this.clearRecordingTimer()
|
||||
finishRequested = true
|
||||
this.setData({
|
||||
recording: false,
|
||||
recordingPaused: false,
|
||||
voiceBusy: true,
|
||||
voiceStatus: '正在保留已录内容'
|
||||
})
|
||||
void this.finalizeCurrentLesson()
|
||||
})
|
||||
},
|
||||
|
||||
async loadProfiles() {
|
||||
this.setData({ loading: true, errorMessage: '' })
|
||||
try {
|
||||
const profiles = await listProfiles()
|
||||
const matchedProfileIndex = profiles.findIndex((profile) => profile.id === this.data.draft.profileId)
|
||||
const selectedProfileIndex = matchedProfileIndex + 1
|
||||
const profileId = matchedProfileIndex >= 0 ? profiles[matchedProfileIndex].id : null
|
||||
this.setData({
|
||||
profiles,
|
||||
profileOptions: ['不关联学生', ...profiles.map((profile) => `${profile.name} · ${profile.subject}`)],
|
||||
selectedProfileIndex,
|
||||
'draft.profileId': matchedProfileIndex >= 0 ? profiles[matchedProfileIndex].id : null
|
||||
'draft.profileId': profileId
|
||||
})
|
||||
await this.loadActiveSession(profileId)
|
||||
} catch (error) {
|
||||
this.setData({ errorMessage: getApiErrorMessage(error) })
|
||||
} finally {
|
||||
@@ -60,29 +209,483 @@ Page({
|
||||
}
|
||||
},
|
||||
|
||||
async loadActiveSession(profileId?: string | null) {
|
||||
try {
|
||||
const targetProfileId = profileId === undefined ? this.data.draft.profileId : profileId
|
||||
const session = await getActiveFeedbackSession(targetProfileId)
|
||||
if (targetProfileId !== this.data.draft.profileId) return
|
||||
const nextData: Record<string, unknown> = {
|
||||
session,
|
||||
'draft.feedbackSessionId': session?.id || null
|
||||
}
|
||||
if (session) {
|
||||
nextData['draft.feedbackDate'] = session.feedbackDate
|
||||
nextData['draft.content'] = session.content
|
||||
nextData.contentLength = session.content.length
|
||||
}
|
||||
this.setData(nextData)
|
||||
this.syncVoicePresentation(session)
|
||||
this.scheduleVoicePolling(session)
|
||||
} catch (error) {
|
||||
this.setData({ errorMessage: getApiErrorMessage(error) })
|
||||
}
|
||||
},
|
||||
|
||||
syncVoicePresentation(session: FeedbackSession | null) {
|
||||
if (this.data.recording || this.data.voiceBusy) return
|
||||
if (failedLocalUploads.length > 0) {
|
||||
this.setData({ voiceStatus: '上传失败,点击重试', voiceActionable: true })
|
||||
return
|
||||
}
|
||||
if (!session || session.lessonCount === 0) {
|
||||
this.setData({ voiceStatus: '语音录入', voiceActionable: false, primaryActionText: '保存反馈' })
|
||||
return
|
||||
}
|
||||
|
||||
const prefix = `${session.lessonCount}节 · ${formatVoiceDuration(session.totalDurationMs)}`
|
||||
let suffix = '已就绪'
|
||||
if (session.failedSegmentCount > 0) suffix = '转录失败,点击重试'
|
||||
else if (session.processingSegmentCount > 0) suffix = '正在处理'
|
||||
let primaryActionText = session.needsGeneration ? '生成反馈' : '保存反馈'
|
||||
if (session.processingSegmentCount > 0) primaryActionText = '转录中'
|
||||
this.setData({
|
||||
voiceStatus: `${prefix} · ${suffix}`,
|
||||
voiceActionable: true,
|
||||
primaryActionText
|
||||
})
|
||||
},
|
||||
|
||||
scheduleVoicePolling(session: FeedbackSession | null) {
|
||||
this.clearVoicePollTimer()
|
||||
if (!pageVisible || !session || session.processingSegmentCount === 0) return
|
||||
voicePollTimer = setTimeout(() => {
|
||||
voicePollTimer = null
|
||||
void this.pollVoiceSession()
|
||||
}, VOICE_POLL_INTERVAL_MS) as unknown as number
|
||||
},
|
||||
|
||||
clearVoicePollTimer() {
|
||||
if (voicePollTimer === null) return
|
||||
clearTimeout(voicePollTimer)
|
||||
voicePollTimer = null
|
||||
},
|
||||
|
||||
async pollVoiceSession() {
|
||||
if (!pageVisible) return
|
||||
if (voicePollBusy || this.data.recording || this.data.voiceBusy) {
|
||||
this.scheduleVoicePolling(this.data.session)
|
||||
return
|
||||
}
|
||||
voicePollBusy = true
|
||||
const profileId = this.data.draft.profileId
|
||||
try {
|
||||
const session = await getActiveFeedbackSession(profileId)
|
||||
if (!pageVisible || profileId !== this.data.draft.profileId) return
|
||||
this.setData({ session, 'draft.feedbackSessionId': session?.id || null })
|
||||
this.syncVoicePresentation(session)
|
||||
if (!session) return
|
||||
|
||||
const readyShortLessons = session.lessons.filter(
|
||||
(lesson) =>
|
||||
lesson.status === 'ready' &&
|
||||
lesson.durationMs <= SHORT_RECORDING_DURATION_MS &&
|
||||
!lesson.appliedDirectly &&
|
||||
!lesson.includedInGeneration
|
||||
)
|
||||
for (const lesson of readyShortLessons) {
|
||||
const finished = await finishVoiceLesson(lesson.id)
|
||||
await this.applyFinishedLesson(finished)
|
||||
}
|
||||
if (readyShortLessons.length > 0) await this.loadActiveSession(profileId)
|
||||
} catch (error) {
|
||||
console.warn('Failed to refresh voice transcription status', error)
|
||||
} finally {
|
||||
voicePollBusy = false
|
||||
this.scheduleVoicePolling(this.data.session)
|
||||
}
|
||||
},
|
||||
|
||||
retryLoad() {
|
||||
void this.loadProfiles()
|
||||
},
|
||||
|
||||
onProfileChange(event: PickerEvent) {
|
||||
this.clearDraftSaveTimer()
|
||||
const selectedProfileIndex = Number(event.detail.value)
|
||||
const profile = this.data.profiles[selectedProfileIndex - 1]
|
||||
const profileId = profile?.id || null
|
||||
this.setData({
|
||||
selectedProfileIndex,
|
||||
'draft.profileId': profile?.id || null
|
||||
'draft.profileId': profileId,
|
||||
session: null,
|
||||
'draft.feedbackSessionId': null,
|
||||
'draft.content': '',
|
||||
contentLength: 0,
|
||||
voiceActionable: false,
|
||||
primaryActionText: '保存反馈'
|
||||
})
|
||||
void this.loadActiveSession(profileId)
|
||||
},
|
||||
|
||||
onDateChange(event: InputEvent) {
|
||||
this.setData({ 'draft.feedbackDate': event.detail.value })
|
||||
this.scheduleDraftSave()
|
||||
},
|
||||
|
||||
onContentInput(event: InputEvent) {
|
||||
const content = event.detail.value
|
||||
this.setData({ 'draft.content': content, contentLength: content.length })
|
||||
this.scheduleDraftSave()
|
||||
},
|
||||
|
||||
scheduleDraftSave() {
|
||||
this.clearDraftSaveTimer()
|
||||
if (!this.data.session) return
|
||||
draftSaveTimer = setTimeout(() => {
|
||||
void this.persistSessionDraft()
|
||||
}, 800) as unknown as number
|
||||
},
|
||||
|
||||
clearDraftSaveTimer() {
|
||||
if (draftSaveTimer === null) return
|
||||
clearTimeout(draftSaveTimer)
|
||||
draftSaveTimer = null
|
||||
},
|
||||
|
||||
async persistSessionDraft() {
|
||||
const session = this.data.session
|
||||
if (!session) return
|
||||
const sessionId = session.id
|
||||
try {
|
||||
const updated = await updateFeedbackSession(
|
||||
sessionId,
|
||||
this.data.draft.feedbackDate,
|
||||
this.data.draft.content
|
||||
)
|
||||
if (this.data.session?.id !== sessionId) return
|
||||
this.setData({ session: updated })
|
||||
this.syncVoicePresentation(updated)
|
||||
} catch (error) {
|
||||
console.warn('Failed to auto-save feedback session', error)
|
||||
}
|
||||
},
|
||||
|
||||
ensureRecordPermission(): Promise<boolean> {
|
||||
return new Promise((resolve) => {
|
||||
wx.authorize({
|
||||
scope: 'scope.record',
|
||||
success: () => resolve(true),
|
||||
fail: () => {
|
||||
wx.showModal({
|
||||
title: '需要麦克风权限',
|
||||
content: '请在设置中允许使用麦克风,才能录音转录。',
|
||||
confirmText: '去设置',
|
||||
success: (modalResult) => {
|
||||
if (!modalResult.confirm) {
|
||||
resolve(false)
|
||||
return
|
||||
}
|
||||
wx.openSetting({
|
||||
success: (settingResult) => resolve(settingResult.authSetting['scope.record'] === true),
|
||||
fail: () => resolve(false)
|
||||
})
|
||||
},
|
||||
fail: () => resolve(false)
|
||||
})
|
||||
}
|
||||
})
|
||||
})
|
||||
},
|
||||
|
||||
async toggleRecording() {
|
||||
if (failedLocalUploads.length > 0) {
|
||||
await this.retryPendingUploads()
|
||||
return
|
||||
}
|
||||
if (this.data.recording) {
|
||||
this.stopRecording()
|
||||
return
|
||||
}
|
||||
|
||||
this.setData({ voiceBusy: true, voiceStatus: '正在开启麦克风' })
|
||||
const authorized = await this.ensureRecordPermission()
|
||||
if (!authorized) {
|
||||
this.setData({ voiceBusy: false })
|
||||
this.syncVoicePresentation(this.data.session)
|
||||
return
|
||||
}
|
||||
|
||||
try {
|
||||
const session = await ensureFeedbackSession(
|
||||
this.data.draft.profileId,
|
||||
this.data.draft.feedbackDate,
|
||||
this.data.draft.content
|
||||
)
|
||||
const lesson = await createVoiceLesson(session.id)
|
||||
currentLessonId = lesson.id
|
||||
currentSegmentSequence = 0
|
||||
finishRequested = false
|
||||
pendingUploads = []
|
||||
lessonStartedAt = Date.now()
|
||||
this.setData({
|
||||
session,
|
||||
'draft.feedbackSessionId': session.id,
|
||||
recordingTime: '00:00'
|
||||
})
|
||||
this.startNextRecordingSegment()
|
||||
} catch (error) {
|
||||
this.setData({ voiceBusy: false })
|
||||
this.syncVoicePresentation(this.data.session)
|
||||
wx.showToast({ title: getApiErrorMessage(error), icon: 'none' })
|
||||
}
|
||||
},
|
||||
|
||||
startNextRecordingSegment() {
|
||||
currentSegmentSequence += 1
|
||||
recorderManager.start({
|
||||
duration: RECORDING_SEGMENT_DURATION_MS,
|
||||
sampleRate: 16000,
|
||||
numberOfChannels: 1,
|
||||
encodeBitRate: 48000,
|
||||
format: 'mp3',
|
||||
audioSource: 'auto'
|
||||
})
|
||||
},
|
||||
|
||||
stopRecording() {
|
||||
if (!this.data.recording) return
|
||||
finishRequested = true
|
||||
this.clearRecordingTimer()
|
||||
this.setData({
|
||||
recording: false,
|
||||
recordingPaused: false,
|
||||
voiceBusy: true,
|
||||
voiceStatus: '正在整理录音'
|
||||
})
|
||||
recorderManager.stop()
|
||||
},
|
||||
|
||||
async handleRecorderStop(result: WechatMiniprogram.OnStopListenerResult) {
|
||||
const lessonId = currentLessonId
|
||||
if (!lessonId) return
|
||||
const durationMs = Math.max(1, result.duration || Date.now() - segmentStartedAt)
|
||||
const upload = this.queueSegmentUpload({
|
||||
lessonId,
|
||||
sequence: currentSegmentSequence,
|
||||
durationMs,
|
||||
filePath: result.tempFilePath
|
||||
})
|
||||
pendingUploads.push(upload)
|
||||
|
||||
if (!finishRequested) {
|
||||
this.startNextRecordingSegment()
|
||||
return
|
||||
}
|
||||
await this.finalizeCurrentLesson()
|
||||
},
|
||||
|
||||
async queueSegmentUpload(segment: PendingVoiceUpload): Promise<void> {
|
||||
try {
|
||||
await this.uploadWithRetry(segment)
|
||||
} catch (error) {
|
||||
console.warn('Voice segment upload failed', error)
|
||||
const saved = await this.persistRecordingFile(segment)
|
||||
failedLocalUploads.push(saved)
|
||||
wx.setStorageSync(PENDING_UPLOADS_KEY, failedLocalUploads)
|
||||
throw error
|
||||
}
|
||||
},
|
||||
|
||||
async uploadWithRetry(segment: PendingVoiceUpload): Promise<void> {
|
||||
let lastError: unknown
|
||||
for (let attempt = 0; attempt < 2; attempt += 1) {
|
||||
try {
|
||||
await uploadVoiceSegment(
|
||||
segment.lessonId,
|
||||
segment.filePath,
|
||||
segment.sequence,
|
||||
segment.durationMs
|
||||
)
|
||||
return
|
||||
} catch (error) {
|
||||
lastError = error
|
||||
if (attempt === 0) await wait(800)
|
||||
}
|
||||
}
|
||||
throw lastError
|
||||
},
|
||||
|
||||
persistRecordingFile(segment: PendingVoiceUpload): Promise<PendingVoiceUpload> {
|
||||
return new Promise((resolve) => {
|
||||
wx.getFileSystemManager().saveFile({
|
||||
tempFilePath: segment.filePath,
|
||||
success: (result) => resolve({ ...segment, filePath: result.savedFilePath }),
|
||||
fail: () => resolve(segment)
|
||||
})
|
||||
})
|
||||
},
|
||||
|
||||
async retryPendingUploads() {
|
||||
if (failedLocalUploads.length === 0) return
|
||||
this.setData({ voiceBusy: true, voiceStatus: '正在重试上传' })
|
||||
const remaining: PendingVoiceUpload[] = []
|
||||
for (const segment of failedLocalUploads) {
|
||||
try {
|
||||
await this.uploadWithRetry(segment)
|
||||
this.removeSavedFile(segment.filePath)
|
||||
} catch {
|
||||
remaining.push(segment)
|
||||
}
|
||||
}
|
||||
failedLocalUploads = remaining
|
||||
wx.setStorageSync(PENDING_UPLOADS_KEY, failedLocalUploads)
|
||||
if (remaining.length > 0) {
|
||||
this.setData({ voiceBusy: false, voiceStatus: '上传失败,点击重试', voiceActionable: true })
|
||||
wx.showToast({ title: '录音上传失败,请检查网络', icon: 'none' })
|
||||
return
|
||||
}
|
||||
await this.finalizeCurrentLesson()
|
||||
},
|
||||
|
||||
removeSavedFile(filePath: string) {
|
||||
wx.getFileSystemManager().unlink({ filePath, fail: () => undefined })
|
||||
},
|
||||
|
||||
async finalizeCurrentLesson() {
|
||||
const lessonId = currentLessonId
|
||||
if (!lessonId) {
|
||||
this.setData({ voiceBusy: false, recording: false })
|
||||
return
|
||||
}
|
||||
const results = await Promise.allSettled(pendingUploads)
|
||||
pendingUploads = []
|
||||
if (results.some((result) => result.status === 'rejected') || failedLocalUploads.length > 0) {
|
||||
this.setData({
|
||||
recording: false,
|
||||
voiceBusy: false,
|
||||
voiceStatus: '上传失败,点击重试',
|
||||
voiceActionable: true
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
try {
|
||||
const finished = await finishVoiceLesson(lessonId)
|
||||
await this.applyFinishedLesson(finished)
|
||||
currentLessonId = null
|
||||
currentSegmentSequence = 0
|
||||
finishRequested = false
|
||||
await this.loadActiveSession()
|
||||
} catch (error) {
|
||||
wx.showToast({ title: getApiErrorMessage(error), icon: 'none' })
|
||||
} finally {
|
||||
this.clearRecordingTimer()
|
||||
this.setData({ recording: false, recordingPaused: false, voiceBusy: false, recordingTime: '00:00' })
|
||||
this.syncVoicePresentation(this.data.session)
|
||||
}
|
||||
},
|
||||
|
||||
async applyFinishedLesson(finished: VoiceLessonFinished) {
|
||||
if (finished.status === 'processing') {
|
||||
wx.showToast({ title: '录音已上传,正在后台转写', icon: 'none' })
|
||||
return
|
||||
}
|
||||
if (finished.status === 'failed') {
|
||||
wx.showToast({ title: '部分录音转录失败,点击状态可重试', icon: 'none' })
|
||||
return
|
||||
}
|
||||
if (!finished.canApplyDirectly) {
|
||||
wx.showToast({ title: '录音已就绪,可生成反馈', icon: 'none' })
|
||||
return
|
||||
}
|
||||
|
||||
const merged = appendTranscript(this.data.draft.content, finished.transcript)
|
||||
this.setData({ 'draft.content': merged.content, contentLength: merged.content.length })
|
||||
if (!merged.truncated) {
|
||||
await markVoiceLessonApplied(finished.id, merged.content)
|
||||
wx.showToast({ title: '语音已转为文字', icon: 'none' })
|
||||
} else {
|
||||
wx.showToast({ title: '内容已达上限,请生成汇总反馈', icon: 'none' })
|
||||
}
|
||||
},
|
||||
|
||||
startRecordingTimer() {
|
||||
this.clearRecordingTimer()
|
||||
recordingTimer = setInterval(() => {
|
||||
this.setData({ recordingTime: formatRecordingTime(Date.now() - lessonStartedAt) })
|
||||
}, 1000) as unknown as number
|
||||
},
|
||||
|
||||
clearRecordingTimer() {
|
||||
if (recordingTimer === null) return
|
||||
clearInterval(recordingTimer)
|
||||
recordingTimer = null
|
||||
},
|
||||
|
||||
async openVoiceDetails() {
|
||||
if (this.data.recording || this.data.voiceBusy) return
|
||||
if (failedLocalUploads.length > 0) {
|
||||
await this.retryPendingUploads()
|
||||
return
|
||||
}
|
||||
const session = this.data.session
|
||||
if (!session || session.lessonCount === 0) return
|
||||
const lines = session.lessons.map((lesson) => {
|
||||
const status = lesson.status === 'ready' ? '已就绪' : lesson.status === 'failed' ? '转录失败' : '处理中'
|
||||
return `第${lesson.sequence}节 · ${formatVoiceDuration(lesson.durationMs)} · ${status}`
|
||||
})
|
||||
const hasFailures = session.failedSegmentCount > 0
|
||||
wx.showModal({
|
||||
title: '语音记录',
|
||||
content: lines.join('\n'),
|
||||
showCancel: hasFailures,
|
||||
cancelText: '关闭',
|
||||
confirmText: hasFailures ? '重试失败项' : '知道了',
|
||||
success: (result) => {
|
||||
if (hasFailures && result.confirm) void this.retryFailedSegments()
|
||||
}
|
||||
})
|
||||
},
|
||||
|
||||
async retryFailedSegments() {
|
||||
const session = this.data.session
|
||||
if (!session) return
|
||||
const failedSegments = session.lessons.flatMap((lesson) =>
|
||||
lesson.segments.filter((segment) => segment.status === 'failed')
|
||||
)
|
||||
if (failedSegments.length === 0) return
|
||||
this.setData({ voiceBusy: true, voiceStatus: '正在重试转录' })
|
||||
try {
|
||||
for (const segment of failedSegments) await retryVoiceSegment(segment.id)
|
||||
const failedLessonIds = session.lessons
|
||||
.filter((lesson) => lesson.segments.some((segment) => segment.status === 'failed'))
|
||||
.map((lesson) => lesson.id)
|
||||
for (const lessonId of failedLessonIds) {
|
||||
const finished = await finishVoiceLesson(lessonId)
|
||||
await this.applyFinishedLesson(finished)
|
||||
}
|
||||
await this.loadActiveSession()
|
||||
} catch (error) {
|
||||
wx.showToast({ title: getApiErrorMessage(error), icon: 'none' })
|
||||
} finally {
|
||||
this.setData({ voiceBusy: false })
|
||||
this.syncVoicePresentation(this.data.session)
|
||||
}
|
||||
},
|
||||
|
||||
async saveFeedback() {
|
||||
this.clearDraftSaveTimer()
|
||||
const session = this.data.session
|
||||
if (session && session.processingSegmentCount > 0) {
|
||||
wx.showToast({ title: '录音仍在转写,请稍候', icon: 'none' })
|
||||
this.scheduleVoicePolling(session)
|
||||
return
|
||||
}
|
||||
if (session?.needsGeneration) {
|
||||
await this.generateFeedback()
|
||||
return
|
||||
}
|
||||
|
||||
const content = this.data.draft.content.trim()
|
||||
if (!content) {
|
||||
wx.showToast({ title: '请填写反馈内容', icon: 'none' })
|
||||
@@ -91,8 +694,20 @@ Page({
|
||||
|
||||
this.setData({ saving: true })
|
||||
try {
|
||||
await createFeedbackRecord({ ...this.data.draft, content })
|
||||
this.setData({ draft: buildDraft(), selectedProfileIndex: 0, contentLength: 0 })
|
||||
await createFeedbackRecord({
|
||||
...this.data.draft,
|
||||
content,
|
||||
feedbackSessionId: session?.id || null
|
||||
})
|
||||
this.setData({
|
||||
draft: buildDraft(),
|
||||
session: null,
|
||||
selectedProfileIndex: 0,
|
||||
contentLength: 0,
|
||||
voiceStatus: '语音录入',
|
||||
voiceActionable: false,
|
||||
primaryActionText: '保存反馈'
|
||||
})
|
||||
wx.showToast({ title: '反馈已保存', icon: 'success' })
|
||||
setTimeout(() => wx.switchTab({ url: '/pages/records/index' }), 450)
|
||||
} catch (error) {
|
||||
@@ -102,6 +717,26 @@ Page({
|
||||
}
|
||||
},
|
||||
|
||||
async generateFeedback() {
|
||||
const session = this.data.session
|
||||
if (!session) return
|
||||
this.setData({ generating: true, voiceBusy: true, voiceStatus: '正在生成反馈' })
|
||||
try {
|
||||
const result = await generateFeedbackFromVoice(session.id, this.data.draft.content)
|
||||
this.setData({
|
||||
'draft.content': result.content,
|
||||
contentLength: result.content.length
|
||||
})
|
||||
await this.loadActiveSession()
|
||||
wx.showToast({ title: '反馈已生成,可继续修改', icon: 'none' })
|
||||
} catch (error) {
|
||||
wx.showToast({ title: getApiErrorMessage(error), icon: 'none' })
|
||||
} finally {
|
||||
this.setData({ generating: false, voiceBusy: false })
|
||||
this.syncVoicePresentation(this.data.session)
|
||||
}
|
||||
},
|
||||
|
||||
goStudents() {
|
||||
wx.switchTab({ url: '/pages/students/index' })
|
||||
}
|
||||
|
||||
@@ -12,12 +12,12 @@
|
||||
</view>
|
||||
|
||||
<view class="feedback-form surface">
|
||||
<picker class="feedback-field" mode="selector" range="{{profileOptions}}" value="{{selectedProfileIndex}}" disabled="{{loading}}" bindchange="onProfileChange">
|
||||
<picker class="feedback-field" mode="selector" range="{{profileOptions}}" value="{{selectedProfileIndex}}" disabled="{{loading || recording || voiceBusy}}" bindchange="onProfileChange">
|
||||
<text class="form-label">学生档案</text>
|
||||
<text class="form-value">{{loading ? '正在加载...' : profileOptions[selectedProfileIndex]}}</text>
|
||||
</picker>
|
||||
|
||||
<picker class="feedback-field" mode="date" value="{{draft.feedbackDate}}" bindchange="onDateChange">
|
||||
<picker class="feedback-field" mode="date" value="{{draft.feedbackDate}}" disabled="{{recording || voiceBusy}}" bindchange="onDateChange">
|
||||
<text class="form-label">反馈日期</text>
|
||||
<text class="form-value">{{draft.feedbackDate}}</text>
|
||||
</picker>
|
||||
@@ -31,12 +31,30 @@
|
||||
class="feedback-textarea"
|
||||
value="{{draft.content}}"
|
||||
maxlength="2000"
|
||||
disabled="{{recording || voiceBusy}}"
|
||||
placeholder="记录课堂完成情况、进步与后续建议"
|
||||
bindinput="onContentInput"
|
||||
/>
|
||||
|
||||
<view class="voice-input-row">
|
||||
<button
|
||||
class="voice-button {{recording ? 'voice-button-recording' : ''}}"
|
||||
disabled="{{saving || voiceBusy}}"
|
||||
bindtap="toggleRecording"
|
||||
aria-label="{{recording ? '结束录音' : '开始录音'}}"
|
||||
>
|
||||
<text class="voice-button-icon">{{recording ? '■' : '●'}}</text>
|
||||
<text>{{recording ? '结束录音' : '语音录入'}}</text>
|
||||
</button>
|
||||
<view class="voice-state {{voiceActionable ? 'voice-state-actionable' : ''}}" bindtap="openVoiceDetails">
|
||||
<text class="voice-status">{{voiceStatus}}</text>
|
||||
<text wx:if="{{recording}}" class="recording-time">{{recordingTime}}</text>
|
||||
<text wx:elif="{{voiceActionable}}" class="voice-state-arrow">›</text>
|
||||
</view>
|
||||
</view>
|
||||
</view>
|
||||
|
||||
<button class="save-feedback primary-button" loading="{{saving}}" disabled="{{saving}}" bindtap="saveFeedback">保存反馈</button>
|
||||
<button class="save-feedback primary-button" loading="{{saving || generating}}" disabled="{{saving || generating || recording || voiceBusy || (session && session.processingSegmentCount > 0)}}" bindtap="saveFeedback">{{primaryActionText}}</button>
|
||||
</view>
|
||||
|
||||
<view wx:if="{{!loading && profiles.length === 0}}" class="profiles-hint">
|
||||
|
||||
@@ -42,6 +42,91 @@
|
||||
line-height: 1.6;
|
||||
}
|
||||
|
||||
.voice-input-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 20rpx;
|
||||
margin-top: 18rpx;
|
||||
padding-top: 18rpx;
|
||||
border-top: 2rpx solid #e7eef6;
|
||||
}
|
||||
|
||||
.voice-button {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
gap: 12rpx;
|
||||
width: 220rpx;
|
||||
height: 72rpx;
|
||||
margin: 0;
|
||||
padding: 0 20rpx;
|
||||
border: 2rpx solid #b8cbe1;
|
||||
border-radius: 12rpx;
|
||||
background: #ffffff;
|
||||
color: #2b67e8;
|
||||
font-size: 26rpx;
|
||||
font-weight: 600;
|
||||
line-height: 1;
|
||||
}
|
||||
|
||||
.voice-button::after {
|
||||
border: 0;
|
||||
}
|
||||
|
||||
.voice-button[disabled] {
|
||||
background: #f4f7fa;
|
||||
color: #99a5b3;
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.voice-button-recording,
|
||||
.voice-button-recording[disabled] {
|
||||
border-color: #e46b65;
|
||||
background: #fff4f3;
|
||||
color: #c83f39;
|
||||
}
|
||||
|
||||
.voice-button-icon {
|
||||
font-size: 20rpx;
|
||||
line-height: 1;
|
||||
}
|
||||
|
||||
.voice-state {
|
||||
display: flex;
|
||||
flex: 1;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
min-width: 0;
|
||||
color: #65758a;
|
||||
font-size: 24rpx;
|
||||
}
|
||||
|
||||
.voice-status {
|
||||
flex: 1;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.voice-state-actionable {
|
||||
min-height: 64rpx;
|
||||
}
|
||||
|
||||
.voice-state-arrow {
|
||||
flex: none;
|
||||
margin-left: 10rpx;
|
||||
color: #7f8da0;
|
||||
font-size: 38rpx;
|
||||
line-height: 1;
|
||||
}
|
||||
|
||||
.recording-time {
|
||||
flex: none;
|
||||
margin-left: 12rpx;
|
||||
color: #c83f39;
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
|
||||
.save-feedback {
|
||||
margin-top: 24rpx;
|
||||
}
|
||||
|
||||
@@ -41,9 +41,21 @@ Page({
|
||||
this.setData({ testing: true, connectionText: '正在检测服务...', connectionTone: 'idle' })
|
||||
try {
|
||||
const health = await getHealth()
|
||||
let connectionText = '服务、数据库和语音转录正常'
|
||||
let connectionTone = 'success'
|
||||
if (!health.databaseConfigured) {
|
||||
connectionText = '服务正常,数据库尚未配置'
|
||||
connectionTone = 'warning'
|
||||
} else if (!health.speechConfigured) {
|
||||
connectionText = '服务和数据库正常,语音转录尚未配置'
|
||||
connectionTone = 'warning'
|
||||
} else if (!health.speechAvailable) {
|
||||
connectionText = '服务和数据库正常,本地语音模型尚未就绪'
|
||||
connectionTone = 'warning'
|
||||
}
|
||||
this.setData({
|
||||
connectionText: health.databaseConfigured ? '服务和数据库连接正常' : '服务正常,数据库尚未配置',
|
||||
connectionTone: health.databaseConfigured ? 'success' : 'warning'
|
||||
connectionText,
|
||||
connectionTone
|
||||
})
|
||||
} catch (error) {
|
||||
this.setData({ connectionText: getApiErrorMessage(error), connectionTone: 'error' })
|
||||
|
||||
@@ -37,6 +37,10 @@
|
||||
"value": "server",
|
||||
"type": "folder"
|
||||
},
|
||||
{
|
||||
"value": "asr-service",
|
||||
"type": "folder"
|
||||
},
|
||||
{
|
||||
"value": "node_modules",
|
||||
"type": "folder"
|
||||
@@ -44,6 +48,14 @@
|
||||
{
|
||||
"value": "graphify-out",
|
||||
"type": "folder"
|
||||
},
|
||||
{
|
||||
"value": ".git",
|
||||
"type": "folder"
|
||||
},
|
||||
{
|
||||
"value": "compose.asr.yml",
|
||||
"type": "file"
|
||||
}
|
||||
],
|
||||
"include": []
|
||||
@@ -51,4 +63,4 @@
|
||||
"appid": "wx3fe11e262a4b4885",
|
||||
"simulatorPluginLibVersion": {},
|
||||
"editorSetting": {}
|
||||
}
|
||||
}
|
||||
|
||||
257
utils/api.ts
257
utils/api.ts
@@ -1,11 +1,18 @@
|
||||
import type {
|
||||
FeedbackDraft,
|
||||
FeedbackRecord,
|
||||
FeedbackSession,
|
||||
GeneratedFeedback,
|
||||
HealthStatus,
|
||||
StudentDraft,
|
||||
StudentProfile,
|
||||
StudentProfileDefaults,
|
||||
Term
|
||||
Term,
|
||||
VoiceAudioSegment,
|
||||
VoiceLesson,
|
||||
VoiceLessonCreated,
|
||||
VoiceLessonFinished,
|
||||
VoiceProcessingStatus
|
||||
} from './types'
|
||||
|
||||
const DEFAULT_API_BASE_URL = 'http://127.0.0.1:8080'
|
||||
@@ -50,6 +57,66 @@ interface RawFeedbackRecord {
|
||||
interface RawHealthStatus {
|
||||
status: string
|
||||
database_configured: boolean
|
||||
speech_configured: boolean
|
||||
speech_available: boolean
|
||||
speech_provider: string | null
|
||||
summary_configured: boolean
|
||||
}
|
||||
|
||||
interface RawVoiceAudioSegment {
|
||||
id: string
|
||||
sequence: number
|
||||
duration_ms: number
|
||||
status: 'processing' | 'ready' | 'failed'
|
||||
error_message: string | null
|
||||
}
|
||||
|
||||
interface RawVoiceLesson {
|
||||
id: string
|
||||
sequence: number
|
||||
status: VoiceProcessingStatus
|
||||
duration_ms: number
|
||||
applied_directly: boolean
|
||||
included_in_generation: boolean
|
||||
segments: RawVoiceAudioSegment[]
|
||||
started_at: string
|
||||
ended_at: string | null
|
||||
}
|
||||
|
||||
interface RawFeedbackSession {
|
||||
id: string
|
||||
profile_id: string | null
|
||||
feedback_date: string
|
||||
content: string
|
||||
status: 'active' | 'finalized'
|
||||
lesson_count: number
|
||||
total_duration_ms: number
|
||||
processing_segment_count: number
|
||||
failed_segment_count: number
|
||||
needs_generation: boolean
|
||||
lessons: RawVoiceLesson[]
|
||||
created_at: string
|
||||
updated_at: string
|
||||
}
|
||||
|
||||
interface RawVoiceLessonCreated {
|
||||
id: string
|
||||
sequence: number
|
||||
status: VoiceProcessingStatus
|
||||
started_at: string
|
||||
}
|
||||
|
||||
interface RawVoiceLessonFinished {
|
||||
id: string
|
||||
status: VoiceProcessingStatus
|
||||
duration_ms: number
|
||||
transcript: string
|
||||
can_apply_directly: boolean
|
||||
}
|
||||
|
||||
interface RawGeneratedFeedback {
|
||||
content: string
|
||||
method: 'llm' | 'extractive'
|
||||
}
|
||||
|
||||
type HttpMethod = 'GET' | 'POST' | 'PUT' | 'DELETE'
|
||||
@@ -81,17 +148,17 @@ export function getApiErrorMessage(error: unknown): string {
|
||||
if (!(error instanceof ApiRequestError)) return '请求失败,请稍后重试'
|
||||
if (error.statusCode === 401) return '开发身份未通过后端校验'
|
||||
if (error.statusCode === 404) return '请求的数据不存在或无权访问'
|
||||
if (error.statusCode === 503) return '后端尚未配置数据库连接'
|
||||
if (error.statusCode === 503 && error.message.includes('database')) return '后端尚未配置数据库连接'
|
||||
return error.message
|
||||
}
|
||||
|
||||
function request<T>(path: string, method: HttpMethod = 'GET', data?: object): Promise<T> {
|
||||
function request<T>(path: string, method: HttpMethod = 'GET', data?: object, timeout = 10000): Promise<T> {
|
||||
return new Promise((resolve, reject) => {
|
||||
wx.request({
|
||||
url: `${getApiBaseUrl()}${path}`,
|
||||
method,
|
||||
data,
|
||||
timeout: 10000,
|
||||
timeout,
|
||||
header: {
|
||||
'Content-Type': 'application/json',
|
||||
'X-User-Id': DEVELOPMENT_USER_ID
|
||||
@@ -165,11 +232,57 @@ function mapFeedbackRecord(record: RawFeedbackRecord): FeedbackRecord {
|
||||
}
|
||||
}
|
||||
|
||||
function mapVoiceSegment(segment: RawVoiceAudioSegment): VoiceAudioSegment {
|
||||
return {
|
||||
id: segment.id,
|
||||
sequence: segment.sequence,
|
||||
durationMs: segment.duration_ms,
|
||||
status: segment.status,
|
||||
errorMessage: segment.error_message
|
||||
}
|
||||
}
|
||||
|
||||
function mapVoiceLesson(lesson: RawVoiceLesson): VoiceLesson {
|
||||
return {
|
||||
id: lesson.id,
|
||||
sequence: lesson.sequence,
|
||||
status: lesson.status,
|
||||
durationMs: lesson.duration_ms,
|
||||
appliedDirectly: lesson.applied_directly,
|
||||
includedInGeneration: lesson.included_in_generation,
|
||||
segments: lesson.segments.map(mapVoiceSegment),
|
||||
startedAt: parseTimestamp(lesson.started_at),
|
||||
endedAt: lesson.ended_at ? parseTimestamp(lesson.ended_at) : null
|
||||
}
|
||||
}
|
||||
|
||||
function mapFeedbackSession(session: RawFeedbackSession): FeedbackSession {
|
||||
return {
|
||||
id: session.id,
|
||||
profileId: session.profile_id,
|
||||
feedbackDate: session.feedback_date,
|
||||
content: session.content,
|
||||
status: session.status,
|
||||
lessonCount: session.lesson_count,
|
||||
totalDurationMs: session.total_duration_ms,
|
||||
processingSegmentCount: session.processing_segment_count,
|
||||
failedSegmentCount: session.failed_segment_count,
|
||||
needsGeneration: session.needs_generation,
|
||||
lessons: session.lessons.map(mapVoiceLesson),
|
||||
createdAt: parseTimestamp(session.created_at),
|
||||
updatedAt: parseTimestamp(session.updated_at)
|
||||
}
|
||||
}
|
||||
|
||||
export async function getHealth(): Promise<HealthStatus> {
|
||||
const result = await request<RawHealthStatus>('/health')
|
||||
return {
|
||||
status: result.status,
|
||||
databaseConfigured: result.database_configured
|
||||
databaseConfigured: result.database_configured,
|
||||
speechConfigured: result.speech_configured,
|
||||
speechAvailable: result.speech_available,
|
||||
speechProvider: result.speech_provider,
|
||||
summaryConfigured: result.summary_configured
|
||||
}
|
||||
}
|
||||
|
||||
@@ -209,7 +322,8 @@ export async function createFeedbackRecord(draft: FeedbackDraft): Promise<Feedba
|
||||
const result = await request<RawFeedbackRecord>('/api/v1/feedback-records', 'POST', {
|
||||
profile_id: draft.profileId,
|
||||
feedback_date: draft.feedbackDate,
|
||||
content: draft.content
|
||||
content: draft.content,
|
||||
feedback_session_id: draft.feedbackSessionId || null
|
||||
})
|
||||
return mapFeedbackRecord(result)
|
||||
}
|
||||
@@ -217,3 +331,134 @@ export async function createFeedbackRecord(draft: FeedbackDraft): Promise<Feedba
|
||||
export async function deleteFeedbackRecord(recordId: string): Promise<void> {
|
||||
await request<unknown>(`/api/v1/feedback-records/${encodeURIComponent(recordId)}`, 'DELETE')
|
||||
}
|
||||
|
||||
export async function getActiveFeedbackSession(profileId: string | null): Promise<FeedbackSession | null> {
|
||||
const query = profileId ? `?profile_id=${encodeURIComponent(profileId)}` : ''
|
||||
const result = await request<RawFeedbackSession | null>(`/api/v1/feedback-sessions/active${query}`)
|
||||
return result ? mapFeedbackSession(result) : null
|
||||
}
|
||||
|
||||
export async function ensureFeedbackSession(
|
||||
profileId: string | null,
|
||||
feedbackDate: string,
|
||||
content: string
|
||||
): Promise<FeedbackSession> {
|
||||
const result = await request<RawFeedbackSession>('/api/v1/feedback-sessions', 'POST', {
|
||||
profile_id: profileId,
|
||||
feedback_date: feedbackDate,
|
||||
content
|
||||
})
|
||||
return mapFeedbackSession(result)
|
||||
}
|
||||
|
||||
export async function updateFeedbackSession(
|
||||
sessionId: string,
|
||||
feedbackDate: string,
|
||||
content: string
|
||||
): Promise<FeedbackSession> {
|
||||
const result = await request<RawFeedbackSession>(
|
||||
`/api/v1/feedback-sessions/${encodeURIComponent(sessionId)}`,
|
||||
'PUT',
|
||||
{ feedback_date: feedbackDate, content }
|
||||
)
|
||||
return mapFeedbackSession(result)
|
||||
}
|
||||
|
||||
export async function createVoiceLesson(sessionId: string): Promise<VoiceLessonCreated> {
|
||||
const result = await request<RawVoiceLessonCreated>(
|
||||
`/api/v1/feedback-sessions/${encodeURIComponent(sessionId)}/lessons`,
|
||||
'POST'
|
||||
)
|
||||
return {
|
||||
id: result.id,
|
||||
sequence: result.sequence,
|
||||
status: result.status,
|
||||
startedAt: parseTimestamp(result.started_at)
|
||||
}
|
||||
}
|
||||
|
||||
export function uploadVoiceSegment(
|
||||
lessonId: string,
|
||||
filePath: string,
|
||||
sequence: number,
|
||||
durationMs: number
|
||||
): Promise<VoiceAudioSegment> {
|
||||
return new Promise((resolve, reject) => {
|
||||
wx.uploadFile({
|
||||
url: `${getApiBaseUrl()}/api/v1/feedback-lessons/${encodeURIComponent(lessonId)}/segments`,
|
||||
filePath,
|
||||
name: 'audio',
|
||||
formData: {
|
||||
sequence: String(sequence),
|
||||
duration_ms: String(Math.max(1, Math.round(durationMs))),
|
||||
audio_format: 'mp3'
|
||||
},
|
||||
timeout: 120000,
|
||||
header: { 'X-User-Id': DEVELOPMENT_USER_ID },
|
||||
success(response) {
|
||||
let body: RawVoiceAudioSegment | ApiErrorBody
|
||||
try {
|
||||
body = JSON.parse(response.data) as RawVoiceAudioSegment | ApiErrorBody
|
||||
} catch {
|
||||
reject(new ApiRequestError('服务返回了无效的录音响应', response.statusCode))
|
||||
return
|
||||
}
|
||||
if (response.statusCode >= 200 && response.statusCode < 300) {
|
||||
resolve(mapVoiceSegment(body as RawVoiceAudioSegment))
|
||||
return
|
||||
}
|
||||
reject(new ApiRequestError((body as ApiErrorBody).error || `服务返回 HTTP ${response.statusCode}`, response.statusCode))
|
||||
},
|
||||
fail(error) {
|
||||
reject(new ApiRequestError(`录音上传失败:${error.errMsg}`))
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
export async function finishVoiceLesson(lessonId: string): Promise<VoiceLessonFinished> {
|
||||
const result = await request<RawVoiceLessonFinished>(
|
||||
`/api/v1/feedback-lessons/${encodeURIComponent(lessonId)}/finish`,
|
||||
'POST',
|
||||
undefined,
|
||||
120000
|
||||
)
|
||||
return {
|
||||
id: result.id,
|
||||
status: result.status,
|
||||
durationMs: result.duration_ms,
|
||||
transcript: result.transcript,
|
||||
canApplyDirectly: result.can_apply_directly
|
||||
}
|
||||
}
|
||||
|
||||
export async function markVoiceLessonApplied(lessonId: string, content: string): Promise<void> {
|
||||
await request<unknown>(
|
||||
`/api/v1/feedback-lessons/${encodeURIComponent(lessonId)}/mark-applied`,
|
||||
'POST',
|
||||
{ content }
|
||||
)
|
||||
}
|
||||
|
||||
export async function retryVoiceSegment(segmentId: string): Promise<VoiceAudioSegment> {
|
||||
const result = await request<RawVoiceAudioSegment>(
|
||||
`/api/v1/audio-segments/${encodeURIComponent(segmentId)}/retry`,
|
||||
'POST',
|
||||
undefined,
|
||||
120000
|
||||
)
|
||||
return mapVoiceSegment(result)
|
||||
}
|
||||
|
||||
export async function generateFeedbackFromVoice(
|
||||
sessionId: string,
|
||||
existingContent: string
|
||||
): Promise<GeneratedFeedback> {
|
||||
const result = await request<RawGeneratedFeedback>(
|
||||
`/api/v1/feedback-sessions/${encodeURIComponent(sessionId)}/generate`,
|
||||
'POST',
|
||||
{ existing_content: existingContent },
|
||||
120000
|
||||
)
|
||||
return result
|
||||
}
|
||||
|
||||
@@ -44,9 +44,72 @@ export interface FeedbackDraft {
|
||||
profileId: string | null
|
||||
feedbackDate: string
|
||||
content: string
|
||||
feedbackSessionId?: string | null
|
||||
}
|
||||
|
||||
export interface HealthStatus {
|
||||
status: string
|
||||
databaseConfigured: boolean
|
||||
speechConfigured: boolean
|
||||
speechAvailable: boolean
|
||||
speechProvider: string | null
|
||||
summaryConfigured: boolean
|
||||
}
|
||||
|
||||
export type VoiceProcessingStatus = 'recording' | 'processing' | 'ready' | 'failed'
|
||||
|
||||
export interface VoiceAudioSegment {
|
||||
id: string
|
||||
sequence: number
|
||||
durationMs: number
|
||||
status: 'processing' | 'ready' | 'failed'
|
||||
errorMessage: string | null
|
||||
}
|
||||
|
||||
export interface VoiceLesson {
|
||||
id: string
|
||||
sequence: number
|
||||
status: VoiceProcessingStatus
|
||||
durationMs: number
|
||||
appliedDirectly: boolean
|
||||
includedInGeneration: boolean
|
||||
segments: VoiceAudioSegment[]
|
||||
startedAt: number
|
||||
endedAt: number | null
|
||||
}
|
||||
|
||||
export interface FeedbackSession {
|
||||
id: string
|
||||
profileId: string | null
|
||||
feedbackDate: string
|
||||
content: string
|
||||
status: 'active' | 'finalized'
|
||||
lessonCount: number
|
||||
totalDurationMs: number
|
||||
processingSegmentCount: number
|
||||
failedSegmentCount: number
|
||||
needsGeneration: boolean
|
||||
lessons: VoiceLesson[]
|
||||
createdAt: number
|
||||
updatedAt: number
|
||||
}
|
||||
|
||||
export interface VoiceLessonCreated {
|
||||
id: string
|
||||
sequence: number
|
||||
status: VoiceProcessingStatus
|
||||
startedAt: number
|
||||
}
|
||||
|
||||
export interface VoiceLessonFinished {
|
||||
id: string
|
||||
status: VoiceProcessingStatus
|
||||
durationMs: number
|
||||
transcript: string
|
||||
canApplyDirectly: boolean
|
||||
}
|
||||
|
||||
export interface GeneratedFeedback {
|
||||
content: string
|
||||
method: 'llm' | 'extractive'
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user