feat: add voice transcription feedback workflow
This commit is contained in:
2
.gitignore
vendored
2
.gitignore
vendored
@@ -3,6 +3,8 @@ graphify-out/
|
|||||||
|
|
||||||
# Dependencies and build output
|
# Dependencies and build output
|
||||||
node_modules/
|
node_modules/
|
||||||
|
.venv/
|
||||||
|
.models/
|
||||||
miniprogram_npm/
|
miniprogram_npm/
|
||||||
dist/
|
dist/
|
||||||
coverage/
|
coverage/
|
||||||
|
|||||||
310
DEBUG_GUIDE.md
Normal file
310
DEBUG_GUIDE.md
Normal file
@@ -0,0 +1,310 @@
|
|||||||
|
# 本地调试指南
|
||||||
|
|
||||||
|
本文用于在本机启动 PostgreSQL 连接、中文 FunASR、Rust API 和微信小程序,并验证短录音、长录音及多课节汇总。
|
||||||
|
|
||||||
|
## 1. 服务关系
|
||||||
|
|
||||||
|
```text
|
||||||
|
微信小程序
|
||||||
|
-> Rust API(127.0.0.1:8080)
|
||||||
|
-> PostgreSQL
|
||||||
|
-> FunASR(127.0.0.1:10095)
|
||||||
|
```
|
||||||
|
|
||||||
|
本地录音转写的核心是 FunASR,当前组合为:
|
||||||
|
|
||||||
|
- Paraformer:中文语音识别。
|
||||||
|
- FSMN-VAD:检测并切分有效语音。
|
||||||
|
- CT-Transformer:恢复中文标点。
|
||||||
|
- PyTorch:CPU 推理运行时。
|
||||||
|
|
||||||
|
Rust API 负责音频保存、持久化任务队列、重试、课节状态和反馈汇总。小程序每 8 分钟自动切片,并每 3 秒无感刷新转写状态。
|
||||||
|
|
||||||
|
## 2. 当前主机资源实测
|
||||||
|
|
||||||
|
测试主机为 Apple Silicon、10 核 CPU、16 GB 统一内存,配置为 CPU 单并发:
|
||||||
|
|
||||||
|
| 项目 | 实测值 |
|
||||||
|
| --- | --- |
|
||||||
|
| FunASR 模型缓存 | 约 2.1 GB |
|
||||||
|
| Python 虚拟环境 | 约 871 MB |
|
||||||
|
| 模型加载后的物理内存 | 约 2.8 GB |
|
||||||
|
| 进程历史内存峰值 | 约 4.9 GB |
|
||||||
|
| 8 分钟、16 kHz、单声道、48 kbps MP3 | 约 44.8 秒完成 |
|
||||||
|
| 推理期间 CPU | 约 90% 至 132%,即约 1 至 1.3 个 CPU 核 |
|
||||||
|
|
||||||
|
8 分钟数据使用重复的清晰中文样本测得,真实课堂中的噪声、停顿和说话人数会影响耗时和识别质量。
|
||||||
|
|
||||||
|
容量建议:
|
||||||
|
|
||||||
|
- 当前单教师或低并发调试:不需要 GPU,当前 16 GB 主机足够。
|
||||||
|
- CPU 部署最低建议:4 vCPU、8 GB 内存;8 GB 可能发生交换,16 GB 更稳妥。
|
||||||
|
- 多教师并发:先保持单并发观察队列;若片段持续积压,再考虑增加 CPU 实例或 NVIDIA GPU。
|
||||||
|
- GPU 是可选优化,不是运行前提。CUDA 部署需要单独的 CUDA/PyTorch 镜像,当前 CPU 镜像不能只修改 `ASR_DEVICE=cuda`。
|
||||||
|
- GPU 部署建议从 8 GB 显存起步并用真实课堂录音压测。当前 Apple Silicon 配置没有使用 Apple GPU。
|
||||||
|
- `ASR_WORKER_CONCURRENCY` 和 `ASR_MAX_CONCURRENCY` 应保持一致;每提高一个并发都要重新观察内存和吞吐。
|
||||||
|
|
||||||
|
## 3. 首次准备
|
||||||
|
|
||||||
|
在项目根目录执行:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /Users/zhangheng/project/ai/miniprogram/teaching-feedback-assistant
|
||||||
|
```
|
||||||
|
|
||||||
|
确认 `server/.env` 存在,并填写可访问的 `DATABASE_URL`。不存在时再复制示例,避免覆盖现有配置:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
test -f server/.env || cp server/.env.example server/.env
|
||||||
|
```
|
||||||
|
|
||||||
|
本地 FunASR 默认配置如下,无需腾讯云密钥:
|
||||||
|
|
||||||
|
```env
|
||||||
|
ASR_PROVIDER=local
|
||||||
|
LOCAL_ASR_URL=http://127.0.0.1:10095
|
||||||
|
ASR_REQUEST_TIMEOUT_SECONDS=1800
|
||||||
|
ASR_WORKER_CONCURRENCY=1
|
||||||
|
ASR_JOB_LEASE_SECONDS=3600
|
||||||
|
```
|
||||||
|
|
||||||
|
首次准备 Python 环境:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd asr-service
|
||||||
|
python3 -m venv .venv
|
||||||
|
.venv/bin/python -m pip install --upgrade pip
|
||||||
|
.venv/bin/python -m pip install -r requirements.txt
|
||||||
|
cd ..
|
||||||
|
```
|
||||||
|
|
||||||
|
模型首次启动会下载约 2.1 GB 到 `asr-service/.models`。`.venv` 和 `.models` 均已被 Git 和微信小程序打包忽略。
|
||||||
|
|
||||||
|
## 4. 启动全部服务
|
||||||
|
|
||||||
|
### 4.1 执行数据库迁移
|
||||||
|
|
||||||
|
每次拉取到新迁移后执行一次:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd server
|
||||||
|
cargo run --bin migrate
|
||||||
|
cd ..
|
||||||
|
```
|
||||||
|
|
||||||
|
看到 `PostgreSQL migrations completed` 表示成功。
|
||||||
|
|
||||||
|
### 4.2 启动 FunASR
|
||||||
|
|
||||||
|
终端一:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd asr-service
|
||||||
|
MODELSCOPE_CACHE=.models .venv/bin/uvicorn app:app \
|
||||||
|
--host 127.0.0.1 \
|
||||||
|
--port 10095 \
|
||||||
|
--no-access-log
|
||||||
|
```
|
||||||
|
|
||||||
|
另一个终端检查:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://127.0.0.1:10095/health
|
||||||
|
```
|
||||||
|
|
||||||
|
预期结果:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"status":"ok","model":"paraformer-zh","device":"cpu"}
|
||||||
|
```
|
||||||
|
|
||||||
|
首次下载或缓存加载期间端口可能暂不可用,应等待模型完全加载。
|
||||||
|
|
||||||
|
Docker 方式可替代本机 Python:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f compose.asr.yml up --build -d
|
||||||
|
docker compose -f compose.asr.yml logs -f funasr
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.3 启动 Rust API
|
||||||
|
|
||||||
|
终端二:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd server
|
||||||
|
cargo run
|
||||||
|
```
|
||||||
|
|
||||||
|
检查完整链路:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://127.0.0.1:8080/health | jq .
|
||||||
|
```
|
||||||
|
|
||||||
|
正常状态应包含:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "ok",
|
||||||
|
"database_configured": true,
|
||||||
|
"speech_configured": true,
|
||||||
|
"speech_available": true,
|
||||||
|
"speech_provider": "local-funasr"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
接口文档:<http://127.0.0.1:8080/scalar>
|
||||||
|
|
||||||
|
### 4.4 启动微信开发者工具
|
||||||
|
|
||||||
|
终端三,在项目根目录执行:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cli open \
|
||||||
|
--project /Users/zhangheng/project/ai/miniprogram/teaching-feedback-assistant \
|
||||||
|
--port 63097 \
|
||||||
|
--lang zh
|
||||||
|
```
|
||||||
|
|
||||||
|
自动预览编译检查:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cli auto-preview \
|
||||||
|
--project /Users/zhangheng/project/ai/miniprogram/teaching-feedback-assistant \
|
||||||
|
--port 63097 \
|
||||||
|
--lang zh \
|
||||||
|
--info-output /tmp/teaching-feedback-preview.json
|
||||||
|
```
|
||||||
|
|
||||||
|
开发者工具模拟器默认访问 `http://127.0.0.1:8080`。进入小程序“我的”页面,应看到“服务、数据库和语音转录正常”。
|
||||||
|
|
||||||
|
## 5. 录音测试流程
|
||||||
|
|
||||||
|
### 短录音
|
||||||
|
|
||||||
|
1. 打开“反馈生成”。
|
||||||
|
2. 点击“语音录入”,录制 10 至 30 秒中文。
|
||||||
|
3. 点击“结束录音”。
|
||||||
|
4. 页面短暂显示“正在处理”。
|
||||||
|
5. 转写完成后文字应自动加入反馈内容。
|
||||||
|
|
||||||
|
短录音条件为不超过 60 秒,且转写文本未超过后端短文本限制。
|
||||||
|
|
||||||
|
### 长录音
|
||||||
|
|
||||||
|
1. 录制超过 60 秒的中文。
|
||||||
|
2. 结束后等待状态变成“已就绪”。
|
||||||
|
3. 主按钮变为“生成反馈”。
|
||||||
|
4. 点击一次后,转写内容会汇总进反馈草稿。
|
||||||
|
|
||||||
|
### 多课节汇总
|
||||||
|
|
||||||
|
1. 录制并结束第一节课。
|
||||||
|
2. 再次点击“语音录入”录制下一节。
|
||||||
|
3. 重复完成多节录音。
|
||||||
|
4. 所有课节就绪后点击一次“生成反馈”。
|
||||||
|
|
||||||
|
单节录音超过 8 分钟时,小程序会自动结束当前片段并立即开始下一片段,用户无需操作。
|
||||||
|
|
||||||
|
## 6. 故障恢复测试
|
||||||
|
|
||||||
|
### FunASR 未启动
|
||||||
|
|
||||||
|
停止 FunASR 后录音,Rust API 仍会保存音频,页面保持“正在处理”。重新启动 FunASR 后,后台队列会自动继续。
|
||||||
|
|
||||||
|
### Rust 在推理中退出
|
||||||
|
|
||||||
|
任务使用租约避免多个工作线程重复处理。默认租约为 3600 秒。调试崩溃恢复时,可在 `server/.env` 临时设置:
|
||||||
|
|
||||||
|
```env
|
||||||
|
ASR_JOB_LEASE_SECONDS=60
|
||||||
|
```
|
||||||
|
|
||||||
|
重新启动 Rust API 后,超出租约的任务会再次被领取。生产环境应使用足够覆盖最长片段推理时间的租约。
|
||||||
|
|
||||||
|
### 转写失败
|
||||||
|
|
||||||
|
点击页面中的语音状态,可查看课节状态并重试失败片段。音频保存在 `server/data/audio`,转写失败不会立即删除原文件。
|
||||||
|
|
||||||
|
## 7. 常见问题
|
||||||
|
|
||||||
|
### `speech_configured=true` 但 `speech_available=false`
|
||||||
|
|
||||||
|
- FunASR 尚未启动或模型仍在加载。
|
||||||
|
- `10095` 端口被占用。
|
||||||
|
- `LOCAL_ASR_URL` 配置错误。
|
||||||
|
|
||||||
|
检查:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
lsof -nP -iTCP:10095 -sTCP:LISTEN
|
||||||
|
curl http://127.0.0.1:10095/health
|
||||||
|
```
|
||||||
|
|
||||||
|
### `database_configured=false` 或数据接口返回 503
|
||||||
|
|
||||||
|
检查 `server/.env` 中的 `DATABASE_URL`,然后重新执行迁移并启动 API。
|
||||||
|
|
||||||
|
### 微信模拟器无法连接 API
|
||||||
|
|
||||||
|
```bash
|
||||||
|
lsof -nP -iTCP:8080 -sTCP:LISTEN
|
||||||
|
curl http://127.0.0.1:8080/health
|
||||||
|
```
|
||||||
|
|
||||||
|
再到小程序“我的”页面确认 API 地址为 `http://127.0.0.1:8080`。
|
||||||
|
|
||||||
|
### 真机无法访问 `127.0.0.1`
|
||||||
|
|
||||||
|
真机中的 `127.0.0.1` 指手机自身。局域网调试时:
|
||||||
|
|
||||||
|
1. 将 `server/.env` 的 `HOST` 改为 `0.0.0.0`。
|
||||||
|
2. 确认手机和电脑连接同一网络。
|
||||||
|
3. 在“我的”页面将 API 地址改为 `http://<电脑局域网IP>:8080`。
|
||||||
|
4. 检查系统防火墙和微信开发者工具的域名校验设置。
|
||||||
|
|
||||||
|
正式环境必须使用微信允许的 HTTPS 合法域名。
|
||||||
|
|
||||||
|
## 8. 代码检查
|
||||||
|
|
||||||
|
小程序:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm run typecheck
|
||||||
|
```
|
||||||
|
|
||||||
|
Rust:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd server
|
||||||
|
cargo fmt --check
|
||||||
|
cargo test
|
||||||
|
cargo clippy --all-targets -- -D warnings
|
||||||
|
```
|
||||||
|
|
||||||
|
FunASR 服务:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 -m py_compile asr-service/app.py
|
||||||
|
asr-service/.venv/bin/python -m pip check
|
||||||
|
```
|
||||||
|
|
||||||
|
## 9. 停止服务
|
||||||
|
|
||||||
|
前台运行时,在 FunASR 和 Rust API 各自终端按 `Ctrl+C`。
|
||||||
|
|
||||||
|
Docker 方式:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f compose.asr.yml down
|
||||||
|
```
|
||||||
|
|
||||||
|
确认端口已经释放:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
lsof -nP -iTCP:8080 -sTCP:LISTEN
|
||||||
|
lsof -nP -iTCP:10095 -sTCP:LISTEN
|
||||||
|
```
|
||||||
|
|
||||||
|
正常停止服务不会删除 `asr-service/.models` 模型缓存。除非希望重新下载模型,否则不要删除该目录。
|
||||||
12
README.md
12
README.md
@@ -11,6 +11,8 @@ cli open --project /Users/zhangheng/project/ai/miniprogram/teaching-feedback-ass
|
|||||||
|
|
||||||
3. 在微信开发者工具内编译、预览和调试。
|
3. 在微信开发者工具内编译、预览和调试。
|
||||||
|
|
||||||
|
反馈生成页使用小程序原生录音能力。短录音会直接追加到反馈内容,长录音每 8 分钟自动切片,多节录音可以汇总为一次反馈。真机首次使用时需要允许麦克风权限;中文转录默认使用项目自带的本地 FunASR 服务,不要求腾讯云密钥。
|
||||||
|
|
||||||
小程序默认连接 `http://127.0.0.1:8080`。在开发者工具模拟器中,可进入“我的”页面保存其他 API 地址并检测服务状态。
|
小程序默认连接 `http://127.0.0.1:8080`。在开发者工具模拟器中,可进入“我的”页面保存其他 API 地址并检测服务状态。
|
||||||
|
|
||||||
## 验证
|
## 验证
|
||||||
@@ -33,6 +35,15 @@ cargo run --bin migrate
|
|||||||
cargo run
|
cargo run
|
||||||
```
|
```
|
||||||
|
|
||||||
|
另开一个终端启动中文转录服务。首次启动会下载 Paraformer、VAD 和标点模型,下载完成后健康检查才会显示语音服务可用:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f compose.asr.yml up --build -d
|
||||||
|
curl http://127.0.0.1:10095/health
|
||||||
|
```
|
||||||
|
|
||||||
|
没有 Docker 时可按 [asr-service/README.md](./asr-service/README.md) 使用 Python 3.10 启动。Rust API 默认连接 `http://127.0.0.1:10095`;模型未就绪时已上传录音会保留在队列中,不会阻塞小程序上传。
|
||||||
|
|
||||||
未设置 `DATABASE_URL` 时,仅健康检查可用,数据接口会返回 `503`。数据库配置和迁移说明参见 [server/API.md](./server/API.md)。
|
未设置 `DATABASE_URL` 时,仅健康检查可用,数据接口会返回 `503`。数据库配置和迁移说明参见 [server/API.md](./server/API.md)。
|
||||||
|
|
||||||
服务启动后可打开 `http://127.0.0.1:8080/scalar` 查看并直接调用 Scalar 交互文档,原始规范位于 `http://127.0.0.1:8080/openapi.json`。完整测试顺序参见 [server/API_GUIDE.md](./server/API_GUIDE.md)。
|
服务启动后可打开 `http://127.0.0.1:8080/scalar` 查看并直接调用 Scalar 交互文档,原始规范位于 `http://127.0.0.1:8080/openapi.json`。完整测试顺序参见 [server/API_GUIDE.md](./server/API_GUIDE.md)。
|
||||||
@@ -40,6 +51,7 @@ cargo run
|
|||||||
## 小程序与后端
|
## 小程序与后端
|
||||||
|
|
||||||
- 学生档案、档案预设和反馈记录均通过 `utils/api.ts` 访问 Rust API。
|
- 学生档案、档案预设和反馈记录均通过 `utils/api.ts` 访问 Rust API。
|
||||||
|
- 反馈页只暴露一个语音入口;反馈批次、课节、录音片段和转录状态由后端自动维护。
|
||||||
- 开发者工具模拟器使用默认地址 `http://127.0.0.1:8080`。
|
- 开发者工具模拟器使用默认地址 `http://127.0.0.1:8080`。
|
||||||
- 真机和正式版本不能把 `127.0.0.1` 当作电脑地址;应部署可公网访问的 HTTPS API,并在微信公众平台配置 request 合法域名,然后到小程序“我的”页面修改服务地址。
|
- 真机和正式版本不能把 `127.0.0.1` 当作电脑地址;应部署可公网访问的 HTTPS API,并在微信公众平台配置 request 合法域名,然后到小程序“我的”页面修改服务地址。
|
||||||
- 微信登录尚未接入。业务请求暂时使用固定开发用户 UUID;生产发布前必须改为由后端根据微信登录凭据解析用户身份。
|
- 微信登录尚未接入。业务请求暂时使用固定开发用户 UUID;生产发布前必须改为由后端根据微信登录凭据解析用户身份。
|
||||||
|
|||||||
5
app.json
5
app.json
@@ -13,6 +13,11 @@
|
|||||||
"backgroundColor": "#F4F8FE",
|
"backgroundColor": "#F4F8FE",
|
||||||
"backgroundTextStyle": "light"
|
"backgroundTextStyle": "light"
|
||||||
},
|
},
|
||||||
|
"permission": {
|
||||||
|
"scope.record": {
|
||||||
|
"desc": "用于将课堂反馈录音转成文字"
|
||||||
|
}
|
||||||
|
},
|
||||||
"tabBar": {
|
"tabBar": {
|
||||||
"custom": true,
|
"custom": true,
|
||||||
"color": "#7B8796",
|
"color": "#7B8796",
|
||||||
|
|||||||
4
asr-service/.dockerignore
Normal file
4
asr-service/.dockerignore
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
__pycache__/
|
||||||
|
*.pyc
|
||||||
|
.venv/
|
||||||
|
.pytest_cache/
|
||||||
20
asr-service/Dockerfile
Normal file
20
asr-service/Dockerfile
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
FROM python:3.10-slim
|
||||||
|
|
||||||
|
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||||
|
PYTHONUNBUFFERED=1 \
|
||||||
|
PIP_NO_CACHE_DIR=1
|
||||||
|
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends ffmpeg libsndfile1 \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
COPY requirements.txt ./
|
||||||
|
RUN pip install --upgrade pip \
|
||||||
|
&& pip install torch==2.8.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cpu \
|
||||||
|
&& pip install -r requirements.txt
|
||||||
|
|
||||||
|
COPY app.py ./
|
||||||
|
|
||||||
|
EXPOSE 10095
|
||||||
|
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "10095", "--workers", "1", "--no-access-log"]
|
||||||
38
asr-service/README.md
Normal file
38
asr-service/README.md
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
# 中文录音转写服务
|
||||||
|
|
||||||
|
该服务使用 FunASR 的中文 Paraformer、FSMN-VAD 和标点模型,为 Rust API 提供本地录音转写。模型首次启动时会下载约 2.1 GB 到持久化缓存,完成后 `/health` 才会可用。
|
||||||
|
|
||||||
|
## Docker 启动
|
||||||
|
|
||||||
|
在项目根目录执行:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f compose.asr.yml up --build -d
|
||||||
|
curl http://127.0.0.1:10095/health
|
||||||
|
```
|
||||||
|
|
||||||
|
默认使用 CPU 且只允许一个并发推理任务。可通过 `ASR_DEVICE`、`ASR_MAX_CONCURRENCY`、`ASR_MODEL`、`ASR_VAD_MODEL` 和 `ASR_PUNC_MODEL` 调整。GPU 部署需要使用带 CUDA 的 PyTorch 基础环境,不能只把 CPU 镜像中的 `ASR_DEVICE` 改为 `cuda`。
|
||||||
|
|
||||||
|
## 本机 Python 启动
|
||||||
|
|
||||||
|
支持 Python 3.9 及以上版本,推荐 Python 3.10:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd asr-service
|
||||||
|
python3.10 -m venv .venv
|
||||||
|
source .venv/bin/activate
|
||||||
|
pip install -r requirements.txt
|
||||||
|
MODELSCOPE_CACHE=.models uvicorn app:app --host 127.0.0.1 --port 10095 --no-access-log
|
||||||
|
```
|
||||||
|
|
||||||
|
## 接口
|
||||||
|
|
||||||
|
Rust API 调用 `POST /v1/transcriptions`,以 multipart 上传 `file`、`language` 和 `duration_ms`,响应格式为:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"text": "识别后的中文课堂内容。",
|
||||||
|
"duration_ms": 480000,
|
||||||
|
"request_id": "b2f6fc1c-62c1-4a99-8ab4-6f933f1bd252"
|
||||||
|
}
|
||||||
|
```
|
||||||
173
asr-service/app.py
Normal file
173
asr-service/app.py
Normal file
@@ -0,0 +1,173 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import tempfile
|
||||||
|
import uuid
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import FastAPI, File, Form, HTTPException, UploadFile
|
||||||
|
from funasr import AutoModel
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
|
LOGGER = logging.getLogger("funasr-service")
|
||||||
|
MAX_AUDIO_BYTES = 15 * 1024 * 1024
|
||||||
|
CHUNK_BYTES = 1024 * 1024
|
||||||
|
SUPPORTED_FORMATS = {"mp3", "aac", "wav", "m4a", "amr"}
|
||||||
|
|
||||||
|
|
||||||
|
class Settings:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.model = os.getenv("ASR_MODEL", "paraformer-zh")
|
||||||
|
self.vad_model = os.getenv("ASR_VAD_MODEL", "fsmn-vad")
|
||||||
|
self.punc_model = os.getenv("ASR_PUNC_MODEL", "ct-punc")
|
||||||
|
self.device = os.getenv("ASR_DEVICE", "cpu")
|
||||||
|
self.max_concurrency = positive_int("ASR_MAX_CONCURRENCY", 1)
|
||||||
|
self.batch_size_seconds = positive_int("ASR_BATCH_SIZE_SECONDS", 300)
|
||||||
|
|
||||||
|
|
||||||
|
class HealthResponse(BaseModel):
|
||||||
|
status: str
|
||||||
|
model: str
|
||||||
|
device: str
|
||||||
|
|
||||||
|
|
||||||
|
class TranscriptionResponse(BaseModel):
|
||||||
|
text: str
|
||||||
|
duration_ms: int
|
||||||
|
request_id: str
|
||||||
|
|
||||||
|
|
||||||
|
def positive_int(name: str, default: int) -> int:
|
||||||
|
raw = os.getenv(name, str(default))
|
||||||
|
try:
|
||||||
|
value = int(raw)
|
||||||
|
except ValueError as error:
|
||||||
|
raise RuntimeError(f"{name} must be an integer") from error
|
||||||
|
if value <= 0:
|
||||||
|
raise RuntimeError(f"{name} must be greater than 0")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def load_model(settings: Settings) -> Any:
|
||||||
|
LOGGER.info(
|
||||||
|
"loading FunASR model model=%s vad=%s punc=%s device=%s",
|
||||||
|
settings.model,
|
||||||
|
settings.vad_model,
|
||||||
|
settings.punc_model,
|
||||||
|
settings.device,
|
||||||
|
)
|
||||||
|
model = AutoModel(
|
||||||
|
model=settings.model,
|
||||||
|
vad_model=settings.vad_model,
|
||||||
|
punc_model=settings.punc_model,
|
||||||
|
device=settings.device,
|
||||||
|
disable_update=True,
|
||||||
|
)
|
||||||
|
LOGGER.info("FunASR model is ready")
|
||||||
|
return model
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def lifespan(app: FastAPI):
|
||||||
|
settings = Settings()
|
||||||
|
app.state.settings = settings
|
||||||
|
app.state.model = await asyncio.to_thread(load_model, settings)
|
||||||
|
app.state.inference_slots = asyncio.Semaphore(settings.max_concurrency)
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
|
app = FastAPI(
|
||||||
|
title="Teaching Feedback FunASR Service",
|
||||||
|
version="1.0.0",
|
||||||
|
lifespan=lifespan,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/health", response_model=HealthResponse)
|
||||||
|
async def health() -> HealthResponse:
|
||||||
|
settings: Settings = app.state.settings
|
||||||
|
return HealthResponse(status="ok", model=settings.model, device=settings.device)
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/v1/transcriptions", response_model=TranscriptionResponse)
|
||||||
|
async def create_transcription(
|
||||||
|
file: UploadFile = File(...),
|
||||||
|
language: str = Form("zh"),
|
||||||
|
duration_ms: int = Form(0),
|
||||||
|
hotword: str = Form(""),
|
||||||
|
) -> TranscriptionResponse:
|
||||||
|
if language not in {"zh", "zh-cn", "cmn"}:
|
||||||
|
raise HTTPException(status_code=422, detail="this service is configured for Chinese audio")
|
||||||
|
audio_format = file_extension(file.filename)
|
||||||
|
temporary_path = await save_upload(file, audio_format)
|
||||||
|
request_id = str(uuid.uuid4())
|
||||||
|
try:
|
||||||
|
async with app.state.inference_slots:
|
||||||
|
text = await asyncio.to_thread(
|
||||||
|
run_inference,
|
||||||
|
app.state.model,
|
||||||
|
temporary_path,
|
||||||
|
app.state.settings.batch_size_seconds,
|
||||||
|
hotword.strip(),
|
||||||
|
)
|
||||||
|
except Exception as error:
|
||||||
|
LOGGER.exception("FunASR inference failed request_id=%s", request_id)
|
||||||
|
raise HTTPException(status_code=500, detail=f"speech recognition failed: {error}") from error
|
||||||
|
finally:
|
||||||
|
temporary_path.unlink(missing_ok=True)
|
||||||
|
|
||||||
|
if not text:
|
||||||
|
raise HTTPException(status_code=422, detail="no clear speech was recognized")
|
||||||
|
return TranscriptionResponse(
|
||||||
|
text=text,
|
||||||
|
duration_ms=max(0, duration_ms),
|
||||||
|
request_id=request_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def file_extension(filename: str | None) -> str:
|
||||||
|
suffix = Path(filename or "segment.mp3").suffix.lower().lstrip(".")
|
||||||
|
if suffix not in SUPPORTED_FORMATS:
|
||||||
|
raise HTTPException(status_code=415, detail="unsupported audio format")
|
||||||
|
return suffix
|
||||||
|
|
||||||
|
|
||||||
|
async def save_upload(file: UploadFile, audio_format: str) -> Path:
|
||||||
|
handle = tempfile.NamedTemporaryFile(suffix=f".{audio_format}", delete=False)
|
||||||
|
path = Path(handle.name)
|
||||||
|
total_bytes = 0
|
||||||
|
try:
|
||||||
|
with handle:
|
||||||
|
while chunk := await file.read(CHUNK_BYTES):
|
||||||
|
total_bytes += len(chunk)
|
||||||
|
if total_bytes > MAX_AUDIO_BYTES:
|
||||||
|
raise HTTPException(status_code=413, detail="audio file exceeds 15 MB")
|
||||||
|
handle.write(chunk)
|
||||||
|
except Exception:
|
||||||
|
path.unlink(missing_ok=True)
|
||||||
|
raise
|
||||||
|
finally:
|
||||||
|
await file.close()
|
||||||
|
if total_bytes == 0:
|
||||||
|
path.unlink(missing_ok=True)
|
||||||
|
raise HTTPException(status_code=422, detail="audio file is empty")
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
def run_inference(model: Any, path: Path, batch_size_seconds: int, hotword: str) -> str:
|
||||||
|
options: dict[str, Any] = {
|
||||||
|
"input": str(path),
|
||||||
|
"batch_size_s": batch_size_seconds,
|
||||||
|
"use_itn": True,
|
||||||
|
}
|
||||||
|
if hotword:
|
||||||
|
options["hotword"] = hotword
|
||||||
|
result = model.generate(**options)
|
||||||
|
if not isinstance(result, list):
|
||||||
|
return ""
|
||||||
|
texts = [str(item.get("text", "")).strip() for item in result if isinstance(item, dict)]
|
||||||
|
return "\n".join(text for text in texts if text).strip()
|
||||||
7
asr-service/requirements.txt
Normal file
7
asr-service/requirements.txt
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
fastapi==0.128.8
|
||||||
|
funasr==1.3.14
|
||||||
|
modelscope==1.37.1
|
||||||
|
python-multipart==0.0.20
|
||||||
|
torch==2.8.0
|
||||||
|
torchaudio==2.8.0
|
||||||
|
uvicorn[standard]==0.39.0
|
||||||
31
compose.asr.yml
Normal file
31
compose.asr.yml
Normal file
@@ -0,0 +1,31 @@
|
|||||||
|
services:
|
||||||
|
funasr:
|
||||||
|
build:
|
||||||
|
context: ./asr-service
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:10095:10095"
|
||||||
|
environment:
|
||||||
|
ASR_MODEL: paraformer-zh
|
||||||
|
ASR_VAD_MODEL: fsmn-vad
|
||||||
|
ASR_PUNC_MODEL: ct-punc
|
||||||
|
ASR_DEVICE: cpu
|
||||||
|
ASR_MAX_CONCURRENCY: "1"
|
||||||
|
ASR_BATCH_SIZE_SECONDS: "300"
|
||||||
|
MODELSCOPE_CACHE: /models
|
||||||
|
volumes:
|
||||||
|
- funasr-model-cache:/models
|
||||||
|
restart: unless-stopped
|
||||||
|
healthcheck:
|
||||||
|
test:
|
||||||
|
- CMD
|
||||||
|
- python
|
||||||
|
- -c
|
||||||
|
- "import urllib.request; urllib.request.urlopen('http://127.0.0.1:10095/health', timeout=3)"
|
||||||
|
interval: 15s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 20
|
||||||
|
start_period: 5m
|
||||||
|
shm_size: 1gb
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
funasr-model-cache:
|
||||||
@@ -1,13 +1,54 @@
|
|||||||
import {
|
import {
|
||||||
createFeedbackRecord,
|
createFeedbackRecord,
|
||||||
|
createVoiceLesson,
|
||||||
|
ensureFeedbackSession,
|
||||||
|
finishVoiceLesson,
|
||||||
|
generateFeedbackFromVoice,
|
||||||
|
getActiveFeedbackSession,
|
||||||
getApiErrorMessage,
|
getApiErrorMessage,
|
||||||
listProfiles
|
listProfiles,
|
||||||
|
markVoiceLessonApplied,
|
||||||
|
retryVoiceSegment,
|
||||||
|
updateFeedbackSession,
|
||||||
|
uploadVoiceSegment
|
||||||
} from '../../utils/api'
|
} from '../../utils/api'
|
||||||
import type { FeedbackDraft, StudentProfile } from '../../utils/types'
|
import type {
|
||||||
|
FeedbackDraft,
|
||||||
|
FeedbackSession,
|
||||||
|
StudentProfile,
|
||||||
|
VoiceLessonFinished
|
||||||
|
} from '../../utils/types'
|
||||||
|
|
||||||
type PickerEvent = { detail: { value: string | number } }
|
type PickerEvent = { detail: { value: string | number } }
|
||||||
type InputEvent = { detail: { value: string } }
|
type InputEvent = { detail: { value: string } }
|
||||||
|
|
||||||
|
interface PendingVoiceUpload {
|
||||||
|
lessonId: string
|
||||||
|
sequence: number
|
||||||
|
durationMs: number
|
||||||
|
filePath: string
|
||||||
|
}
|
||||||
|
|
||||||
|
const MAX_CONTENT_LENGTH = 2000
|
||||||
|
const RECORDING_SEGMENT_DURATION_MS = 8 * 60 * 1000
|
||||||
|
const SHORT_RECORDING_DURATION_MS = 60 * 1000
|
||||||
|
const VOICE_POLL_INTERVAL_MS = 3000
|
||||||
|
const PENDING_UPLOADS_KEY = 'teaching-feedback-pending-voice-uploads-v1'
|
||||||
|
|
||||||
|
const recorderManager = wx.getRecorderManager()
|
||||||
|
let recordingTimer: number | null = null
|
||||||
|
let draftSaveTimer: number | null = null
|
||||||
|
let voicePollTimer: number | null = null
|
||||||
|
let voicePollBusy = false
|
||||||
|
let pageVisible = false
|
||||||
|
let lessonStartedAt = 0
|
||||||
|
let segmentStartedAt = 0
|
||||||
|
let currentLessonId: string | null = null
|
||||||
|
let currentSegmentSequence = 0
|
||||||
|
let finishRequested = false
|
||||||
|
let pendingUploads: Promise<void>[] = []
|
||||||
|
let failedLocalUploads: PendingVoiceUpload[] = []
|
||||||
|
|
||||||
function today(): string {
|
function today(): string {
|
||||||
const now = new Date()
|
const now = new Date()
|
||||||
const year = now.getFullYear()
|
const year = now.getFullYear()
|
||||||
@@ -20,39 +61,147 @@ function buildDraft(): FeedbackDraft {
|
|||||||
return {
|
return {
|
||||||
profileId: null,
|
profileId: null,
|
||||||
feedbackDate: today(),
|
feedbackDate: today(),
|
||||||
content: ''
|
content: '',
|
||||||
|
feedbackSessionId: null
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function appendTranscript(content: string, transcript: string): { content: string; truncated: boolean } {
|
||||||
|
const normalizedTranscript = transcript.trim()
|
||||||
|
const separator = content && !/\s$/.test(content) ? '\n' : ''
|
||||||
|
const combined = `${content}${separator}${normalizedTranscript}`
|
||||||
|
return {
|
||||||
|
content: combined.slice(0, MAX_CONTENT_LENGTH),
|
||||||
|
truncated: combined.length > MAX_CONTENT_LENGTH
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function formatRecordingTime(milliseconds: number): string {
|
||||||
|
const totalSeconds = Math.max(0, Math.floor(milliseconds / 1000))
|
||||||
|
const hours = Math.floor(totalSeconds / 3600)
|
||||||
|
const minutes = Math.floor((totalSeconds % 3600) / 60)
|
||||||
|
const seconds = totalSeconds % 60
|
||||||
|
const minuteText = String(minutes).padStart(2, '0')
|
||||||
|
const secondText = String(seconds).padStart(2, '0')
|
||||||
|
return hours > 0 ? `${String(hours).padStart(2, '0')}:${minuteText}:${secondText}` : `${minuteText}:${secondText}`
|
||||||
|
}
|
||||||
|
|
||||||
|
function formatVoiceDuration(milliseconds: number): string {
|
||||||
|
const totalMinutes = Math.floor(milliseconds / 60000)
|
||||||
|
if (totalMinutes < 1) return `${Math.max(1, Math.round(milliseconds / 1000))}秒`
|
||||||
|
return `${totalMinutes}分钟`
|
||||||
|
}
|
||||||
|
|
||||||
|
function wait(milliseconds: number): Promise<void> {
|
||||||
|
return new Promise((resolve) => setTimeout(resolve, milliseconds))
|
||||||
|
}
|
||||||
|
|
||||||
Page({
|
Page({
|
||||||
data: {
|
data: {
|
||||||
profiles: [] as StudentProfile[],
|
profiles: [] as StudentProfile[],
|
||||||
profileOptions: ['不关联学生'],
|
profileOptions: ['不关联学生'],
|
||||||
selectedProfileIndex: 0,
|
selectedProfileIndex: 0,
|
||||||
draft: buildDraft(),
|
draft: buildDraft(),
|
||||||
|
session: null as FeedbackSession | null,
|
||||||
contentLength: 0,
|
contentLength: 0,
|
||||||
loading: false,
|
loading: false,
|
||||||
saving: false,
|
saving: false,
|
||||||
|
generating: false,
|
||||||
|
voiceBusy: false,
|
||||||
|
recording: false,
|
||||||
|
recordingPaused: false,
|
||||||
|
recordingTime: '00:00',
|
||||||
|
voiceStatus: '语音录入',
|
||||||
|
voiceActionable: false,
|
||||||
|
primaryActionText: '保存反馈',
|
||||||
errorMessage: ''
|
errorMessage: ''
|
||||||
},
|
},
|
||||||
|
|
||||||
|
onLoad() {
|
||||||
|
failedLocalUploads = (wx.getStorageSync(PENDING_UPLOADS_KEY) as PendingVoiceUpload[]) || []
|
||||||
|
this.initRecorder()
|
||||||
|
if (failedLocalUploads.length > 0) {
|
||||||
|
currentLessonId = failedLocalUploads[0].lessonId
|
||||||
|
this.setData({ voiceStatus: '上传失败,点击重试', voiceActionable: true })
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
onShow() {
|
onShow() {
|
||||||
|
pageVisible = true
|
||||||
this.getTabBar()?.setData({ selected: 0 })
|
this.getTabBar()?.setData({ selected: 0 })
|
||||||
void this.loadProfiles()
|
void this.loadProfiles()
|
||||||
},
|
},
|
||||||
|
|
||||||
|
onHide() {
|
||||||
|
pageVisible = false
|
||||||
|
this.clearVoicePollTimer()
|
||||||
|
if (this.data.recording) this.stopRecording()
|
||||||
|
},
|
||||||
|
|
||||||
|
onUnload() {
|
||||||
|
pageVisible = false
|
||||||
|
this.clearRecordingTimer()
|
||||||
|
this.clearDraftSaveTimer()
|
||||||
|
this.clearVoicePollTimer()
|
||||||
|
if (this.data.recording) recorderManager.stop()
|
||||||
|
},
|
||||||
|
|
||||||
|
initRecorder() {
|
||||||
|
recorderManager.onStart(() => {
|
||||||
|
segmentStartedAt = Date.now()
|
||||||
|
this.setData({
|
||||||
|
voiceBusy: false,
|
||||||
|
recording: true,
|
||||||
|
recordingPaused: false,
|
||||||
|
voiceStatus: '正在录音'
|
||||||
|
})
|
||||||
|
this.startRecordingTimer()
|
||||||
|
})
|
||||||
|
|
||||||
|
recorderManager.onStop((result) => {
|
||||||
|
void this.handleRecorderStop(result)
|
||||||
|
})
|
||||||
|
|
||||||
|
recorderManager.onPause(() => {
|
||||||
|
this.setData({ recordingPaused: true, voiceStatus: '录音已暂停' })
|
||||||
|
})
|
||||||
|
|
||||||
|
recorderManager.onResume(() => {
|
||||||
|
this.setData({ recordingPaused: false, voiceStatus: '正在录音' })
|
||||||
|
})
|
||||||
|
|
||||||
|
recorderManager.onInterruptionEnd(() => {
|
||||||
|
if (this.data.recordingPaused) recorderManager.resume()
|
||||||
|
})
|
||||||
|
|
||||||
|
recorderManager.onError((error) => {
|
||||||
|
console.warn('RecorderManager error', error)
|
||||||
|
this.clearRecordingTimer()
|
||||||
|
finishRequested = true
|
||||||
|
this.setData({
|
||||||
|
recording: false,
|
||||||
|
recordingPaused: false,
|
||||||
|
voiceBusy: true,
|
||||||
|
voiceStatus: '正在保留已录内容'
|
||||||
|
})
|
||||||
|
void this.finalizeCurrentLesson()
|
||||||
|
})
|
||||||
|
},
|
||||||
|
|
||||||
async loadProfiles() {
|
async loadProfiles() {
|
||||||
this.setData({ loading: true, errorMessage: '' })
|
this.setData({ loading: true, errorMessage: '' })
|
||||||
try {
|
try {
|
||||||
const profiles = await listProfiles()
|
const profiles = await listProfiles()
|
||||||
const matchedProfileIndex = profiles.findIndex((profile) => profile.id === this.data.draft.profileId)
|
const matchedProfileIndex = profiles.findIndex((profile) => profile.id === this.data.draft.profileId)
|
||||||
const selectedProfileIndex = matchedProfileIndex + 1
|
const selectedProfileIndex = matchedProfileIndex + 1
|
||||||
|
const profileId = matchedProfileIndex >= 0 ? profiles[matchedProfileIndex].id : null
|
||||||
this.setData({
|
this.setData({
|
||||||
profiles,
|
profiles,
|
||||||
profileOptions: ['不关联学生', ...profiles.map((profile) => `${profile.name} · ${profile.subject}`)],
|
profileOptions: ['不关联学生', ...profiles.map((profile) => `${profile.name} · ${profile.subject}`)],
|
||||||
selectedProfileIndex,
|
selectedProfileIndex,
|
||||||
'draft.profileId': matchedProfileIndex >= 0 ? profiles[matchedProfileIndex].id : null
|
'draft.profileId': profileId
|
||||||
})
|
})
|
||||||
|
await this.loadActiveSession(profileId)
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
this.setData({ errorMessage: getApiErrorMessage(error) })
|
this.setData({ errorMessage: getApiErrorMessage(error) })
|
||||||
} finally {
|
} finally {
|
||||||
@@ -60,29 +209,483 @@ Page({
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
|
||||||
|
async loadActiveSession(profileId?: string | null) {
|
||||||
|
try {
|
||||||
|
const targetProfileId = profileId === undefined ? this.data.draft.profileId : profileId
|
||||||
|
const session = await getActiveFeedbackSession(targetProfileId)
|
||||||
|
if (targetProfileId !== this.data.draft.profileId) return
|
||||||
|
const nextData: Record<string, unknown> = {
|
||||||
|
session,
|
||||||
|
'draft.feedbackSessionId': session?.id || null
|
||||||
|
}
|
||||||
|
if (session) {
|
||||||
|
nextData['draft.feedbackDate'] = session.feedbackDate
|
||||||
|
nextData['draft.content'] = session.content
|
||||||
|
nextData.contentLength = session.content.length
|
||||||
|
}
|
||||||
|
this.setData(nextData)
|
||||||
|
this.syncVoicePresentation(session)
|
||||||
|
this.scheduleVoicePolling(session)
|
||||||
|
} catch (error) {
|
||||||
|
this.setData({ errorMessage: getApiErrorMessage(error) })
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
syncVoicePresentation(session: FeedbackSession | null) {
|
||||||
|
if (this.data.recording || this.data.voiceBusy) return
|
||||||
|
if (failedLocalUploads.length > 0) {
|
||||||
|
this.setData({ voiceStatus: '上传失败,点击重试', voiceActionable: true })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (!session || session.lessonCount === 0) {
|
||||||
|
this.setData({ voiceStatus: '语音录入', voiceActionable: false, primaryActionText: '保存反馈' })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
const prefix = `${session.lessonCount}节 · ${formatVoiceDuration(session.totalDurationMs)}`
|
||||||
|
let suffix = '已就绪'
|
||||||
|
if (session.failedSegmentCount > 0) suffix = '转录失败,点击重试'
|
||||||
|
else if (session.processingSegmentCount > 0) suffix = '正在处理'
|
||||||
|
let primaryActionText = session.needsGeneration ? '生成反馈' : '保存反馈'
|
||||||
|
if (session.processingSegmentCount > 0) primaryActionText = '转录中'
|
||||||
|
this.setData({
|
||||||
|
voiceStatus: `${prefix} · ${suffix}`,
|
||||||
|
voiceActionable: true,
|
||||||
|
primaryActionText
|
||||||
|
})
|
||||||
|
},
|
||||||
|
|
||||||
|
scheduleVoicePolling(session: FeedbackSession | null) {
|
||||||
|
this.clearVoicePollTimer()
|
||||||
|
if (!pageVisible || !session || session.processingSegmentCount === 0) return
|
||||||
|
voicePollTimer = setTimeout(() => {
|
||||||
|
voicePollTimer = null
|
||||||
|
void this.pollVoiceSession()
|
||||||
|
}, VOICE_POLL_INTERVAL_MS) as unknown as number
|
||||||
|
},
|
||||||
|
|
||||||
|
clearVoicePollTimer() {
|
||||||
|
if (voicePollTimer === null) return
|
||||||
|
clearTimeout(voicePollTimer)
|
||||||
|
voicePollTimer = null
|
||||||
|
},
|
||||||
|
|
||||||
|
async pollVoiceSession() {
|
||||||
|
if (!pageVisible) return
|
||||||
|
if (voicePollBusy || this.data.recording || this.data.voiceBusy) {
|
||||||
|
this.scheduleVoicePolling(this.data.session)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
voicePollBusy = true
|
||||||
|
const profileId = this.data.draft.profileId
|
||||||
|
try {
|
||||||
|
const session = await getActiveFeedbackSession(profileId)
|
||||||
|
if (!pageVisible || profileId !== this.data.draft.profileId) return
|
||||||
|
this.setData({ session, 'draft.feedbackSessionId': session?.id || null })
|
||||||
|
this.syncVoicePresentation(session)
|
||||||
|
if (!session) return
|
||||||
|
|
||||||
|
const readyShortLessons = session.lessons.filter(
|
||||||
|
(lesson) =>
|
||||||
|
lesson.status === 'ready' &&
|
||||||
|
lesson.durationMs <= SHORT_RECORDING_DURATION_MS &&
|
||||||
|
!lesson.appliedDirectly &&
|
||||||
|
!lesson.includedInGeneration
|
||||||
|
)
|
||||||
|
for (const lesson of readyShortLessons) {
|
||||||
|
const finished = await finishVoiceLesson(lesson.id)
|
||||||
|
await this.applyFinishedLesson(finished)
|
||||||
|
}
|
||||||
|
if (readyShortLessons.length > 0) await this.loadActiveSession(profileId)
|
||||||
|
} catch (error) {
|
||||||
|
console.warn('Failed to refresh voice transcription status', error)
|
||||||
|
} finally {
|
||||||
|
voicePollBusy = false
|
||||||
|
this.scheduleVoicePolling(this.data.session)
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
retryLoad() {
|
retryLoad() {
|
||||||
void this.loadProfiles()
|
void this.loadProfiles()
|
||||||
},
|
},
|
||||||
|
|
||||||
onProfileChange(event: PickerEvent) {
|
onProfileChange(event: PickerEvent) {
|
||||||
|
this.clearDraftSaveTimer()
|
||||||
const selectedProfileIndex = Number(event.detail.value)
|
const selectedProfileIndex = Number(event.detail.value)
|
||||||
const profile = this.data.profiles[selectedProfileIndex - 1]
|
const profile = this.data.profiles[selectedProfileIndex - 1]
|
||||||
|
const profileId = profile?.id || null
|
||||||
this.setData({
|
this.setData({
|
||||||
selectedProfileIndex,
|
selectedProfileIndex,
|
||||||
'draft.profileId': profile?.id || null
|
'draft.profileId': profileId,
|
||||||
|
session: null,
|
||||||
|
'draft.feedbackSessionId': null,
|
||||||
|
'draft.content': '',
|
||||||
|
contentLength: 0,
|
||||||
|
voiceActionable: false,
|
||||||
|
primaryActionText: '保存反馈'
|
||||||
})
|
})
|
||||||
|
void this.loadActiveSession(profileId)
|
||||||
},
|
},
|
||||||
|
|
||||||
onDateChange(event: InputEvent) {
|
onDateChange(event: InputEvent) {
|
||||||
this.setData({ 'draft.feedbackDate': event.detail.value })
|
this.setData({ 'draft.feedbackDate': event.detail.value })
|
||||||
|
this.scheduleDraftSave()
|
||||||
},
|
},
|
||||||
|
|
||||||
onContentInput(event: InputEvent) {
|
onContentInput(event: InputEvent) {
|
||||||
const content = event.detail.value
|
const content = event.detail.value
|
||||||
this.setData({ 'draft.content': content, contentLength: content.length })
|
this.setData({ 'draft.content': content, contentLength: content.length })
|
||||||
|
this.scheduleDraftSave()
|
||||||
|
},
|
||||||
|
|
||||||
|
scheduleDraftSave() {
|
||||||
|
this.clearDraftSaveTimer()
|
||||||
|
if (!this.data.session) return
|
||||||
|
draftSaveTimer = setTimeout(() => {
|
||||||
|
void this.persistSessionDraft()
|
||||||
|
}, 800) as unknown as number
|
||||||
|
},
|
||||||
|
|
||||||
|
clearDraftSaveTimer() {
|
||||||
|
if (draftSaveTimer === null) return
|
||||||
|
clearTimeout(draftSaveTimer)
|
||||||
|
draftSaveTimer = null
|
||||||
|
},
|
||||||
|
|
||||||
|
async persistSessionDraft() {
|
||||||
|
const session = this.data.session
|
||||||
|
if (!session) return
|
||||||
|
const sessionId = session.id
|
||||||
|
try {
|
||||||
|
const updated = await updateFeedbackSession(
|
||||||
|
sessionId,
|
||||||
|
this.data.draft.feedbackDate,
|
||||||
|
this.data.draft.content
|
||||||
|
)
|
||||||
|
if (this.data.session?.id !== sessionId) return
|
||||||
|
this.setData({ session: updated })
|
||||||
|
this.syncVoicePresentation(updated)
|
||||||
|
} catch (error) {
|
||||||
|
console.warn('Failed to auto-save feedback session', error)
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
ensureRecordPermission(): Promise<boolean> {
|
||||||
|
return new Promise((resolve) => {
|
||||||
|
wx.authorize({
|
||||||
|
scope: 'scope.record',
|
||||||
|
success: () => resolve(true),
|
||||||
|
fail: () => {
|
||||||
|
wx.showModal({
|
||||||
|
title: '需要麦克风权限',
|
||||||
|
content: '请在设置中允许使用麦克风,才能录音转录。',
|
||||||
|
confirmText: '去设置',
|
||||||
|
success: (modalResult) => {
|
||||||
|
if (!modalResult.confirm) {
|
||||||
|
resolve(false)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
wx.openSetting({
|
||||||
|
success: (settingResult) => resolve(settingResult.authSetting['scope.record'] === true),
|
||||||
|
fail: () => resolve(false)
|
||||||
|
})
|
||||||
|
},
|
||||||
|
fail: () => resolve(false)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
})
|
||||||
|
})
|
||||||
|
},
|
||||||
|
|
||||||
|
async toggleRecording() {
|
||||||
|
if (failedLocalUploads.length > 0) {
|
||||||
|
await this.retryPendingUploads()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (this.data.recording) {
|
||||||
|
this.stopRecording()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
this.setData({ voiceBusy: true, voiceStatus: '正在开启麦克风' })
|
||||||
|
const authorized = await this.ensureRecordPermission()
|
||||||
|
if (!authorized) {
|
||||||
|
this.setData({ voiceBusy: false })
|
||||||
|
this.syncVoicePresentation(this.data.session)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const session = await ensureFeedbackSession(
|
||||||
|
this.data.draft.profileId,
|
||||||
|
this.data.draft.feedbackDate,
|
||||||
|
this.data.draft.content
|
||||||
|
)
|
||||||
|
const lesson = await createVoiceLesson(session.id)
|
||||||
|
currentLessonId = lesson.id
|
||||||
|
currentSegmentSequence = 0
|
||||||
|
finishRequested = false
|
||||||
|
pendingUploads = []
|
||||||
|
lessonStartedAt = Date.now()
|
||||||
|
this.setData({
|
||||||
|
session,
|
||||||
|
'draft.feedbackSessionId': session.id,
|
||||||
|
recordingTime: '00:00'
|
||||||
|
})
|
||||||
|
this.startNextRecordingSegment()
|
||||||
|
} catch (error) {
|
||||||
|
this.setData({ voiceBusy: false })
|
||||||
|
this.syncVoicePresentation(this.data.session)
|
||||||
|
wx.showToast({ title: getApiErrorMessage(error), icon: 'none' })
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
startNextRecordingSegment() {
|
||||||
|
currentSegmentSequence += 1
|
||||||
|
recorderManager.start({
|
||||||
|
duration: RECORDING_SEGMENT_DURATION_MS,
|
||||||
|
sampleRate: 16000,
|
||||||
|
numberOfChannels: 1,
|
||||||
|
encodeBitRate: 48000,
|
||||||
|
format: 'mp3',
|
||||||
|
audioSource: 'auto'
|
||||||
|
})
|
||||||
|
},
|
||||||
|
|
||||||
|
stopRecording() {
|
||||||
|
if (!this.data.recording) return
|
||||||
|
finishRequested = true
|
||||||
|
this.clearRecordingTimer()
|
||||||
|
this.setData({
|
||||||
|
recording: false,
|
||||||
|
recordingPaused: false,
|
||||||
|
voiceBusy: true,
|
||||||
|
voiceStatus: '正在整理录音'
|
||||||
|
})
|
||||||
|
recorderManager.stop()
|
||||||
|
},
|
||||||
|
|
||||||
|
async handleRecorderStop(result: WechatMiniprogram.OnStopListenerResult) {
|
||||||
|
const lessonId = currentLessonId
|
||||||
|
if (!lessonId) return
|
||||||
|
const durationMs = Math.max(1, result.duration || Date.now() - segmentStartedAt)
|
||||||
|
const upload = this.queueSegmentUpload({
|
||||||
|
lessonId,
|
||||||
|
sequence: currentSegmentSequence,
|
||||||
|
durationMs,
|
||||||
|
filePath: result.tempFilePath
|
||||||
|
})
|
||||||
|
pendingUploads.push(upload)
|
||||||
|
|
||||||
|
if (!finishRequested) {
|
||||||
|
this.startNextRecordingSegment()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
await this.finalizeCurrentLesson()
|
||||||
|
},
|
||||||
|
|
||||||
|
async queueSegmentUpload(segment: PendingVoiceUpload): Promise<void> {
|
||||||
|
try {
|
||||||
|
await this.uploadWithRetry(segment)
|
||||||
|
} catch (error) {
|
||||||
|
console.warn('Voice segment upload failed', error)
|
||||||
|
const saved = await this.persistRecordingFile(segment)
|
||||||
|
failedLocalUploads.push(saved)
|
||||||
|
wx.setStorageSync(PENDING_UPLOADS_KEY, failedLocalUploads)
|
||||||
|
throw error
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
async uploadWithRetry(segment: PendingVoiceUpload): Promise<void> {
|
||||||
|
let lastError: unknown
|
||||||
|
for (let attempt = 0; attempt < 2; attempt += 1) {
|
||||||
|
try {
|
||||||
|
await uploadVoiceSegment(
|
||||||
|
segment.lessonId,
|
||||||
|
segment.filePath,
|
||||||
|
segment.sequence,
|
||||||
|
segment.durationMs
|
||||||
|
)
|
||||||
|
return
|
||||||
|
} catch (error) {
|
||||||
|
lastError = error
|
||||||
|
if (attempt === 0) await wait(800)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
throw lastError
|
||||||
|
},
|
||||||
|
|
||||||
|
persistRecordingFile(segment: PendingVoiceUpload): Promise<PendingVoiceUpload> {
|
||||||
|
return new Promise((resolve) => {
|
||||||
|
wx.getFileSystemManager().saveFile({
|
||||||
|
tempFilePath: segment.filePath,
|
||||||
|
success: (result) => resolve({ ...segment, filePath: result.savedFilePath }),
|
||||||
|
fail: () => resolve(segment)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
},
|
||||||
|
|
||||||
|
async retryPendingUploads() {
|
||||||
|
if (failedLocalUploads.length === 0) return
|
||||||
|
this.setData({ voiceBusy: true, voiceStatus: '正在重试上传' })
|
||||||
|
const remaining: PendingVoiceUpload[] = []
|
||||||
|
for (const segment of failedLocalUploads) {
|
||||||
|
try {
|
||||||
|
await this.uploadWithRetry(segment)
|
||||||
|
this.removeSavedFile(segment.filePath)
|
||||||
|
} catch {
|
||||||
|
remaining.push(segment)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
failedLocalUploads = remaining
|
||||||
|
wx.setStorageSync(PENDING_UPLOADS_KEY, failedLocalUploads)
|
||||||
|
if (remaining.length > 0) {
|
||||||
|
this.setData({ voiceBusy: false, voiceStatus: '上传失败,点击重试', voiceActionable: true })
|
||||||
|
wx.showToast({ title: '录音上传失败,请检查网络', icon: 'none' })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
await this.finalizeCurrentLesson()
|
||||||
|
},
|
||||||
|
|
||||||
|
removeSavedFile(filePath: string) {
|
||||||
|
wx.getFileSystemManager().unlink({ filePath, fail: () => undefined })
|
||||||
|
},
|
||||||
|
|
||||||
|
async finalizeCurrentLesson() {
|
||||||
|
const lessonId = currentLessonId
|
||||||
|
if (!lessonId) {
|
||||||
|
this.setData({ voiceBusy: false, recording: false })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
const results = await Promise.allSettled(pendingUploads)
|
||||||
|
pendingUploads = []
|
||||||
|
if (results.some((result) => result.status === 'rejected') || failedLocalUploads.length > 0) {
|
||||||
|
this.setData({
|
||||||
|
recording: false,
|
||||||
|
voiceBusy: false,
|
||||||
|
voiceStatus: '上传失败,点击重试',
|
||||||
|
voiceActionable: true
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const finished = await finishVoiceLesson(lessonId)
|
||||||
|
await this.applyFinishedLesson(finished)
|
||||||
|
currentLessonId = null
|
||||||
|
currentSegmentSequence = 0
|
||||||
|
finishRequested = false
|
||||||
|
await this.loadActiveSession()
|
||||||
|
} catch (error) {
|
||||||
|
wx.showToast({ title: getApiErrorMessage(error), icon: 'none' })
|
||||||
|
} finally {
|
||||||
|
this.clearRecordingTimer()
|
||||||
|
this.setData({ recording: false, recordingPaused: false, voiceBusy: false, recordingTime: '00:00' })
|
||||||
|
this.syncVoicePresentation(this.data.session)
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
async applyFinishedLesson(finished: VoiceLessonFinished) {
|
||||||
|
if (finished.status === 'processing') {
|
||||||
|
wx.showToast({ title: '录音已上传,正在后台转写', icon: 'none' })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (finished.status === 'failed') {
|
||||||
|
wx.showToast({ title: '部分录音转录失败,点击状态可重试', icon: 'none' })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (!finished.canApplyDirectly) {
|
||||||
|
wx.showToast({ title: '录音已就绪,可生成反馈', icon: 'none' })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
const merged = appendTranscript(this.data.draft.content, finished.transcript)
|
||||||
|
this.setData({ 'draft.content': merged.content, contentLength: merged.content.length })
|
||||||
|
if (!merged.truncated) {
|
||||||
|
await markVoiceLessonApplied(finished.id, merged.content)
|
||||||
|
wx.showToast({ title: '语音已转为文字', icon: 'none' })
|
||||||
|
} else {
|
||||||
|
wx.showToast({ title: '内容已达上限,请生成汇总反馈', icon: 'none' })
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
startRecordingTimer() {
|
||||||
|
this.clearRecordingTimer()
|
||||||
|
recordingTimer = setInterval(() => {
|
||||||
|
this.setData({ recordingTime: formatRecordingTime(Date.now() - lessonStartedAt) })
|
||||||
|
}, 1000) as unknown as number
|
||||||
|
},
|
||||||
|
|
||||||
|
clearRecordingTimer() {
|
||||||
|
if (recordingTimer === null) return
|
||||||
|
clearInterval(recordingTimer)
|
||||||
|
recordingTimer = null
|
||||||
|
},
|
||||||
|
|
||||||
|
async openVoiceDetails() {
|
||||||
|
if (this.data.recording || this.data.voiceBusy) return
|
||||||
|
if (failedLocalUploads.length > 0) {
|
||||||
|
await this.retryPendingUploads()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
const session = this.data.session
|
||||||
|
if (!session || session.lessonCount === 0) return
|
||||||
|
const lines = session.lessons.map((lesson) => {
|
||||||
|
const status = lesson.status === 'ready' ? '已就绪' : lesson.status === 'failed' ? '转录失败' : '处理中'
|
||||||
|
return `第${lesson.sequence}节 · ${formatVoiceDuration(lesson.durationMs)} · ${status}`
|
||||||
|
})
|
||||||
|
const hasFailures = session.failedSegmentCount > 0
|
||||||
|
wx.showModal({
|
||||||
|
title: '语音记录',
|
||||||
|
content: lines.join('\n'),
|
||||||
|
showCancel: hasFailures,
|
||||||
|
cancelText: '关闭',
|
||||||
|
confirmText: hasFailures ? '重试失败项' : '知道了',
|
||||||
|
success: (result) => {
|
||||||
|
if (hasFailures && result.confirm) void this.retryFailedSegments()
|
||||||
|
}
|
||||||
|
})
|
||||||
|
},
|
||||||
|
|
||||||
|
async retryFailedSegments() {
|
||||||
|
const session = this.data.session
|
||||||
|
if (!session) return
|
||||||
|
const failedSegments = session.lessons.flatMap((lesson) =>
|
||||||
|
lesson.segments.filter((segment) => segment.status === 'failed')
|
||||||
|
)
|
||||||
|
if (failedSegments.length === 0) return
|
||||||
|
this.setData({ voiceBusy: true, voiceStatus: '正在重试转录' })
|
||||||
|
try {
|
||||||
|
for (const segment of failedSegments) await retryVoiceSegment(segment.id)
|
||||||
|
const failedLessonIds = session.lessons
|
||||||
|
.filter((lesson) => lesson.segments.some((segment) => segment.status === 'failed'))
|
||||||
|
.map((lesson) => lesson.id)
|
||||||
|
for (const lessonId of failedLessonIds) {
|
||||||
|
const finished = await finishVoiceLesson(lessonId)
|
||||||
|
await this.applyFinishedLesson(finished)
|
||||||
|
}
|
||||||
|
await this.loadActiveSession()
|
||||||
|
} catch (error) {
|
||||||
|
wx.showToast({ title: getApiErrorMessage(error), icon: 'none' })
|
||||||
|
} finally {
|
||||||
|
this.setData({ voiceBusy: false })
|
||||||
|
this.syncVoicePresentation(this.data.session)
|
||||||
|
}
|
||||||
},
|
},
|
||||||
|
|
||||||
async saveFeedback() {
|
async saveFeedback() {
|
||||||
|
this.clearDraftSaveTimer()
|
||||||
|
const session = this.data.session
|
||||||
|
if (session && session.processingSegmentCount > 0) {
|
||||||
|
wx.showToast({ title: '录音仍在转写,请稍候', icon: 'none' })
|
||||||
|
this.scheduleVoicePolling(session)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (session?.needsGeneration) {
|
||||||
|
await this.generateFeedback()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
const content = this.data.draft.content.trim()
|
const content = this.data.draft.content.trim()
|
||||||
if (!content) {
|
if (!content) {
|
||||||
wx.showToast({ title: '请填写反馈内容', icon: 'none' })
|
wx.showToast({ title: '请填写反馈内容', icon: 'none' })
|
||||||
@@ -91,8 +694,20 @@ Page({
|
|||||||
|
|
||||||
this.setData({ saving: true })
|
this.setData({ saving: true })
|
||||||
try {
|
try {
|
||||||
await createFeedbackRecord({ ...this.data.draft, content })
|
await createFeedbackRecord({
|
||||||
this.setData({ draft: buildDraft(), selectedProfileIndex: 0, contentLength: 0 })
|
...this.data.draft,
|
||||||
|
content,
|
||||||
|
feedbackSessionId: session?.id || null
|
||||||
|
})
|
||||||
|
this.setData({
|
||||||
|
draft: buildDraft(),
|
||||||
|
session: null,
|
||||||
|
selectedProfileIndex: 0,
|
||||||
|
contentLength: 0,
|
||||||
|
voiceStatus: '语音录入',
|
||||||
|
voiceActionable: false,
|
||||||
|
primaryActionText: '保存反馈'
|
||||||
|
})
|
||||||
wx.showToast({ title: '反馈已保存', icon: 'success' })
|
wx.showToast({ title: '反馈已保存', icon: 'success' })
|
||||||
setTimeout(() => wx.switchTab({ url: '/pages/records/index' }), 450)
|
setTimeout(() => wx.switchTab({ url: '/pages/records/index' }), 450)
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
@@ -102,6 +717,26 @@ Page({
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
|
||||||
|
async generateFeedback() {
|
||||||
|
const session = this.data.session
|
||||||
|
if (!session) return
|
||||||
|
this.setData({ generating: true, voiceBusy: true, voiceStatus: '正在生成反馈' })
|
||||||
|
try {
|
||||||
|
const result = await generateFeedbackFromVoice(session.id, this.data.draft.content)
|
||||||
|
this.setData({
|
||||||
|
'draft.content': result.content,
|
||||||
|
contentLength: result.content.length
|
||||||
|
})
|
||||||
|
await this.loadActiveSession()
|
||||||
|
wx.showToast({ title: '反馈已生成,可继续修改', icon: 'none' })
|
||||||
|
} catch (error) {
|
||||||
|
wx.showToast({ title: getApiErrorMessage(error), icon: 'none' })
|
||||||
|
} finally {
|
||||||
|
this.setData({ generating: false, voiceBusy: false })
|
||||||
|
this.syncVoicePresentation(this.data.session)
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
goStudents() {
|
goStudents() {
|
||||||
wx.switchTab({ url: '/pages/students/index' })
|
wx.switchTab({ url: '/pages/students/index' })
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,12 +12,12 @@
|
|||||||
</view>
|
</view>
|
||||||
|
|
||||||
<view class="feedback-form surface">
|
<view class="feedback-form surface">
|
||||||
<picker class="feedback-field" mode="selector" range="{{profileOptions}}" value="{{selectedProfileIndex}}" disabled="{{loading}}" bindchange="onProfileChange">
|
<picker class="feedback-field" mode="selector" range="{{profileOptions}}" value="{{selectedProfileIndex}}" disabled="{{loading || recording || voiceBusy}}" bindchange="onProfileChange">
|
||||||
<text class="form-label">学生档案</text>
|
<text class="form-label">学生档案</text>
|
||||||
<text class="form-value">{{loading ? '正在加载...' : profileOptions[selectedProfileIndex]}}</text>
|
<text class="form-value">{{loading ? '正在加载...' : profileOptions[selectedProfileIndex]}}</text>
|
||||||
</picker>
|
</picker>
|
||||||
|
|
||||||
<picker class="feedback-field" mode="date" value="{{draft.feedbackDate}}" bindchange="onDateChange">
|
<picker class="feedback-field" mode="date" value="{{draft.feedbackDate}}" disabled="{{recording || voiceBusy}}" bindchange="onDateChange">
|
||||||
<text class="form-label">反馈日期</text>
|
<text class="form-label">反馈日期</text>
|
||||||
<text class="form-value">{{draft.feedbackDate}}</text>
|
<text class="form-value">{{draft.feedbackDate}}</text>
|
||||||
</picker>
|
</picker>
|
||||||
@@ -31,12 +31,30 @@
|
|||||||
class="feedback-textarea"
|
class="feedback-textarea"
|
||||||
value="{{draft.content}}"
|
value="{{draft.content}}"
|
||||||
maxlength="2000"
|
maxlength="2000"
|
||||||
|
disabled="{{recording || voiceBusy}}"
|
||||||
placeholder="记录课堂完成情况、进步与后续建议"
|
placeholder="记录课堂完成情况、进步与后续建议"
|
||||||
bindinput="onContentInput"
|
bindinput="onContentInput"
|
||||||
/>
|
/>
|
||||||
|
|
||||||
|
<view class="voice-input-row">
|
||||||
|
<button
|
||||||
|
class="voice-button {{recording ? 'voice-button-recording' : ''}}"
|
||||||
|
disabled="{{saving || voiceBusy}}"
|
||||||
|
bindtap="toggleRecording"
|
||||||
|
aria-label="{{recording ? '结束录音' : '开始录音'}}"
|
||||||
|
>
|
||||||
|
<text class="voice-button-icon">{{recording ? '■' : '●'}}</text>
|
||||||
|
<text>{{recording ? '结束录音' : '语音录入'}}</text>
|
||||||
|
</button>
|
||||||
|
<view class="voice-state {{voiceActionable ? 'voice-state-actionable' : ''}}" bindtap="openVoiceDetails">
|
||||||
|
<text class="voice-status">{{voiceStatus}}</text>
|
||||||
|
<text wx:if="{{recording}}" class="recording-time">{{recordingTime}}</text>
|
||||||
|
<text wx:elif="{{voiceActionable}}" class="voice-state-arrow">›</text>
|
||||||
|
</view>
|
||||||
|
</view>
|
||||||
</view>
|
</view>
|
||||||
|
|
||||||
<button class="save-feedback primary-button" loading="{{saving}}" disabled="{{saving}}" bindtap="saveFeedback">保存反馈</button>
|
<button class="save-feedback primary-button" loading="{{saving || generating}}" disabled="{{saving || generating || recording || voiceBusy || (session && session.processingSegmentCount > 0)}}" bindtap="saveFeedback">{{primaryActionText}}</button>
|
||||||
</view>
|
</view>
|
||||||
|
|
||||||
<view wx:if="{{!loading && profiles.length === 0}}" class="profiles-hint">
|
<view wx:if="{{!loading && profiles.length === 0}}" class="profiles-hint">
|
||||||
|
|||||||
@@ -42,6 +42,91 @@
|
|||||||
line-height: 1.6;
|
line-height: 1.6;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.voice-input-row {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 20rpx;
|
||||||
|
margin-top: 18rpx;
|
||||||
|
padding-top: 18rpx;
|
||||||
|
border-top: 2rpx solid #e7eef6;
|
||||||
|
}
|
||||||
|
|
||||||
|
.voice-button {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
gap: 12rpx;
|
||||||
|
width: 220rpx;
|
||||||
|
height: 72rpx;
|
||||||
|
margin: 0;
|
||||||
|
padding: 0 20rpx;
|
||||||
|
border: 2rpx solid #b8cbe1;
|
||||||
|
border-radius: 12rpx;
|
||||||
|
background: #ffffff;
|
||||||
|
color: #2b67e8;
|
||||||
|
font-size: 26rpx;
|
||||||
|
font-weight: 600;
|
||||||
|
line-height: 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
.voice-button::after {
|
||||||
|
border: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.voice-button[disabled] {
|
||||||
|
background: #f4f7fa;
|
||||||
|
color: #99a5b3;
|
||||||
|
opacity: 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
.voice-button-recording,
|
||||||
|
.voice-button-recording[disabled] {
|
||||||
|
border-color: #e46b65;
|
||||||
|
background: #fff4f3;
|
||||||
|
color: #c83f39;
|
||||||
|
}
|
||||||
|
|
||||||
|
.voice-button-icon {
|
||||||
|
font-size: 20rpx;
|
||||||
|
line-height: 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
.voice-state {
|
||||||
|
display: flex;
|
||||||
|
flex: 1;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: space-between;
|
||||||
|
min-width: 0;
|
||||||
|
color: #65758a;
|
||||||
|
font-size: 24rpx;
|
||||||
|
}
|
||||||
|
|
||||||
|
.voice-status {
|
||||||
|
flex: 1;
|
||||||
|
overflow: hidden;
|
||||||
|
text-overflow: ellipsis;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.voice-state-actionable {
|
||||||
|
min-height: 64rpx;
|
||||||
|
}
|
||||||
|
|
||||||
|
.voice-state-arrow {
|
||||||
|
flex: none;
|
||||||
|
margin-left: 10rpx;
|
||||||
|
color: #7f8da0;
|
||||||
|
font-size: 38rpx;
|
||||||
|
line-height: 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
.recording-time {
|
||||||
|
flex: none;
|
||||||
|
margin-left: 12rpx;
|
||||||
|
color: #c83f39;
|
||||||
|
font-variant-numeric: tabular-nums;
|
||||||
|
}
|
||||||
|
|
||||||
.save-feedback {
|
.save-feedback {
|
||||||
margin-top: 24rpx;
|
margin-top: 24rpx;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -41,9 +41,21 @@ Page({
|
|||||||
this.setData({ testing: true, connectionText: '正在检测服务...', connectionTone: 'idle' })
|
this.setData({ testing: true, connectionText: '正在检测服务...', connectionTone: 'idle' })
|
||||||
try {
|
try {
|
||||||
const health = await getHealth()
|
const health = await getHealth()
|
||||||
|
let connectionText = '服务、数据库和语音转录正常'
|
||||||
|
let connectionTone = 'success'
|
||||||
|
if (!health.databaseConfigured) {
|
||||||
|
connectionText = '服务正常,数据库尚未配置'
|
||||||
|
connectionTone = 'warning'
|
||||||
|
} else if (!health.speechConfigured) {
|
||||||
|
connectionText = '服务和数据库正常,语音转录尚未配置'
|
||||||
|
connectionTone = 'warning'
|
||||||
|
} else if (!health.speechAvailable) {
|
||||||
|
connectionText = '服务和数据库正常,本地语音模型尚未就绪'
|
||||||
|
connectionTone = 'warning'
|
||||||
|
}
|
||||||
this.setData({
|
this.setData({
|
||||||
connectionText: health.databaseConfigured ? '服务和数据库连接正常' : '服务正常,数据库尚未配置',
|
connectionText,
|
||||||
connectionTone: health.databaseConfigured ? 'success' : 'warning'
|
connectionTone
|
||||||
})
|
})
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
this.setData({ connectionText: getApiErrorMessage(error), connectionTone: 'error' })
|
this.setData({ connectionText: getApiErrorMessage(error), connectionTone: 'error' })
|
||||||
|
|||||||
@@ -37,6 +37,10 @@
|
|||||||
"value": "server",
|
"value": "server",
|
||||||
"type": "folder"
|
"type": "folder"
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
"value": "asr-service",
|
||||||
|
"type": "folder"
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"value": "node_modules",
|
"value": "node_modules",
|
||||||
"type": "folder"
|
"type": "folder"
|
||||||
@@ -44,6 +48,14 @@
|
|||||||
{
|
{
|
||||||
"value": "graphify-out",
|
"value": "graphify-out",
|
||||||
"type": "folder"
|
"type": "folder"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": ".git",
|
||||||
|
"type": "folder"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "compose.asr.yml",
|
||||||
|
"type": "file"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"include": []
|
"include": []
|
||||||
@@ -51,4 +63,4 @@
|
|||||||
"appid": "wx3fe11e262a4b4885",
|
"appid": "wx3fe11e262a4b4885",
|
||||||
"simulatorPluginLibVersion": {},
|
"simulatorPluginLibVersion": {},
|
||||||
"editorSetting": {}
|
"editorSetting": {}
|
||||||
}
|
}
|
||||||
|
|||||||
257
utils/api.ts
257
utils/api.ts
@@ -1,11 +1,18 @@
|
|||||||
import type {
|
import type {
|
||||||
FeedbackDraft,
|
FeedbackDraft,
|
||||||
FeedbackRecord,
|
FeedbackRecord,
|
||||||
|
FeedbackSession,
|
||||||
|
GeneratedFeedback,
|
||||||
HealthStatus,
|
HealthStatus,
|
||||||
StudentDraft,
|
StudentDraft,
|
||||||
StudentProfile,
|
StudentProfile,
|
||||||
StudentProfileDefaults,
|
StudentProfileDefaults,
|
||||||
Term
|
Term,
|
||||||
|
VoiceAudioSegment,
|
||||||
|
VoiceLesson,
|
||||||
|
VoiceLessonCreated,
|
||||||
|
VoiceLessonFinished,
|
||||||
|
VoiceProcessingStatus
|
||||||
} from './types'
|
} from './types'
|
||||||
|
|
||||||
const DEFAULT_API_BASE_URL = 'http://127.0.0.1:8080'
|
const DEFAULT_API_BASE_URL = 'http://127.0.0.1:8080'
|
||||||
@@ -50,6 +57,66 @@ interface RawFeedbackRecord {
|
|||||||
interface RawHealthStatus {
|
interface RawHealthStatus {
|
||||||
status: string
|
status: string
|
||||||
database_configured: boolean
|
database_configured: boolean
|
||||||
|
speech_configured: boolean
|
||||||
|
speech_available: boolean
|
||||||
|
speech_provider: string | null
|
||||||
|
summary_configured: boolean
|
||||||
|
}
|
||||||
|
|
||||||
|
interface RawVoiceAudioSegment {
|
||||||
|
id: string
|
||||||
|
sequence: number
|
||||||
|
duration_ms: number
|
||||||
|
status: 'processing' | 'ready' | 'failed'
|
||||||
|
error_message: string | null
|
||||||
|
}
|
||||||
|
|
||||||
|
interface RawVoiceLesson {
|
||||||
|
id: string
|
||||||
|
sequence: number
|
||||||
|
status: VoiceProcessingStatus
|
||||||
|
duration_ms: number
|
||||||
|
applied_directly: boolean
|
||||||
|
included_in_generation: boolean
|
||||||
|
segments: RawVoiceAudioSegment[]
|
||||||
|
started_at: string
|
||||||
|
ended_at: string | null
|
||||||
|
}
|
||||||
|
|
||||||
|
interface RawFeedbackSession {
|
||||||
|
id: string
|
||||||
|
profile_id: string | null
|
||||||
|
feedback_date: string
|
||||||
|
content: string
|
||||||
|
status: 'active' | 'finalized'
|
||||||
|
lesson_count: number
|
||||||
|
total_duration_ms: number
|
||||||
|
processing_segment_count: number
|
||||||
|
failed_segment_count: number
|
||||||
|
needs_generation: boolean
|
||||||
|
lessons: RawVoiceLesson[]
|
||||||
|
created_at: string
|
||||||
|
updated_at: string
|
||||||
|
}
|
||||||
|
|
||||||
|
interface RawVoiceLessonCreated {
|
||||||
|
id: string
|
||||||
|
sequence: number
|
||||||
|
status: VoiceProcessingStatus
|
||||||
|
started_at: string
|
||||||
|
}
|
||||||
|
|
||||||
|
interface RawVoiceLessonFinished {
|
||||||
|
id: string
|
||||||
|
status: VoiceProcessingStatus
|
||||||
|
duration_ms: number
|
||||||
|
transcript: string
|
||||||
|
can_apply_directly: boolean
|
||||||
|
}
|
||||||
|
|
||||||
|
interface RawGeneratedFeedback {
|
||||||
|
content: string
|
||||||
|
method: 'llm' | 'extractive'
|
||||||
}
|
}
|
||||||
|
|
||||||
type HttpMethod = 'GET' | 'POST' | 'PUT' | 'DELETE'
|
type HttpMethod = 'GET' | 'POST' | 'PUT' | 'DELETE'
|
||||||
@@ -81,17 +148,17 @@ export function getApiErrorMessage(error: unknown): string {
|
|||||||
if (!(error instanceof ApiRequestError)) return '请求失败,请稍后重试'
|
if (!(error instanceof ApiRequestError)) return '请求失败,请稍后重试'
|
||||||
if (error.statusCode === 401) return '开发身份未通过后端校验'
|
if (error.statusCode === 401) return '开发身份未通过后端校验'
|
||||||
if (error.statusCode === 404) return '请求的数据不存在或无权访问'
|
if (error.statusCode === 404) return '请求的数据不存在或无权访问'
|
||||||
if (error.statusCode === 503) return '后端尚未配置数据库连接'
|
if (error.statusCode === 503 && error.message.includes('database')) return '后端尚未配置数据库连接'
|
||||||
return error.message
|
return error.message
|
||||||
}
|
}
|
||||||
|
|
||||||
function request<T>(path: string, method: HttpMethod = 'GET', data?: object): Promise<T> {
|
function request<T>(path: string, method: HttpMethod = 'GET', data?: object, timeout = 10000): Promise<T> {
|
||||||
return new Promise((resolve, reject) => {
|
return new Promise((resolve, reject) => {
|
||||||
wx.request({
|
wx.request({
|
||||||
url: `${getApiBaseUrl()}${path}`,
|
url: `${getApiBaseUrl()}${path}`,
|
||||||
method,
|
method,
|
||||||
data,
|
data,
|
||||||
timeout: 10000,
|
timeout,
|
||||||
header: {
|
header: {
|
||||||
'Content-Type': 'application/json',
|
'Content-Type': 'application/json',
|
||||||
'X-User-Id': DEVELOPMENT_USER_ID
|
'X-User-Id': DEVELOPMENT_USER_ID
|
||||||
@@ -165,11 +232,57 @@ function mapFeedbackRecord(record: RawFeedbackRecord): FeedbackRecord {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function mapVoiceSegment(segment: RawVoiceAudioSegment): VoiceAudioSegment {
|
||||||
|
return {
|
||||||
|
id: segment.id,
|
||||||
|
sequence: segment.sequence,
|
||||||
|
durationMs: segment.duration_ms,
|
||||||
|
status: segment.status,
|
||||||
|
errorMessage: segment.error_message
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function mapVoiceLesson(lesson: RawVoiceLesson): VoiceLesson {
|
||||||
|
return {
|
||||||
|
id: lesson.id,
|
||||||
|
sequence: lesson.sequence,
|
||||||
|
status: lesson.status,
|
||||||
|
durationMs: lesson.duration_ms,
|
||||||
|
appliedDirectly: lesson.applied_directly,
|
||||||
|
includedInGeneration: lesson.included_in_generation,
|
||||||
|
segments: lesson.segments.map(mapVoiceSegment),
|
||||||
|
startedAt: parseTimestamp(lesson.started_at),
|
||||||
|
endedAt: lesson.ended_at ? parseTimestamp(lesson.ended_at) : null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function mapFeedbackSession(session: RawFeedbackSession): FeedbackSession {
|
||||||
|
return {
|
||||||
|
id: session.id,
|
||||||
|
profileId: session.profile_id,
|
||||||
|
feedbackDate: session.feedback_date,
|
||||||
|
content: session.content,
|
||||||
|
status: session.status,
|
||||||
|
lessonCount: session.lesson_count,
|
||||||
|
totalDurationMs: session.total_duration_ms,
|
||||||
|
processingSegmentCount: session.processing_segment_count,
|
||||||
|
failedSegmentCount: session.failed_segment_count,
|
||||||
|
needsGeneration: session.needs_generation,
|
||||||
|
lessons: session.lessons.map(mapVoiceLesson),
|
||||||
|
createdAt: parseTimestamp(session.created_at),
|
||||||
|
updatedAt: parseTimestamp(session.updated_at)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
export async function getHealth(): Promise<HealthStatus> {
|
export async function getHealth(): Promise<HealthStatus> {
|
||||||
const result = await request<RawHealthStatus>('/health')
|
const result = await request<RawHealthStatus>('/health')
|
||||||
return {
|
return {
|
||||||
status: result.status,
|
status: result.status,
|
||||||
databaseConfigured: result.database_configured
|
databaseConfigured: result.database_configured,
|
||||||
|
speechConfigured: result.speech_configured,
|
||||||
|
speechAvailable: result.speech_available,
|
||||||
|
speechProvider: result.speech_provider,
|
||||||
|
summaryConfigured: result.summary_configured
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -209,7 +322,8 @@ export async function createFeedbackRecord(draft: FeedbackDraft): Promise<Feedba
|
|||||||
const result = await request<RawFeedbackRecord>('/api/v1/feedback-records', 'POST', {
|
const result = await request<RawFeedbackRecord>('/api/v1/feedback-records', 'POST', {
|
||||||
profile_id: draft.profileId,
|
profile_id: draft.profileId,
|
||||||
feedback_date: draft.feedbackDate,
|
feedback_date: draft.feedbackDate,
|
||||||
content: draft.content
|
content: draft.content,
|
||||||
|
feedback_session_id: draft.feedbackSessionId || null
|
||||||
})
|
})
|
||||||
return mapFeedbackRecord(result)
|
return mapFeedbackRecord(result)
|
||||||
}
|
}
|
||||||
@@ -217,3 +331,134 @@ export async function createFeedbackRecord(draft: FeedbackDraft): Promise<Feedba
|
|||||||
export async function deleteFeedbackRecord(recordId: string): Promise<void> {
|
export async function deleteFeedbackRecord(recordId: string): Promise<void> {
|
||||||
await request<unknown>(`/api/v1/feedback-records/${encodeURIComponent(recordId)}`, 'DELETE')
|
await request<unknown>(`/api/v1/feedback-records/${encodeURIComponent(recordId)}`, 'DELETE')
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export async function getActiveFeedbackSession(profileId: string | null): Promise<FeedbackSession | null> {
|
||||||
|
const query = profileId ? `?profile_id=${encodeURIComponent(profileId)}` : ''
|
||||||
|
const result = await request<RawFeedbackSession | null>(`/api/v1/feedback-sessions/active${query}`)
|
||||||
|
return result ? mapFeedbackSession(result) : null
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function ensureFeedbackSession(
|
||||||
|
profileId: string | null,
|
||||||
|
feedbackDate: string,
|
||||||
|
content: string
|
||||||
|
): Promise<FeedbackSession> {
|
||||||
|
const result = await request<RawFeedbackSession>('/api/v1/feedback-sessions', 'POST', {
|
||||||
|
profile_id: profileId,
|
||||||
|
feedback_date: feedbackDate,
|
||||||
|
content
|
||||||
|
})
|
||||||
|
return mapFeedbackSession(result)
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function updateFeedbackSession(
|
||||||
|
sessionId: string,
|
||||||
|
feedbackDate: string,
|
||||||
|
content: string
|
||||||
|
): Promise<FeedbackSession> {
|
||||||
|
const result = await request<RawFeedbackSession>(
|
||||||
|
`/api/v1/feedback-sessions/${encodeURIComponent(sessionId)}`,
|
||||||
|
'PUT',
|
||||||
|
{ feedback_date: feedbackDate, content }
|
||||||
|
)
|
||||||
|
return mapFeedbackSession(result)
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function createVoiceLesson(sessionId: string): Promise<VoiceLessonCreated> {
|
||||||
|
const result = await request<RawVoiceLessonCreated>(
|
||||||
|
`/api/v1/feedback-sessions/${encodeURIComponent(sessionId)}/lessons`,
|
||||||
|
'POST'
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
id: result.id,
|
||||||
|
sequence: result.sequence,
|
||||||
|
status: result.status,
|
||||||
|
startedAt: parseTimestamp(result.started_at)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function uploadVoiceSegment(
|
||||||
|
lessonId: string,
|
||||||
|
filePath: string,
|
||||||
|
sequence: number,
|
||||||
|
durationMs: number
|
||||||
|
): Promise<VoiceAudioSegment> {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
wx.uploadFile({
|
||||||
|
url: `${getApiBaseUrl()}/api/v1/feedback-lessons/${encodeURIComponent(lessonId)}/segments`,
|
||||||
|
filePath,
|
||||||
|
name: 'audio',
|
||||||
|
formData: {
|
||||||
|
sequence: String(sequence),
|
||||||
|
duration_ms: String(Math.max(1, Math.round(durationMs))),
|
||||||
|
audio_format: 'mp3'
|
||||||
|
},
|
||||||
|
timeout: 120000,
|
||||||
|
header: { 'X-User-Id': DEVELOPMENT_USER_ID },
|
||||||
|
success(response) {
|
||||||
|
let body: RawVoiceAudioSegment | ApiErrorBody
|
||||||
|
try {
|
||||||
|
body = JSON.parse(response.data) as RawVoiceAudioSegment | ApiErrorBody
|
||||||
|
} catch {
|
||||||
|
reject(new ApiRequestError('服务返回了无效的录音响应', response.statusCode))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (response.statusCode >= 200 && response.statusCode < 300) {
|
||||||
|
resolve(mapVoiceSegment(body as RawVoiceAudioSegment))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
reject(new ApiRequestError((body as ApiErrorBody).error || `服务返回 HTTP ${response.statusCode}`, response.statusCode))
|
||||||
|
},
|
||||||
|
fail(error) {
|
||||||
|
reject(new ApiRequestError(`录音上传失败:${error.errMsg}`))
|
||||||
|
}
|
||||||
|
})
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function finishVoiceLesson(lessonId: string): Promise<VoiceLessonFinished> {
|
||||||
|
const result = await request<RawVoiceLessonFinished>(
|
||||||
|
`/api/v1/feedback-lessons/${encodeURIComponent(lessonId)}/finish`,
|
||||||
|
'POST',
|
||||||
|
undefined,
|
||||||
|
120000
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
id: result.id,
|
||||||
|
status: result.status,
|
||||||
|
durationMs: result.duration_ms,
|
||||||
|
transcript: result.transcript,
|
||||||
|
canApplyDirectly: result.can_apply_directly
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function markVoiceLessonApplied(lessonId: string, content: string): Promise<void> {
|
||||||
|
await request<unknown>(
|
||||||
|
`/api/v1/feedback-lessons/${encodeURIComponent(lessonId)}/mark-applied`,
|
||||||
|
'POST',
|
||||||
|
{ content }
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function retryVoiceSegment(segmentId: string): Promise<VoiceAudioSegment> {
|
||||||
|
const result = await request<RawVoiceAudioSegment>(
|
||||||
|
`/api/v1/audio-segments/${encodeURIComponent(segmentId)}/retry`,
|
||||||
|
'POST',
|
||||||
|
undefined,
|
||||||
|
120000
|
||||||
|
)
|
||||||
|
return mapVoiceSegment(result)
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function generateFeedbackFromVoice(
|
||||||
|
sessionId: string,
|
||||||
|
existingContent: string
|
||||||
|
): Promise<GeneratedFeedback> {
|
||||||
|
const result = await request<RawGeneratedFeedback>(
|
||||||
|
`/api/v1/feedback-sessions/${encodeURIComponent(sessionId)}/generate`,
|
||||||
|
'POST',
|
||||||
|
{ existing_content: existingContent },
|
||||||
|
120000
|
||||||
|
)
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|||||||
@@ -44,9 +44,72 @@ export interface FeedbackDraft {
|
|||||||
profileId: string | null
|
profileId: string | null
|
||||||
feedbackDate: string
|
feedbackDate: string
|
||||||
content: string
|
content: string
|
||||||
|
feedbackSessionId?: string | null
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface HealthStatus {
|
export interface HealthStatus {
|
||||||
status: string
|
status: string
|
||||||
databaseConfigured: boolean
|
databaseConfigured: boolean
|
||||||
|
speechConfigured: boolean
|
||||||
|
speechAvailable: boolean
|
||||||
|
speechProvider: string | null
|
||||||
|
summaryConfigured: boolean
|
||||||
|
}
|
||||||
|
|
||||||
|
export type VoiceProcessingStatus = 'recording' | 'processing' | 'ready' | 'failed'
|
||||||
|
|
||||||
|
export interface VoiceAudioSegment {
|
||||||
|
id: string
|
||||||
|
sequence: number
|
||||||
|
durationMs: number
|
||||||
|
status: 'processing' | 'ready' | 'failed'
|
||||||
|
errorMessage: string | null
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface VoiceLesson {
|
||||||
|
id: string
|
||||||
|
sequence: number
|
||||||
|
status: VoiceProcessingStatus
|
||||||
|
durationMs: number
|
||||||
|
appliedDirectly: boolean
|
||||||
|
includedInGeneration: boolean
|
||||||
|
segments: VoiceAudioSegment[]
|
||||||
|
startedAt: number
|
||||||
|
endedAt: number | null
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface FeedbackSession {
|
||||||
|
id: string
|
||||||
|
profileId: string | null
|
||||||
|
feedbackDate: string
|
||||||
|
content: string
|
||||||
|
status: 'active' | 'finalized'
|
||||||
|
lessonCount: number
|
||||||
|
totalDurationMs: number
|
||||||
|
processingSegmentCount: number
|
||||||
|
failedSegmentCount: number
|
||||||
|
needsGeneration: boolean
|
||||||
|
lessons: VoiceLesson[]
|
||||||
|
createdAt: number
|
||||||
|
updatedAt: number
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface VoiceLessonCreated {
|
||||||
|
id: string
|
||||||
|
sequence: number
|
||||||
|
status: VoiceProcessingStatus
|
||||||
|
startedAt: number
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface VoiceLessonFinished {
|
||||||
|
id: string
|
||||||
|
status: VoiceProcessingStatus
|
||||||
|
durationMs: number
|
||||||
|
transcript: string
|
||||||
|
canApplyDirectly: boolean
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface GeneratedFeedback {
|
||||||
|
content: string
|
||||||
|
method: 'llm' | 'extractive'
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user