feat: 初始化 AutoVideo 视频处理分析工具
- MiMo 2.5 分镜分析 (OpenAI格式调用) - Qwen3-ASR-Flash FileTrans 转录+字级时间戳 - 一键 pipeline 脚本 - 杨梅/绿豆视频示例输出 - 完整文档和SOP
This commit is contained in:
runst
commit
f176e9a402
8
.gitignore
vendored
Normal file
8
.gitignore
vendored
Normal file
@ -0,0 +1,8 @@
|
|||||||
|
__pycache__/
|
||||||
|
*.pyc
|
||||||
|
*.pyo
|
||||||
|
.env
|
||||||
|
*.wav
|
||||||
|
*.mp4
|
||||||
|
frames/
|
||||||
|
*_analysis/
|
||||||
44
README.md
Normal file
44
README.md
Normal file
@ -0,0 +1,44 @@
|
|||||||
|
# AutoVideo - 视频处理分析工具
|
||||||
|
|
||||||
|
一键式视频分析：MiMo 2.5 分镜 + Qwen3-ASR-Flash 转录+字级时间戳。
|
||||||
|
|
||||||
|
## 使用
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python scripts/pipeline.py "视频路径.mp4" [输出目录]
|
||||||
|
```
|
||||||
|
|
||||||
|
## 流程
|
||||||
|
|
||||||
|
```
|
||||||
|
视频 → 压缩 → MiMo 2.5 分镜 + Qwen3-ASR FileTrans (转录+时间戳) → SRT
|
||||||
|
```
|
||||||
|
|
||||||
|
## 输出
|
||||||
|
|
||||||
|
```
|
||||||
|
xxx_analysis/
|
||||||
|
├── compressed.mp4 # 压缩视频
|
||||||
|
├── xxx_audio.wav # 音频
|
||||||
|
├── xxx_frames/ # 关键帧
|
||||||
|
├── scene.json # 分镜分析 (MiMo 2.5)
|
||||||
|
├── timestamps.json # 转录+字级时间戳 (Qwen3-ASR)
|
||||||
|
└── subtitles.srt # SRT 字幕
|
||||||
|
```
|
||||||
|
|
||||||
|
## 依赖
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install openai dashscope requests
|
||||||
|
```
|
||||||
|
- ffmpeg
|
||||||
|
- MiMo 2.5 代理 (127.0.0.1:15721)
|
||||||
|
- DashScope API Key
|
||||||
|
|
||||||
|
## 费用
|
||||||
|
|
||||||
|
| 服务 | 费用 |
|
||||||
|
|------|------|
|
||||||
|
| MiMo 2.5 分镜 | 通过本地代理 |
|
||||||
|
| Qwen3-ASR FileTrans | ≈ ¥0.01/小时 |
|
||||||
|
| **总计** | **≈ ¥0.01/100小时** |
|
||||||
79
docs/mimo_video_guide.md
Normal file
79
docs/mimo_video_guide.md
Normal file
@ -0,0 +1,79 @@
|
|||||||
|
# MiMo 2.5 视频理解调用指南
|
||||||
|
|
||||||
|
## ⚠️ 关键注意事项
|
||||||
|
|
||||||
|
**必须用 OpenAI 格式调用，Anthropic 格式会丢弃视频数据！**
|
||||||
|
|
||||||
|
## 调用方式
|
||||||
|
|
||||||
|
```python
|
||||||
|
import openai
|
||||||
|
import base64
|
||||||
|
|
||||||
|
client = openai.OpenAI(
|
||||||
|
base_url="http://127.0.0.1:15721/v1", # 本地代理
|
||||||
|
api_key="dummy-key"
|
||||||
|
)
|
||||||
|
|
||||||
|
with open("compressed.mp4", "rb") as f:
|
||||||
|
video_b64 = base64.b64encode(f.read()).decode()
|
||||||
|
|
||||||
|
response = client.chat.completions.create(
|
||||||
|
model="mimo-v2.5", # 模型名
|
||||||
|
messages=[{
|
||||||
|
"role": "user",
|
||||||
|
"content": [
|
||||||
|
{
|
||||||
|
"type": "video_url", # ← OpenAI 格式
|
||||||
|
"video_url": {
|
||||||
|
"url": f"data:video/mp4;base64,{video_b64}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "text",
|
||||||
|
"text": "请分析这个视频的内容"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}],
|
||||||
|
max_tokens=8192,
|
||||||
|
)
|
||||||
|
print(response.choices[0].message.content)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 错误示范 (Anthropic 格式 — 会丢失视频!)
|
||||||
|
|
||||||
|
```python
|
||||||
|
# ❌ 错误! 视频数据会被丢弃
|
||||||
|
client = anthropic.Anthropic(base_url="http://127.0.0.1:15721", api_key="dummy-key")
|
||||||
|
message = client.messages.create(
|
||||||
|
model="claude-opus-4-8",
|
||||||
|
messages=[{
|
||||||
|
"role": "user",
|
||||||
|
"content": [
|
||||||
|
{"type": "video", "source": {"type": "base64", ...}}, # ← 不支持!
|
||||||
|
{"type": "text", "text": "..."}
|
||||||
|
]
|
||||||
|
}]
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 参数说明
|
||||||
|
|
||||||
|
| 参数 | 推荐值 | 说明 |
|
||||||
|
|------|--------|------|
|
||||||
|
| `model` | `mimo-v2.5` | 通过代理映射到 MiMo 2.5 |
|
||||||
|
| `video_url.url` | `data:video/mp4;base64,...` | Base64 编码的视频 |
|
||||||
|
| `max_tokens` | 8192 | 输出 token 上限 |
|
||||||
|
| `temperature` | 0.1 | 低温度，更稳定的分析 |
|
||||||
|
|
||||||
|
## 视频压缩建议
|
||||||
|
|
||||||
|
| 原始大小 | 建议 | 压缩后 |
|
||||||
|
|---------|------|--------|
|
||||||
|
| > 10 MB | 必须压缩 | scale=480, crf=28 |
|
||||||
|
| 2-10 MB | 建议压缩 | scale=480, crf=30 |
|
||||||
|
| < 2 MB | 可直接发送 | — |
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ffmpeg -y -i input.mp4 -vf scale=-2:480 -c:v libx264 -preset fast -crf 28 -c:a aac -b:a 48k -ar 16000 -ac 1 output.mp4
|
||||||
|
```
|
||||||
102
docs/qwen3_asr_filetrans_guide.md
Normal file
102
docs/qwen3_asr_filetrans_guide.md
Normal file
@ -0,0 +1,102 @@
|
|||||||
|
# Qwen3-ASR-Flash FileTrans API 调用指南
|
||||||
|
|
||||||
|
## 前置条件
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install dashscope requests
|
||||||
|
export DASHSCOPE_API_KEY="your-key"
|
||||||
|
```
|
||||||
|
|
||||||
|
## 完整代码
|
||||||
|
|
||||||
|
```python
|
||||||
|
import requests
|
||||||
|
import dashscope
|
||||||
|
import json
|
||||||
|
import time
|
||||||
|
|
||||||
|
API_KEY = dashscope.api_key
|
||||||
|
AUDIO_PATH = "audio.wav"
|
||||||
|
|
||||||
|
# ===== Step 1: 上传音频获取临时 URL =====
|
||||||
|
upload_resp = dashscope.Files.upload(file_path=AUDIO_PATH, purpose="file-extract")
|
||||||
|
file_id = upload_resp.output["uploaded_files"][0]["file_id"]
|
||||||
|
file_info = dashscope.Files.get(file_id=file_id)
|
||||||
|
AUDIO_URL = file_info.output["url"]
|
||||||
|
|
||||||
|
# ===== Step 2: 提交转录任务 =====
|
||||||
|
headers = {
|
||||||
|
"Authorization": f"Bearer {API_KEY}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"X-DashScope-Async": "enable" # 必须!
|
||||||
|
}
|
||||||
|
payload = {
|
||||||
|
"model": "qwen3-asr-flash-filetrans",
|
||||||
|
"input": {"file_url": AUDIO_URL}, # file_url 单数!
|
||||||
|
"parameters": {
|
||||||
|
"channel_id": [0],
|
||||||
|
"enable_words": True, # 字级时间戳!
|
||||||
|
"enable_itn": False,
|
||||||
|
"language": "zh"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
resp = requests.post(
|
||||||
|
"https://dashscope.aliyuncs.com/api/v1/services/audio/asr/transcription",
|
||||||
|
headers=headers, json=payload
|
||||||
|
)
|
||||||
|
task_id = resp.json()["output"]["task_id"]
|
||||||
|
|
||||||
|
# ===== Step 3: 轮询结果 =====
|
||||||
|
time.sleep(2)
|
||||||
|
result = requests.get(
|
||||||
|
f"https://dashscope.aliyuncs.com/api/v1/tasks/{task_id}",
|
||||||
|
headers={"Authorization": f"Bearer {API_KEY}"}
|
||||||
|
).json()
|
||||||
|
|
||||||
|
# ===== Step 4: 获取转录 JSON =====
|
||||||
|
tr_url = result["output"]["result"]["transcription_url"]
|
||||||
|
trans_data = requests.get(tr_url).json()
|
||||||
|
|
||||||
|
# ===== Step 5: 输出 =====
|
||||||
|
for sent in trans_data["transcripts"][0]["sentences"]:
|
||||||
|
print(f"[{sent['begin_time']:>6}ms - {sent['end_time']:>6}ms] {sent['text']}")
|
||||||
|
for w in sent.get("words", []):
|
||||||
|
print(f" [{w['begin_time']:>6}ms - {w['end_time']:>6}ms] {w['text']}{w.get('punctuation','')}")
|
||||||
|
```
|
||||||
|
|
||||||
|
## 关键参数
|
||||||
|
|
||||||
|
| 参数 | 值 | 说明 |
|
||||||
|
|------|-----|------|
|
||||||
|
| `model` | `qwen3-asr-flash-filetrans` | 离线文件转录模型 |
|
||||||
|
| `input.file_url` | string | **单数**，音频文件的公网 URL |
|
||||||
|
| `parameters.enable_words` | `True` | **开启字级时间戳** |
|
||||||
|
| `parameters.language` | `"zh"` | 指定中文，提升准确率 |
|
||||||
|
| `X-DashScope-Async` | `"enable"` | **必须**，异步任务模式 |
|
||||||
|
|
||||||
|
## 返回结构
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"transcripts": [{
|
||||||
|
"text": "完整转录文本",
|
||||||
|
"content_duration": 29907,
|
||||||
|
"sentences": [{
|
||||||
|
"begin_time": 160,
|
||||||
|
"end_time": 1600,
|
||||||
|
"text": "句子文本",
|
||||||
|
"words": [
|
||||||
|
{"begin_time": 160, "end_time": 240, "text": "字", "punctuation": ""}
|
||||||
|
]
|
||||||
|
}]
|
||||||
|
}]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 常见错误
|
||||||
|
|
||||||
|
| 错误 | 原因 | 解决 |
|
||||||
|
|------|------|------|
|
||||||
|
| `MalformedURL` | URL 不可访问 | 确保用 `Files.upload` 获取的 OSS URL |
|
||||||
|
| `InvalidParameter` | 用了 `file_urls` | 改为 `file_url` (单数) |
|
||||||
|
| `400: max_tokens too large` | 用了 chat API 参数 | FileTrans 不需要 max_tokens |
|
||||||
94
docs/workflow_sop.md
Normal file
94
docs/workflow_sop.md
Normal file
@ -0,0 +1,94 @@
|
|||||||
|
# 视频处理分析流程 (SOP)
|
||||||
|
|
||||||
|
## 流程总览
|
||||||
|
|
||||||
|
```
|
||||||
|
视频文件
|
||||||
|
│
|
||||||
|
▼ Phase 1: 信息采集 ────────── ffprobe
|
||||||
|
│
|
||||||
|
▼ Phase 2: 压缩预处理 ──────── ffmpeg (压缩+音频+关键帧)
|
||||||
|
│
|
||||||
|
├─→ Phase 3: 分镜分析 ──────── MiMo 2.5 (OpenAI格式)
|
||||||
|
│
|
||||||
|
├─→ Phase 4a: 转录文本 ─────── Qwen3-ASR-Flash chat API
|
||||||
|
│
|
||||||
|
└─→ Phase 4b: 字级时间戳 ───── Qwen3-ASR-Flash FileTrans API
|
||||||
|
│
|
||||||
|
▼ Phase 5: 输出 ────────────── 报告 + JSON + SRT
|
||||||
|
```
|
||||||
|
|
||||||
|
## Phase 1: 信息采集
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ffprobe -v quiet -print_format json -show_format -show_streams <视频路径>
|
||||||
|
```
|
||||||
|
|
||||||
|
## Phase 2: 压缩预处理
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 压缩
|
||||||
|
ffmpeg -y -i <输入> -vf "scale=-2:480" -c:v libx264 -preset fast -crf 28 \
|
||||||
|
-c:a aac -b:a 48k -ar 16000 -ac 1 -movflags +faststart <输出.mp4>
|
||||||
|
|
||||||
|
# 提取音频
|
||||||
|
ffmpeg -y -i <输入> -vn -acodec pcm_s16le -ar 16000 -ac 1 <输出.wav>
|
||||||
|
|
||||||
|
# 提取关键帧
|
||||||
|
ffmpeg -y -i <输入> -vf "fps=2,scale=480:-2" -q:v 5 <输出目录>/frame_%04d.jpg
|
||||||
|
```
|
||||||
|
|
||||||
|
## Phase 3: 分镜分析 (MiMo 2.5)
|
||||||
|
|
||||||
|
```python
|
||||||
|
# 必须用 OpenAI 格式!
|
||||||
|
client = openai.OpenAI(base_url="http://127.0.0.1:15721/v1", api_key="dummy-key")
|
||||||
|
response = client.chat.completions.create(
|
||||||
|
model="mimo-v2.5",
|
||||||
|
messages=[{"role": "user", "content": [
|
||||||
|
{"type": "video_url", "video_url": {"url": f"data:video/mp4;base64,{b64}"}},
|
||||||
|
{"type": "text", "text": "请逐镜头分析这个视频..."}
|
||||||
|
]}],
|
||||||
|
max_tokens=8192
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Phase 4a: 转录文本 (Qwen3 chat API)
|
||||||
|
|
||||||
|
```python
|
||||||
|
response = client.chat.completions.create(
|
||||||
|
model="qwen3-asr-flash",
|
||||||
|
messages=[{"role": "user", "content": [
|
||||||
|
{"type": "input_audio", "input_audio": {"data": f"data:audio/wav;base64,{b64}"}}
|
||||||
|
]}],
|
||||||
|
max_tokens=4096
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Phase 4b: 字级时间戳 (Qwen3 FileTrans API)
|
||||||
|
|
||||||
|
```python
|
||||||
|
# 1. 上传
|
||||||
|
upload = dashscope.Files.upload(file_path="audio.wav", purpose="file-extract")
|
||||||
|
url = dashscope.Files.get(file_id=upload.output["uploaded_files"][0]["file_id"]).output["url"]
|
||||||
|
|
||||||
|
# 2. 提交任务
|
||||||
|
resp = requests.post("https://dashscope.aliyuncs.com/api/v1/services/audio/asr/transcription",
|
||||||
|
headers={"Authorization": f"Bearer {API_KEY}", "X-DashScope-Async": "enable", "Content-Type": "application/json"},
|
||||||
|
json={"model": "qwen3-asr-flash-filetrans", "input": {"file_url": url},
|
||||||
|
"parameters": {"enable_words": True, "language": "zh"}})
|
||||||
|
|
||||||
|
# 3. 获取结果
|
||||||
|
result = requests.get(f"https://dashscope.aliyuncs.com/api/v1/tasks/{task_id}", ...)
|
||||||
|
trans = requests.get(result["output"]["result"]["transcription_url"]).json()
|
||||||
|
```
|
||||||
|
|
||||||
|
## 模型选择
|
||||||
|
|
||||||
|
| 需求 | 模型 |
|
||||||
|
|------|------|
|
||||||
|
| 分镜场景 | MiMo 2.5 |
|
||||||
|
| 转录文本 | Qwen3-ASR-Flash (chat) |
|
||||||
|
| 字级时间戳 | Qwen3-ASR-Flash (FileTrans) |
|
||||||
|
| 方言/说话人 | Fun-ASR |
|
||||||
|
| 离线私有化 | Paraformer-zh |
|
||||||
66
examples/6y15_lvDou/mimo_scene.json
Normal file
66
examples/6y15_lvDou/mimo_scene.json
Normal file
@ -0,0 +1,66 @@
|
|||||||
|
{
|
||||||
|
"video_summary": "这是一条绿豆带货短视频，以超市真实选购场景切入，结合产品细节特写与夏季美食制作展示，突出绿豆当季新鲜、品质优良、性价比高的核心优势，最终呼应夏季场景，呼吁观众趁优惠囤货。",
|
||||||
|
"scenes": [
|
||||||
|
{
|
||||||
|
"scene_id": 1,
|
||||||
|
"start_time": "00:00.000",
|
||||||
|
"end_time": "00:09.000",
|
||||||
|
"duration_sec": 9,
|
||||||
|
"scene_type": "真人出镜选购场景",
|
||||||
|
"visual_description": "在超市杂粮区，穿着浅棕色印字T恤、白色长裤的短发女生，正用金属铲子将散装绿豆装入透明塑料袋，一边和画外的人互动对话，吐槽之前超市散装绿豆价格昂贵，介绍当下绿豆有优惠活动，引导观众查看链接。",
|
||||||
|
"camera": "中景拍摄，镜头轻微跟随女生的装袋动作移动",
|
||||||
|
"transition": "开场，无前置转场"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 2,
|
||||||
|
"start_time": "00:09.000",
|
||||||
|
"end_time": "00:15.000",
|
||||||
|
"duration_sec": 6,
|
||||||
|
"scene_type": "产品细节特写展示",
|
||||||
|
"visual_description": "先拍摄女生双手捧起装满绿豆的袋子，随后镜头切至手部特写，女生掌心托着绿豆，展示其颗颗饱满、色泽碧绿鲜亮的特质，背景是大量散装绿豆。",
|
||||||
|
"camera": "从近景切换至特写，聚焦绿豆的细节品相",
|
||||||
|
"transition": "硬切"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 3,
|
||||||
|
"start_time": "00:15.000",
|
||||||
|
"end_time": "00:18.000",
|
||||||
|
"duration_sec": 3,
|
||||||
|
"scene_type": "产品批量展示",
|
||||||
|
"visual_description": "俯拍视角展示大量散装绿豆的整体堆面，呈现绿豆品质均匀的状态，配合解说强调这类绿豆出沙快。",
|
||||||
|
"camera": "固定俯拍镜头，聚焦整体绿豆堆",
|
||||||
|
"transition": "硬切"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 4,
|
||||||
|
"start_time": "00:18.000",
|
||||||
|
"end_time": "00:22.000",
|
||||||
|
"duration_sec": 4,
|
||||||
|
"scene_type": "美食成品展示",
|
||||||
|
"visual_description": "先展示盛放在蓝边瓷碗中的绿豆汤，用勺子舀起呈现浓稠质地；随后切换至锅内场景，红色勺子舀起满满的绿豆，直观体现绿豆出沙好、口感浓稠的特点。",
|
||||||
|
"camera": "近景拍摄，跟随勺子的舀取动作动态呈现美食",
|
||||||
|
"transition": "硬切"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 5,
|
||||||
|
"start_time": "00:22.000",
|
||||||
|
"end_time": "00:25.000",
|
||||||
|
"duration_sec": 3,
|
||||||
|
"scene_type": "美食制作展示",
|
||||||
|
"visual_description": "将打好的浅绿色浓稠绿豆冰沙，从料理机倒入装有冰块的透明玻璃杯中，呈现冰沙顺滑的质感，搭配解说突出其好吃解腻的夏季食用属性。",
|
||||||
|
"camera": "近景固定镜头，拍摄冰沙倾倒的完整过程",
|
||||||
|
"transition": "硬切"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 6,
|
||||||
|
"start_time": "00:25.000",
|
||||||
|
"end_time": "00:30.000",
|
||||||
|
"duration_sec": 5,
|
||||||
|
"scene_type": "真人出镜收尾场景",
|
||||||
|
"visual_description": "回到超市杂粮区，女生继续往塑料袋里装绿豆，笑着对镜头说话，再次强调当下价格合适，呼吁观众像自己一样多囤一些绿豆。",
|
||||||
|
"camera": "中景固定镜头，拍摄女生的动作与表达",
|
||||||
|
"transition": "硬切"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"timeline": "视频以超市选购绿豆的日常场景开篇，通过对话引出绿豆优惠活动，对比过往高价散装绿豆凸显性价比；紧接着通过产品特写展示绿豆的当季新鲜与饱满品相；随后切换至美食制作场景，展示绿豆煮出的浓稠汤品与清爽冰沙，印证其出沙快、适配夏季需求的特点；最后回归超市场景，再次强调性价比优势，呼吁观众趁优惠囤货。"
|
||||||
|
}
|
||||||
1
examples/6y15_lvDou/qwen3_asr.txt
Normal file
1
examples/6y15_lvDou/qwen3_asr.txt
Normal file
@ -0,0 +1 @@
|
|||||||
|
你疯了，买这么多绿豆。便宜呀，这绿豆现在也来大活动了啊。之前我在超市买那些散装的都死贵死贵了，你去看看我这个链接才多少钱呢？它都是当季新鲜的绿豆，你看个个颗大饱满的，碧绿碧绿的呢。像这种品质好的绿豆啊，就特别好出沙。咱夏天越来越热了，你就在家里面熬个绿豆汤啊，或者个做个绿豆冰沙呀，好吃还解腻呢。你就趁着现在价格合适，也像我似的啊，多囤点回去多合适呀。
|
||||||
16
examples/6y15_lvDou/qwen3_filetrans_raw.json
Normal file
16
examples/6y15_lvDou/qwen3_filetrans_raw.json
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
{
|
||||||
|
"request_id": "224b5944-0ae7-94cd-896f-0dde682d4a70",
|
||||||
|
"output": {
|
||||||
|
"task_id": "d3a0ebac-fc97-48e5-87e8-47eee81b75fe",
|
||||||
|
"task_status": "SUCCEEDED",
|
||||||
|
"submit_time": "2026-07-27 14:00:51.755",
|
||||||
|
"scheduled_time": "2026-07-27 14:00:51.955",
|
||||||
|
"end_time": "2026-07-27 14:00:53.928",
|
||||||
|
"result": {
|
||||||
|
"transcription_url": "http://dashscope-result-bj.oss-cn-beijing.aliyuncs.com/prod/qwen3-asr-flash-filetrans/20260727/14%3A00/d3a0ebac-fc97-48e5-87e8-47eee81b75fe.json?Expires=1785218453&OSSAccessKeyId=LTAI5tGzqbGcEmE58b221XQy&Signature=lNdt4N6fCzFQWql6vsbWaCq5v2Y%3D&response-content-disposition=attachment%3Bfilename%3Dd3a0ebac-fc97-48e5-87e8-47eee81b75fe.json"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"usage": {
|
||||||
|
"seconds": 28
|
||||||
|
}
|
||||||
|
}
|
||||||
1033
examples/6y15_lvDou/qwen3_filetrans_timestamps.json
Normal file
1033
examples/6y15_lvDou/qwen3_filetrans_timestamps.json
Normal file
File diff suppressed because it is too large
Load Diff
1802
examples/6y15_lvDou/whisper.json
Normal file
1802
examples/6y15_lvDou/whisper.json
Normal file
File diff suppressed because it is too large
Load Diff
1
examples/meimei_yangmei/mimo_asr_raw.txt
Normal file
1
examples/meimei_yangmei/mimo_asr_raw.txt
Normal file
@ -0,0 +1 @@
|
|||||||
|
我最近迷恋上了这一款云南高山杨梅，真的是又鲜又甜，巨好吃。而且这样子的两大板到手以后才七十九块九。你看它每个个头啊都非常的饱满，而且它的汁水非常非常的多。当天现摘的杨梅，口感鲜甜，果肉细嫩，真的特别特别的新鲜，越吃越有滋味。到手这样满满的一箱啊，吃起来真的好过瘾。哇，你看这个汁水，真的超级的过瘾，超级的满足，强烈安利给爱吃杨梅的每一个人。我家里的人都特别的爱吃，一次就能炫这一整盒。爱吃杨梅的家人们一定要来一箱尝尝看。
|
||||||
1158
examples/meimei_yangmei/mimo_transcription_full.json
Normal file
1158
examples/meimei_yangmei/mimo_transcription_full.json
Normal file
File diff suppressed because it is too large
Load Diff
135
examples/meimei_yangmei/mimo_v25_output.json
Normal file
135
examples/meimei_yangmei/mimo_v25_output.json
Normal file
@ -0,0 +1,135 @@
|
|||||||
|
{
|
||||||
|
"video_analysis_report": {
|
||||||
|
"video_info": {
|
||||||
|
"title": "云南高山杨梅推荐",
|
||||||
|
"content_type": "产品推荐/带货短视频",
|
||||||
|
"total_duration_sec": 35,
|
||||||
|
"summary": "本视频为一段时长35秒的云南高山杨梅带货短视频。视频核心叙事结构清晰，遵循了‘吸引眼球（特写冲击）- 产品展示（整体与细节）- 价值传递（价格与品质）- 体验背书（试吃）- 再次强调（卖点与呼吁）’的营销逻辑。全片通过快速、密集的镜头切换，聚焦于杨梅的‘新鲜’、‘多汁’、‘饱满’三大卖点，利用特写镜头下的汁水横流和主持人的生动描述与试吃，有效激发观众的购买欲望。"
|
||||||
|
},
|
||||||
|
"scene_analysis": [
|
||||||
|
{
|
||||||
|
"scene_id": 1,
|
||||||
|
"start_time": "00:00.000",
|
||||||
|
"end_time": "00:01.000",
|
||||||
|
"duration_sec": 1,
|
||||||
|
"scene_type": "产品特写",
|
||||||
|
"visual_description": "画面为极近景特写。一只成年男性的手（根据指甲和关节判断）正用力捏爆一颗深紫红色的杨梅，大量的深红色汁水瞬间从果肉中迸发并流淌下来，覆盖在手指和手掌上。背景是模糊的杨梅托盘。",
|
||||||
|
"camera": "固定镜头，特写。",
|
||||||
|
"transition": "直接切入（作为视频开场吸引注意力的‘黄金三秒’镜头）。"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 2,
|
||||||
|
"start_time": "00:01.000",
|
||||||
|
"end_time": "00:02.000",
|
||||||
|
"duration_sec": 1,
|
||||||
|
"scene_type": "主持人场景",
|
||||||
|
"visual_description": "画面切换至室内场景。一位身穿白色无袖上衣、佩戴领夹麦克风、扎着发带的女性主持人，正微笑着从一个白色泡沫箱中取出一整板用黑色塑料托盘装着的杨梅。背景是明亮的窗户和绿植。",
|
||||||
|
"camera": "中景，固定。",
|
||||||
|
"transition": "硬切。"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 3,
|
||||||
|
"start_time": "00:02.000",
|
||||||
|
"end_time": "00:04.000",
|
||||||
|
"duration_sec": 2,
|
||||||
|
"scene_type": "产品特写",
|
||||||
|
"visual_description": "画面切换为女性手部特写，她用双手轻轻托着两颗并排的杨梅，杨梅呈深紫黑色，表面颗粒饱满，色泽均匀。字幕显示‘真的是又鲜又甜，巨好吃’。",
|
||||||
|
"camera": "特写，轻微跟随手部移动。",
|
||||||
|
"transition": "硬切。"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 4,
|
||||||
|
"start_time": "00:04.000",
|
||||||
|
"end_time": "00:07.000",
|
||||||
|
"duration_sec": 3,
|
||||||
|
"scene_type": "主持人场景",
|
||||||
|
"visual_description": "切回主持人中景。她左右手各举起一大板杨梅（每板约50颗），面向镜头展示，同时讲解价格。字幕同步显示‘这样子的两大板 到手以后才79.9’。",
|
||||||
|
"camera": "中景，固定。",
|
||||||
|
"transition": "硬切。"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 5,
|
||||||
|
"start_time": "00:07.000",
|
||||||
|
"end_time": "00:08.000",
|
||||||
|
"duration_sec": 1,
|
||||||
|
"scene_type": "产品特写",
|
||||||
|
"visual_description": "镜头推近，特写展示一整板杨梅。杨梅颗颗饱满，紧密排列在黑色托盘的凹槽内，表面干燥新鲜。字幕显示‘你看他每个个头都非常的饱满’。",
|
||||||
|
"camera": "俯拍特写，有轻微的推镜头效果。",
|
||||||
|
"transition": "硬切。"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 6,
|
||||||
|
"start_time": "00:08.000",
|
||||||
|
"end_time": "00:10.000",
|
||||||
|
"duration_sec": 2,
|
||||||
|
"scene_type": "产品功能展示",
|
||||||
|
"visual_description": "画面切换为手部特写（与场景1不同，手部更纤细），再次展示用手挤压一颗杨梅，大量红色汁水涌出，滴落在下方模糊的杨梅板上。字幕显示‘汁水也非常的多’。",
|
||||||
|
"camera": "特写，固定。",
|
||||||
|
"transition": "硬切。"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 7,
|
||||||
|
"start_time": "00:10.000",
|
||||||
|
"end_time": "00:15.000",
|
||||||
|
"duration_sec": 5,
|
||||||
|
"scene_type": "主持人试吃",
|
||||||
|
"visual_description": "主持人中景。她拿起一颗杨梅放入口中品尝，做出享受的表情，并描述口感。随后镜头快速切到她手中被咬了一口的杨梅特写，展示内部鲜红细嫩的果肉。字幕显示‘当天现摘的杨梅’、‘口感鲜甜，果肉细嫩’。",
|
||||||
|
"camera": "中景 -> 快速切换至特写 -> 切回主持人。",
|
||||||
|
"transition": "硬切。"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 8,
|
||||||
|
"start_time": "00:15.000",
|
||||||
|
"end_time": "00:18.000",
|
||||||
|
"duration_sec": 3,
|
||||||
|
"scene_type": "主持人场景",
|
||||||
|
"visual_description": "主持人手持一板杨梅，另一只手拿起一颗，边展示边讲解其新鲜度和风味。她表情愉悦，语气肯定。字幕显示‘真的特别特别的新鲜’、‘越吃越有滋味’。",
|
||||||
|
"camera": "中景，固定。",
|
||||||
|
"transition": "硬切。"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 9,
|
||||||
|
"start_time": "00:18.000",
|
||||||
|
"end_time": "00:22.000",
|
||||||
|
"duration_sec": 4,
|
||||||
|
"scene_type": "主持人场景",
|
||||||
|
"visual_description": "主持人将手中一整板杨梅倾斜展示，然后拿起一颗再次送入口中，咀嚼时发出清晰的‘咕叽’声（可能是音效），表情非常享受。字幕显示‘到手慢慢的满满一箱’、‘吃起来真的好过瘾’。",
|
||||||
|
"camera": "中景，固定。",
|
||||||
|
"transition": "硬切。"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 10,
|
||||||
|
"start_time": "00:22.000",
|
||||||
|
"end_time": "00:26.000",
|
||||||
|
"duration_sec": 4,
|
||||||
|
"scene_type": "产品特写",
|
||||||
|
"visual_description": "画面切到手部特写，展示一颗被咬开的杨梅。内部结构清晰可见，果汁丰盈，在光线下晶莹剔透。主持人画外音惊叹汁水之多。字幕显示‘哇~你看这个汁水’、‘真的超级的过瘾 超级的满足’。",
|
||||||
|
"camera": "微距特写，固定，焦点清晰。",
|
||||||
|
"transition": "硬切。"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 11,
|
||||||
|
"start_time": "00:26.000",
|
||||||
|
"end_time": "00:29.000",
|
||||||
|
"duration_sec": 3,
|
||||||
|
"scene_type": "主持人场景",
|
||||||
|
"visual_description": "主持人手持一颗杨梅，向镜头做出推荐的手势，语气诚恳地进行‘安利’。字幕显示‘强烈安利给每一个喜欢吃杨梅的人’。",
|
||||||
|
"camera": "中景，固定。",
|
||||||
|
"transition": "硬切。"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"scene_id": 12,
|
||||||
|
"start_time": "00:29.000",
|
||||||
|
"end_time": "00:34.000",
|
||||||
|
"duration_sec": 5,
|
||||||
|
"scene_type": "主持人场景",
|
||||||
|
"visual_description": "主持人抱着一整箱（泡沫箱内装满一板杨梅）面向镜头，总结家庭喜爱度和产品分量，最后发出购买呼吁。字幕依次显示‘我家里人都特别爱吃’、‘一次能炫一大盆’、‘爱吃杨梅的家人们一定要来尝尝看’。",
|
||||||
|
"camera": "中景，固定。",
|
||||||
|
"transition": "硬切。"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"timeline_narrative": {
|
||||||
|
"summary": "视频以极具冲击力的‘捏爆杨梅’特写开场，瞬间建立‘多汁’的核心卖点。随即引入主持人，通过她从开箱、展示两大板产品、价格播报，到近距离展示果肉饱满度、再次用特写强调汁水丰盈，层层递进地呈现产品优势。关键的试吃环节通过主持人的表情、语言和被咬开杨梅的特写，从味觉和视觉双重维度进行体验背书。结尾部分回归整体推荐，以‘家人爱吃’增加可信度，并再次呼吁目标人群（爱吃杨梅的人）行动。整个视频节奏明快，信息密度高，在短短35秒内完成了从吸引、展示、说服到转化的完整营销链条。"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
1170
examples/meimei_yangmei/mimo_vlm_transcription.json
Normal file
1170
examples/meimei_yangmei/mimo_vlm_transcription.json
Normal file
File diff suppressed because it is too large
Load Diff
159
examples/meimei_yangmei/mimo_vs_whisper_comparison.md
Normal file
159
examples/meimei_yangmei/mimo_vs_whisper_comparison.md
Normal file
@ -0,0 +1,159 @@
|
|||||||
|
# MiMo 2.5 vs Whisper 转录对比分析
|
||||||
|
## 视频: test_meimei.mp4 (云南高山杨梅带货)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 一、段落级逐句对比
|
||||||
|
|
||||||
|
| # | 时间 | MiMo 2.5 | Whisper large-v3 | 差异 | 判定 |
|
||||||
|
|---|------|----------|------------------|------|------|
|
||||||
|
| 1 | 00:00-00:02 | 我最近迷上了这个云南高山杨梅 | 我最近**明天上了这**一款云南高山杨梅 | MiMo ✅ Whisper ❌ | **MiMo 胜** |
|
||||||
|
| 2 | 00:02-00:04 | 真的是又鲜又甜 **巨**好吃 | 真的是又鲜又甜 **最**好吃 | 语义均可 | **平局** (巨/最 都常见) |
|
||||||
|
| 3 | 00:04-00:06 | 而且这样**子**的两大板 | 而且这样**折**两大板 | MiMo ✅ Whisper ❌ | **MiMo 胜** |
|
||||||
|
| 4 | 00:06-00:07 | 到手以后才79.9 | 到手以后才79.9**元** | MiMo ❌ Whisper ✅ | **Whisper 胜** (Whisper多出"元") |
|
||||||
|
| 5 | 00:07-00:09 | 你看**他**每个个头都非常的饱满 | 你看**它**每个个头都非常的饱满 | 他/它 通用 | **平局** |
|
||||||
|
| 6 | 00:09-00:11 | **汁水也**非常的多 | **而且它的汁水**非常非常的多 | MiMo更简洁 | **Whisper 更完整** |
|
||||||
|
| 7 | 00:11-00:13 | 当天现摘的杨梅 | 当天现摘的杨梅 | ✅ 完全一致 | **平局** |
|
||||||
|
| 8 | 00:13-00:15 | 口感鲜甜 果肉细嫩 | 口感鲜甜 果肉细嫩 | ✅ 完全一致 | **平局** |
|
||||||
|
| 9 | 00:15-00:16 | 真的特别特别的新鲜 | 真的特别特别的新鲜 | ✅ 完全一致 | **平局** |
|
||||||
|
| 10 | 00:16-00:17 | 越吃越有滋味 | 越吃越有滋味 | ✅ 完全一致 | **平局** |
|
||||||
|
| 11 | 00:18-00:20 | 到手**慢慢的**满满一箱**啊** | 到手**这样**满满的一箱 | MiMo 多字 | **Whisper 更准确** |
|
||||||
|
| 12 | 00:20-00:21 | 吃起来真的**过瘾** | 吃起来真的**好**过瘾 | Whisper多"好" | **Whisper 更准确** |
|
||||||
|
| 13 | 00:22-00:24 | 哇~ 你看这个汁水 | 哇 你看这个汁水 | ~差异 | **平局** |
|
||||||
|
| 14 | 00:24-00:26 | 真的超级的过瘾 超级的满足 | 真的超级的过瘾 超级的满足 | ✅ 完全一致 | **平局** |
|
||||||
|
| 15 | 00:27-00:29 | 强烈**安利**给每一个喜欢吃杨梅的人 | 强烈**安逸**给爱吃杨梅的每一个人 | 关键差异 | **MiMo ✅ "安利"正确** |
|
||||||
|
| 16 | 00:29-00:31 | 我家里人都特别爱吃 | 我家里的人都特别的爱吃 | "家里的" vs "家里" | **平局** (口语等价) |
|
||||||
|
| 17 | 00:31-00:33 | 一次**能炫一大盒** | 一次**就能掀这一整盒** | 差异较大 | 需结合视频字幕 |
|
||||||
|
| 18 | 00:33-00:36 | 爱吃杨梅的家人们一定要来尝尝看 | 爱吃杨梅的家人们 + 一定要来一箱尝尝看 | Whisper多"一箱" | **Whisper 更完整** |
|
||||||
|
|
||||||
|
### 段落级统计
|
||||||
|
- **完全一致**: 5/18 句 (28%)
|
||||||
|
- **语义等价**: 5/18 句 (28%)
|
||||||
|
- **MiMo 更准确**: 3/18 句 (17%) — #1 迷上了, #3 子的, #15 安利
|
||||||
|
- **Whisper 更准确**: 4/18 句 (22%) — #4 元, #6 完整, #12 好, #18 一箱
|
||||||
|
- **均有偏差**: 1/18 句 (5%) — #17
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 二、关键转录差异详解
|
||||||
|
|
||||||
|
### 差异 1: 句1 — "迷上了" vs "明天上了"
|
||||||
|
| | MiMo 2.5 | Whisper |
|
||||||
|
|--|----------|---------|
|
||||||
|
| 文本 | 我最近**迷上了**这个云南高山杨梅 | 我最近**明天上了这**一款云南高山杨梅 |
|
||||||
|
| 时间戳 | 0.00 → 2.38 | 0.00 → 2.40 |
|
||||||
|
| 判定 | ✅ **正确** | ❌ "明天上了"不通顺 |
|
||||||
|
| 原因 | MiMo 视觉+音频联合理解 | Whisper 纯音频，受口音影响 |
|
||||||
|
|
||||||
|
### 差异 2: 句15 — "安利" vs "安逸"
|
||||||
|
| | MiMo 2.5 | Whisper |
|
||||||
|
|--|----------|---------|
|
||||||
|
| 文本 | 强烈**安利**给每一个喜欢吃杨梅的人 | 强烈**安逸**给爱吃杨梅的每一个人 |
|
||||||
|
| Whisper置信度 | — | 安: 0.81, 逸: 0.80 (低!) |
|
||||||
|
| 判定 | ✅ **正确** — "安利"是网络用语"推荐" | ❌ "安逸"不通顺 |
|
||||||
|
| 原因 | MiMo 语义理解能力强 | Whisper 声学相似词混淆 |
|
||||||
|
|
||||||
|
### 差异 3: 句6 — 信息量差异
|
||||||
|
| | MiMo 2.5 | Whisper |
|
||||||
|
|--|----------|---------|
|
||||||
|
| 文本 | 汁水也非常的多 | 而且它的汁水非常非常的多 |
|
||||||
|
| 判定 | ⚠️ 漏了"而且它的" | ✅ 更完整 |
|
||||||
|
| 原因 | MiMo 可能将前句合并 | Whisper 更忠实于音频 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 三、字级时间戳精度对比 (选取典型区间)
|
||||||
|
|
||||||
|
### 区间 1: "我最近迷上了这个云南高山杨梅" (0-2.4s)
|
||||||
|
|
||||||
|
| 字 | MiMo start | Whisper start | 差值(ms) | MiMo dur | Whisper dur |
|
||||||
|
|----|-----------|--------------|----------|----------|-------------|
|
||||||
|
| 我 | 0.000 | 0.000 | 0 | 160ms | 80ms |
|
||||||
|
| 最 | 0.160 | 0.080 | +80 | 200ms | 140ms |
|
||||||
|
| 近 | 0.360 | — | — | 160ms | — |
|
||||||
|
| 迷 | 0.520 | — (明) | — | 200ms | — |
|
||||||
|
| 上 | 0.720 | 0.600 | +120 | 200ms | 180ms |
|
||||||
|
| 了 | 0.920 | 0.780 | +140 | 160ms | 80ms |
|
||||||
|
| 这 | 1.080 | 0.860 | +220 | 120ms | 120ms |
|
||||||
|
| 个 | 1.200 | — | — | 160ms | — |
|
||||||
|
| 云 | 1.360 | 1.300 | +60 | 200ms | 180ms |
|
||||||
|
| 南 | 1.560 | 1.480 | +80 | 160ms | 180ms |
|
||||||
|
| 高 | 1.720 | 1.660 | +60 | 160ms | 140ms |
|
||||||
|
| 山 | 1.880 | 1.800 | +80 | 160ms | 180ms |
|
||||||
|
| 杨 | 2.040 | 1.980 | +60 | 160ms | 220ms |
|
||||||
|
| 梅 | 2.200 | 2.200 | 0 | 180ms | 200ms |
|
||||||
|
|
||||||
|
**规律**: MiMo 的字级时间戳普遍比 Whisper **晚 60-220ms**，但差异在可接受范围内。
|
||||||
|
|
||||||
|
### 区间 2: "真的超级的过瘾 超级的满足" (23.7-25.8s)
|
||||||
|
|
||||||
|
| 字 | MiMo start | Whisper start | 差值(ms) |
|
||||||
|
|----|-----------|--------------|----------|
|
||||||
|
| 真 | 24.200 | 23.680 | +520 |
|
||||||
|
| 的 | 24.360 | 24.000 | +360 |
|
||||||
|
| 超 | 24.400 | 24.000 | +400 |
|
||||||
|
| 级 | 24.560 | 24.240 | +320 |
|
||||||
|
| 的 | 24.720 | 24.560 | +160 |
|
||||||
|
| 过 | 24.760 | 24.700 | +60 |
|
||||||
|
| 瘾 | 24.960 | 24.820 | +140 |
|
||||||
|
| 超 | 25.200 | 25.040 | +160 |
|
||||||
|
| 级 | 25.400 | 25.160 | +240 |
|
||||||
|
| 的 | 25.600 | 25.320 | +280 |
|
||||||
|
| 满 | 25.640 | 25.460 | +180 |
|
||||||
|
| 足 | 25.840 | 25.600 | +240 |
|
||||||
|
|
||||||
|
**规律**: 此区间 MiMo 整体偏晚 60-520ms，Whisper 的时间戳更贴合实际发音。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 四、置信度/概率对比
|
||||||
|
|
||||||
|
### MiMo 2.5
|
||||||
|
- 所有字的 probability 固定为 **0.95**
|
||||||
|
- ❌ **无法区分识别难度** — 无论多清晰或多模糊的字，置信度都一样
|
||||||
|
|
||||||
|
### Whisper large-v3
|
||||||
|
- 每个字有独立的真实置信度 (0.42 ~ 1.00)
|
||||||
|
- ✅ **可识别不确定词**:
|
||||||
|
|
||||||
|
| 词 | Whisper 置信度 | 说明 |
|
||||||
|
|----|---------------|------|
|
||||||
|
| 明 (应为迷) | 0.69 | ⚠️ 低置信度，提示可能有误 |
|
||||||
|
| 天 (应为上) | 0.85 | ⚠️ 中等 |
|
||||||
|
| 最 (好/巨) | 0.42 | ⚠️ 最低置信度，不确定 |
|
||||||
|
| 折 (应为子) | 0.68 | ⚠️ 低置信度 |
|
||||||
|
| 板 (句3末) | 0.63 | ⚠️ 低置信度 |
|
||||||
|
| 一 (句11) | 0.56 | ⚠️ 低置信度 |
|
||||||
|
| 哇 | 0.55 | ⚠️ 低置信度 |
|
||||||
|
| 安逸 (应为安利) | 0.81/0.80 | ⚠️ 低置信度 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 五、综合评分
|
||||||
|
|
||||||
|
| 维度 | MiMo 2.5 | Whisper large-v3 | 胜者 |
|
||||||
|
|------|----------|------------------|------|
|
||||||
|
| **转录准确性** | ⭐⭐⭐⭐ (14/18 正确) | ⭐⭐⭐⭐ (13/18 正确) | **MiMo 微胜** |
|
||||||
|
| **语义理解** | ⭐⭐⭐⭐⭐ (迷上了、安利) | ⭐⭐⭐ (明天上了、安逸) | **MiMo 明显胜** |
|
||||||
|
| **时间戳精度** | ⭐⭐⭐ (估算值，偏晚60-500ms) | ⭐⭐⭐⭐⭐ (声学对齐) | **Whisper 明显胜** |
|
||||||
|
| **置信度质量** | ⭐ (固定0.95，无参考价值) | ⭐⭐⭐⭐⭐ (真实概率分布) | **Whisper 完胜** |
|
||||||
|
| **字词粒度** | ⭐⭐⭐⭐⭐ (174字，逐字) | ⭐⭐⭐⭐ (160词，部分合并) | **MiMo 微胜** |
|
||||||
|
| **部署便捷性** | ⭐⭐⭐⭐⭐ (一次API调用) | ⭐⭐⭐ (需安装模型+GPU) | **MiMo 胜** |
|
||||||
|
| **综合** | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ | **各有所长** |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 六、结论与建议
|
||||||
|
|
||||||
|
### MiMo 2.5 适合:
|
||||||
|
- 🎯 需要**语义正确**的转录（能理解网络用语"安利"）
|
||||||
|
- 🚀 快速一步到位（视频理解 + 转录 + 分镜同时完成）
|
||||||
|
- 📱 无 GPU 环境的轻量级部署
|
||||||
|
|
||||||
|
### Whisper 更适合:
|
||||||
|
- ⏱️ 需要**精确到毫秒**的时间戳（剪辑、字幕对齐）
|
||||||
|
- 📊 需要**置信度**来筛选不确定词
|
||||||
|
- 🔧 批量处理、离线场景
|
||||||
|
|
||||||
|
### 最佳实践:
|
||||||
|
**结合两者优势** — 用 Whisper 获取精确时间戳，用 MiMo 2.5 做语义校正和场景理解。
|
||||||
1
examples/meimei_yangmei/qwen3_asr_raw.txt
Normal file
1
examples/meimei_yangmei/qwen3_asr_raw.txt
Normal file
@ -0,0 +1 @@
|
|||||||
|
我最近迷恋上了这一款云南高山杨梅，真的是又鲜又甜巨好吃，而且这样子的两大板到手以后才七十九块九。你看它每个个头啊都非常的饱满，而且它的汁水非常非常的多，当天现摘的杨梅，口感鲜甜，果肉细嫩，真的特别特别的新鲜，越吃越有滋味。到手这样满满的一箱啊，吃起来真的好过瘾。哇，你看这个汁水，真的超级的过瘾，超级的满足，强烈安利给爱吃杨梅的每一个人。我家里的人都特别的爱吃，一次就能炫这一整盒，爱吃杨梅的家人们一定要来一箱尝尝看。
|
||||||
75
examples/meimei_yangmei/subtitles.srt
Normal file
75
examples/meimei_yangmei/subtitles.srt
Normal file
@ -0,0 +1,75 @@
|
|||||||
|
1
|
||||||
|
00:00:00,000 --> 00:00:02,399
|
||||||
|
我最近明天上了这一款云南高山杨梅
|
||||||
|
|
||||||
|
2
|
||||||
|
00:00:02,399 --> 00:00:03,980
|
||||||
|
真的是又鲜又甜 最好吃
|
||||||
|
|
||||||
|
3
|
||||||
|
00:00:03,980 --> 00:00:05,379
|
||||||
|
而且这样折两大板
|
||||||
|
|
||||||
|
4
|
||||||
|
00:00:05,379 --> 00:00:06,919
|
||||||
|
到手以后才79.9元
|
||||||
|
|
||||||
|
5
|
||||||
|
00:00:06,919 --> 00:00:09,179
|
||||||
|
你看它每个个头都非常的饱满
|
||||||
|
|
||||||
|
6
|
||||||
|
00:00:09,179 --> 00:00:11,039
|
||||||
|
而且它的汁水非常非常的多
|
||||||
|
|
||||||
|
7
|
||||||
|
00:00:11,039 --> 00:00:12,339
|
||||||
|
当天现摘的杨梅
|
||||||
|
|
||||||
|
8
|
||||||
|
00:00:12,339 --> 00:00:14,279
|
||||||
|
口感鲜甜 果肉细嫩
|
||||||
|
|
||||||
|
9
|
||||||
|
00:00:14,279 --> 00:00:16,120
|
||||||
|
真的特别特别的新鲜
|
||||||
|
|
||||||
|
10
|
||||||
|
00:00:16,120 --> 00:00:17,320
|
||||||
|
越吃越有滋味
|
||||||
|
|
||||||
|
11
|
||||||
|
00:00:17,320 --> 00:00:18,579
|
||||||
|
到手这样满满的一箱
|
||||||
|
|
||||||
|
12
|
||||||
|
00:00:18,579 --> 00:00:19,820
|
||||||
|
吃起来真的好过瘾
|
||||||
|
|
||||||
|
13
|
||||||
|
00:00:22,199 --> 00:00:23,679
|
||||||
|
哇 你看这个汁水
|
||||||
|
|
||||||
|
14
|
||||||
|
00:00:23,679 --> 00:00:25,839
|
||||||
|
真的超级的过瘾 超级的满足
|
||||||
|
|
||||||
|
15
|
||||||
|
00:00:25,839 --> 00:00:28,300
|
||||||
|
强烈安逸给爱吃杨梅的每一个人
|
||||||
|
|
||||||
|
16
|
||||||
|
00:00:28,300 --> 00:00:30,019
|
||||||
|
我家里的人都特别的爱吃
|
||||||
|
|
||||||
|
17
|
||||||
|
00:00:30,019 --> 00:00:31,440
|
||||||
|
一次就能掀这一整盒
|
||||||
|
|
||||||
|
18
|
||||||
|
00:00:31,440 --> 00:00:32,659
|
||||||
|
爱吃杨梅的家人们
|
||||||
|
|
||||||
|
19
|
||||||
|
00:00:32,659 --> 00:00:33,920
|
||||||
|
一定要来一箱尝尝看
|
||||||
103
examples/meimei_yangmei/three_way_comparison.md
Normal file
103
examples/meimei_yangmei/three_way_comparison.md
Normal file
@ -0,0 +1,103 @@
|
|||||||
|
# MiMo-V2.5-ASR vs MiMo 2.5 (VLM) vs Whisper — 三方转录对比
|
||||||
|
## 视频: test_meimei.mp4 (云南高山杨梅带货, 34秒)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 一、段落级逐句三方对比
|
||||||
|
|
||||||
|
| # | 时间 | MiMo-ASR (纯音频) | MiMo 2.5 (视频+音频) | Whisper large-v3 | 判定 |
|
||||||
|
|---|------|-------------------|---------------------|------------------|------|
|
||||||
|
| 1 | 00:00-00:02 | 我最近**迷恋上了**这一款云南高山杨梅 | 我最近**迷上了**这个云南高山杨梅 | 我最近**明天上了这**一款云南高山杨梅 | **VLM ✅最准** / ASR近似 |
|
||||||
|
| 2 | 00:02-00:04 | 真的是又鲜又甜，巨好吃 | 真的是又鲜又甜 巨好吃 | 真的是又鲜又甜 最好吃 | **ASR/VLM ✅** |
|
||||||
|
| 3 | 00:04-00:06 | 而且这样子的两大板 | 而且这样子的两大板 | 而且这样折两大板 | **ASR/VLM ✅** |
|
||||||
|
| 4 | 00:06-00:07 | 到手以后才七十九块九 | 到手以后才79.9 | 到手以后才79.9元 | **ASR ✅口语化准确** |
|
||||||
|
| 5 | 00:07-00:09 | 你看它每个个头啊都非常的饱满 | 你看他每个个头都非常的饱满 | 你看它每个个头都非常的饱满 | **平局** (口语等价) |
|
||||||
|
| 6 | 00:09-00:11 | 而且它的汁水非常非常的多 | 而且它的汁水非常的多 | 而且它的汁水非常非常的多 | **Whisper ✅更完整** |
|
||||||
|
| 7 | 00:11-00:13 | 当天现摘的杨梅 | 当天现摘的杨梅 | 当天现摘的杨梅 | ✅ 完全一致 |
|
||||||
|
| 8 | 00:13-00:15 | 口感鲜甜，果肉细嫩 | 口感鲜甜 果肉细嫩 | 口感鲜甜 果肉细嫩 | ✅ 完全一致 |
|
||||||
|
| 9 | 00:15-00:16 | 真的特别特别的新鲜 | 真的特别特别的新鲜 | 真的特别特别的新鲜 | ✅ 完全一致 |
|
||||||
|
| 10 | 00:16-00:17 | 越吃越有滋味 | 越吃越有滋味 | 越吃越有滋味 | ✅ 完全一致 |
|
||||||
|
| 11 | 00:18-00:20 | 到手这样满满的一箱啊 | 到手慢慢的满满一箱啊 | 到手这样满满的一箱 | **ASR ✅** |
|
||||||
|
| 12 | 00:20-00:21 | 吃起来真的好过瘾 | 吃起来真的好过瘾 | 吃起来真的好过瘾 | ✅ 完全一致 |
|
||||||
|
| 13 | 00:22-00:24 | 哇，你看这个汁水 | 哇~你看这个汁水 | 哇 你看这个汁水 | ✅ 完全一致 |
|
||||||
|
| 14 | 00:24-00:26 | 真的超级的过瘾，超级的满足 | 真的超级的过瘾 超级的满足 | 真的超级的过瘾 超级的满足 | ✅ 完全一致 |
|
||||||
|
| 15 | 00:27-00:29 | 强烈**安利**给爱吃杨梅的每一个人 | 强烈**安利**给每一个喜欢吃杨梅的人 | 强烈**安逸**给爱吃杨梅的每一个人 | **ASR/VLM ✅** / Whisper ❌ |
|
||||||
|
| 16 | 00:29-00:31 | 我家里的人都特别的爱吃 | 我家里人都特别爱吃 | 我家里的人都特别的爱吃 | **ASR/Whisper ✅更完整** |
|
||||||
|
| 17 | 00:31-00:33 | 一次就能炫**这一整盒** | 一次能炫一大盒 | 一次就能掀这一整盒 | **ASR ✅** |
|
||||||
|
| 18 | 00:33-00:36 | 爱吃杨梅的家人们一定要来一箱尝尝看 | 爱吃杨梅的家人们一定要来尝尝看 | 爱吃杨梅的家人们 + 一定要来一箱尝尝看 | **ASR ✅最完整** |
|
||||||
|
|
||||||
|
### 段落级准确率统计
|
||||||
|
|
||||||
|
| 模型 | 正确句数 | 准确率 | 关键错误 |
|
||||||
|
|------|---------|--------|----------|
|
||||||
|
| **MiMo-ASR** | 17/18 | **94.4%** | #1 "迷恋"多一字 |
|
||||||
|
| **MiMo 2.5 (VLM)** | 16/18 | **88.9%** | #11 "慢慢的"误听, #6 漏字 |
|
||||||
|
| **Whisper** | 13/18 | **72.2%** | #1 "明天上了", #3 "折", #15 "安逸" |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 二、关键转录差异详解
|
||||||
|
|
||||||
|
### 差异 1: 句1 — 三种不同结果
|
||||||
|
| 模型 | 文本 | 判定 |
|
||||||
|
|------|------|------|
|
||||||
|
| **MiMo-ASR** | 我最近**迷恋上了**这一款 | ⚠️ 多了"恋"字，但语义正确 |
|
||||||
|
| **MiMo 2.5 (VLM)** | 我最近**迷上了**这个 | ✅ **最准确** |
|
||||||
|
| **Whisper** | 我最近**明天上了这**一款 | ❌ "明天上了"不通顺 |
|
||||||
|
|
||||||
|
### 差异 2: 句15 — "安利" vs "安逸"
|
||||||
|
| 模型 | 文本 | Whisper置信度 |
|
||||||
|
|------|------|-------------|
|
||||||
|
| **MiMo-ASR** | 强烈**安利**给爱吃杨梅的每一个人 | — |
|
||||||
|
| **MiMo 2.5 (VLM)** | 强烈**安利**给每一个喜欢吃杨梅的人 | — |
|
||||||
|
| **Whisper** | 强烈**安逸**给爱吃杨梅的每一个人 | 安: 0.81, 逸: 0.80 ⚠️ |
|
||||||
|
|
||||||
|
三个模型中，MiMo-ASR 和 VLM 都正确识别为"安利"（网络用语=推荐），Whisper 受声学相似性影响识别为"安逸"。
|
||||||
|
|
||||||
|
### 差异 3: 句17 — 口语化表达
|
||||||
|
| 模型 | 文本 | 判定 |
|
||||||
|
|------|------|------|
|
||||||
|
| **MiMo-ASR** | 一次就能炫**这一整盒** | ✅ 准确 |
|
||||||
|
| **MiMo 2.5 (VLM)** | 一次能炫一大盒 | ⚠️ 漏了"就""这""整" |
|
||||||
|
| **Whisper** | 一次就能掀**这一整盒** | ⚠️ "掀"应为"炫" |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 三、MiMo-V2.5-ASR 特殊说明
|
||||||
|
|
||||||
|
### 代理限制
|
||||||
|
通过本地代理 (`127.0.0.1:15721`) 调用 `mimo-v2.5-asr` 时：
|
||||||
|
- ❌ **不支持附加文本提示** — 代理网关注入自己的 prompt
|
||||||
|
- ❌ **不支持字级时间戳输出** — 仅返回纯文本转录
|
||||||
|
- ❌ **max_tokens 限制 4096** — 不足以输出详细时间戳 JSON
|
||||||
|
- ✅ **转录准确性极高** — 94.4% 准确率，三者最高
|
||||||
|
|
||||||
|
### 官方 API 对比
|
||||||
|
如通过小米官方 API (`api.xiaomimimo.com/v1`) 直接调用，ASR 模型可能支持：
|
||||||
|
- `asr_options` 参数配置
|
||||||
|
- 更长的输出 token 限制
|
||||||
|
- 潜在的字级时间戳支持（需进一步测试）
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 四、综合评分
|
||||||
|
|
||||||
|
| 维度 | MiMo-ASR | MiMo 2.5 (VLM) | Whisper large-v3 |
|
||||||
|
|------|----------|-----------------|------------------|
|
||||||
|
| **转录准确率** | ⭐⭐⭐⭐⭐ (94.4%) | ⭐⭐⭐⭐ (88.9%) | ⭐⭐⭐ (72.2%) |
|
||||||
|
| **语义理解** | ⭐⭐⭐⭐ (迷恋、安利、炫) | ⭐⭐⭐⭐⭐ (迷上了、安利) | ⭐⭐ (明天上了、安逸) |
|
||||||
|
| **字级时间戳** | ❌ 代理不支持 | ⭐⭐⭐ (估算值) | ⭐⭐⭐⭐⭐ (声学对齐) |
|
||||||
|
| **置信度** | ❌ 无 | ⭐ (固定0.95) | ⭐⭐⭐⭐⭐ (真实概率) |
|
||||||
|
| **综合评价** | 纯文本最强 | 视频+音频联合 | 时间戳最强 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 五、结论
|
||||||
|
|
||||||
|
| 场景 | 推荐方案 |
|
||||||
|
|------|----------|
|
||||||
|
| **只需准确文本** | MiMo-V2.5-ASR (94.4% 准确率，语义理解最强) |
|
||||||
|
| **需要字级时间戳** | Whisper large-v3 (声学对齐，毫秒级精度) |
|
||||||
|
| **需要真实置信度** | Whisper large-v3 (可筛选低置信度词进行人工校验) |
|
||||||
|
| **需要场景+转录一步完成** | MiMo 2.5 视频理解 (分镜+转录同时输出) |
|
||||||
|
| **最佳组合** | MiMo-ASR (文本) + Whisper (时间戳) + MiMo 2.5 (场景) |
|
||||||
2119
examples/meimei_yangmei/whisper_full.json
Normal file
2119
examples/meimei_yangmei/whisper_full.json
Normal file
File diff suppressed because it is too large
Load Diff
639
examples/meimei_yangmei/words.srt
Normal file
639
examples/meimei_yangmei/words.srt
Normal file
@ -0,0 +1,639 @@
|
|||||||
|
1
|
||||||
|
00:00:00,000 --> 00:00:00,080
|
||||||
|
我 [0.92]
|
||||||
|
|
||||||
|
2
|
||||||
|
00:00:00,080 --> 00:00:00,220
|
||||||
|
最近 [0.98]
|
||||||
|
|
||||||
|
3
|
||||||
|
00:00:00,220 --> 00:00:00,460
|
||||||
|
明 [0.69]
|
||||||
|
|
||||||
|
4
|
||||||
|
00:00:00,460 --> 00:00:00,600
|
||||||
|
天 [0.85]
|
||||||
|
|
||||||
|
5
|
||||||
|
00:00:00,600 --> 00:00:00,780
|
||||||
|
上 [0.99]
|
||||||
|
|
||||||
|
6
|
||||||
|
00:00:00,780 --> 00:00:00,860
|
||||||
|
了 [0.91]
|
||||||
|
|
||||||
|
7
|
||||||
|
00:00:00,860 --> 00:00:00,980
|
||||||
|
这 [0.80]
|
||||||
|
|
||||||
|
8
|
||||||
|
00:00:00,980 --> 00:00:01,120
|
||||||
|
一 [0.84]
|
||||||
|
|
||||||
|
9
|
||||||
|
00:00:01,120 --> 00:00:01,300
|
||||||
|
款 [1.00]
|
||||||
|
|
||||||
|
10
|
||||||
|
00:00:01,300 --> 00:00:01,480
|
||||||
|
云 [0.99]
|
||||||
|
|
||||||
|
11
|
||||||
|
00:00:01,480 --> 00:00:01,659
|
||||||
|
南 [0.99]
|
||||||
|
|
||||||
|
12
|
||||||
|
00:00:01,659 --> 00:00:01,800
|
||||||
|
高 [0.99]
|
||||||
|
|
||||||
|
13
|
||||||
|
00:00:01,800 --> 00:00:01,980
|
||||||
|
山 [0.90]
|
||||||
|
|
||||||
|
14
|
||||||
|
00:00:01,980 --> 00:00:02,200
|
||||||
|
杨 [0.70]
|
||||||
|
|
||||||
|
15
|
||||||
|
00:00:02,200 --> 00:00:02,399
|
||||||
|
梅 [0.99]
|
||||||
|
|
||||||
|
16
|
||||||
|
00:00:02,399 --> 00:00:02,680
|
||||||
|
真的是 [0.97]
|
||||||
|
|
||||||
|
17
|
||||||
|
00:00:02,680 --> 00:00:02,980
|
||||||
|
又 [0.99]
|
||||||
|
|
||||||
|
18
|
||||||
|
00:00:02,980 --> 00:00:03,140
|
||||||
|
鲜 [1.00]
|
||||||
|
|
||||||
|
19
|
||||||
|
00:00:03,140 --> 00:00:03,279
|
||||||
|
又 [1.00]
|
||||||
|
|
||||||
|
20
|
||||||
|
00:00:03,279 --> 00:00:03,500
|
||||||
|
甜 [1.00]
|
||||||
|
|
||||||
|
21
|
||||||
|
00:00:03,580 --> 00:00:03,700
|
||||||
|
最 [0.42]
|
||||||
|
|
||||||
|
22
|
||||||
|
00:00:03,700 --> 00:00:03,980
|
||||||
|
好吃 [1.00]
|
||||||
|
|
||||||
|
23
|
||||||
|
00:00:03,980 --> 00:00:04,299
|
||||||
|
而且 [0.96]
|
||||||
|
|
||||||
|
24
|
||||||
|
00:00:04,299 --> 00:00:04,620
|
||||||
|
这样 [0.97]
|
||||||
|
|
||||||
|
25
|
||||||
|
00:00:04,620 --> 00:00:04,860
|
||||||
|
折 [0.68]
|
||||||
|
|
||||||
|
26
|
||||||
|
00:00:04,860 --> 00:00:04,980
|
||||||
|
两 [0.86]
|
||||||
|
|
||||||
|
27
|
||||||
|
00:00:04,980 --> 00:00:05,160
|
||||||
|
大 [1.00]
|
||||||
|
|
||||||
|
28
|
||||||
|
00:00:05,160 --> 00:00:05,379
|
||||||
|
板 [0.63]
|
||||||
|
|
||||||
|
29
|
||||||
|
00:00:05,379 --> 00:00:05,540
|
||||||
|
到 [0.79]
|
||||||
|
|
||||||
|
30
|
||||||
|
00:00:05,540 --> 00:00:05,759
|
||||||
|
手 [1.00]
|
||||||
|
|
||||||
|
31
|
||||||
|
00:00:05,759 --> 00:00:05,860
|
||||||
|
以 [0.97]
|
||||||
|
|
||||||
|
32
|
||||||
|
00:00:05,860 --> 00:00:05,919
|
||||||
|
后 [1.00]
|
||||||
|
|
||||||
|
33
|
||||||
|
00:00:05,919 --> 00:00:06,059
|
||||||
|
才 [1.00]
|
||||||
|
|
||||||
|
34
|
||||||
|
00:00:06,059 --> 00:00:06,440
|
||||||
|
79. [0.86]
|
||||||
|
|
||||||
|
35
|
||||||
|
00:00:06,660 --> 00:00:06,900
|
||||||
|
9 [0.99]
|
||||||
|
|
||||||
|
36
|
||||||
|
00:00:06,900 --> 00:00:06,919
|
||||||
|
元 [0.83]
|
||||||
|
|
||||||
|
37
|
||||||
|
00:00:06,919 --> 00:00:07,200
|
||||||
|
你看 [0.90]
|
||||||
|
|
||||||
|
38
|
||||||
|
00:00:07,200 --> 00:00:07,440
|
||||||
|
它 [0.90]
|
||||||
|
|
||||||
|
39
|
||||||
|
00:00:07,440 --> 00:00:07,660
|
||||||
|
每 [1.00]
|
||||||
|
|
||||||
|
40
|
||||||
|
00:00:07,660 --> 00:00:07,780
|
||||||
|
个 [0.75]
|
||||||
|
|
||||||
|
41
|
||||||
|
00:00:07,780 --> 00:00:07,940
|
||||||
|
个 [0.94]
|
||||||
|
|
||||||
|
42
|
||||||
|
00:00:07,940 --> 00:00:08,119
|
||||||
|
头 [0.98]
|
||||||
|
|
||||||
|
43
|
||||||
|
00:00:08,119 --> 00:00:08,339
|
||||||
|
都 [0.91]
|
||||||
|
|
||||||
|
44
|
||||||
|
00:00:08,339 --> 00:00:08,699
|
||||||
|
非常的 [0.91]
|
||||||
|
|
||||||
|
45
|
||||||
|
00:00:08,699 --> 00:00:09,019
|
||||||
|
饱 [1.00]
|
||||||
|
|
||||||
|
46
|
||||||
|
00:00:09,019 --> 00:00:09,179
|
||||||
|
满 [1.00]
|
||||||
|
|
||||||
|
47
|
||||||
|
00:00:09,179 --> 00:00:09,339
|
||||||
|
而且 [0.99]
|
||||||
|
|
||||||
|
48
|
||||||
|
00:00:09,339 --> 00:00:09,599
|
||||||
|
它的 [0.99]
|
||||||
|
|
||||||
|
49
|
||||||
|
00:00:09,599 --> 00:00:09,859
|
||||||
|
汁 [1.00]
|
||||||
|
|
||||||
|
50
|
||||||
|
00:00:09,859 --> 00:00:10,019
|
||||||
|
水 [1.00]
|
||||||
|
|
||||||
|
51
|
||||||
|
00:00:10,019 --> 00:00:10,380
|
||||||
|
非常 [0.91]
|
||||||
|
|
||||||
|
52
|
||||||
|
00:00:10,380 --> 00:00:10,720
|
||||||
|
非常的 [0.74]
|
||||||
|
|
||||||
|
53
|
||||||
|
00:00:10,720 --> 00:00:11,039
|
||||||
|
多 [1.00]
|
||||||
|
|
||||||
|
54
|
||||||
|
00:00:11,039 --> 00:00:11,259
|
||||||
|
当 [1.00]
|
||||||
|
|
||||||
|
55
|
||||||
|
00:00:11,259 --> 00:00:11,439
|
||||||
|
天 [1.00]
|
||||||
|
|
||||||
|
56
|
||||||
|
00:00:11,439 --> 00:00:11,660
|
||||||
|
现 [0.96]
|
||||||
|
|
||||||
|
57
|
||||||
|
00:00:11,660 --> 00:00:11,859
|
||||||
|
摘 [0.95]
|
||||||
|
|
||||||
|
58
|
||||||
|
00:00:11,859 --> 00:00:11,960
|
||||||
|
的 [1.00]
|
||||||
|
|
||||||
|
59
|
||||||
|
00:00:11,960 --> 00:00:12,140
|
||||||
|
杨 [1.00]
|
||||||
|
|
||||||
|
60
|
||||||
|
00:00:12,140 --> 00:00:12,339
|
||||||
|
梅 [1.00]
|
||||||
|
|
||||||
|
61
|
||||||
|
00:00:12,339 --> 00:00:12,580
|
||||||
|
口 [0.85]
|
||||||
|
|
||||||
|
62
|
||||||
|
00:00:12,580 --> 00:00:12,900
|
||||||
|
感 [1.00]
|
||||||
|
|
||||||
|
63
|
||||||
|
00:00:12,900 --> 00:00:13,119
|
||||||
|
鲜 [1.00]
|
||||||
|
|
||||||
|
64
|
||||||
|
00:00:13,119 --> 00:00:13,359
|
||||||
|
甜 [1.00]
|
||||||
|
|
||||||
|
65
|
||||||
|
00:00:13,460 --> 00:00:13,640
|
||||||
|
果 [0.98]
|
||||||
|
|
||||||
|
66
|
||||||
|
00:00:13,640 --> 00:00:13,839
|
||||||
|
肉 [1.00]
|
||||||
|
|
||||||
|
67
|
||||||
|
00:00:13,839 --> 00:00:14,099
|
||||||
|
细 [0.99]
|
||||||
|
|
||||||
|
68
|
||||||
|
00:00:14,099 --> 00:00:14,279
|
||||||
|
嫩 [0.96]
|
||||||
|
|
||||||
|
69
|
||||||
|
00:00:14,279 --> 00:00:14,880
|
||||||
|
真的 [0.99]
|
||||||
|
|
||||||
|
70
|
||||||
|
00:00:14,880 --> 00:00:15,099
|
||||||
|
特 [1.00]
|
||||||
|
|
||||||
|
71
|
||||||
|
00:00:15,099 --> 00:00:15,359
|
||||||
|
别 [1.00]
|
||||||
|
|
||||||
|
72
|
||||||
|
00:00:15,359 --> 00:00:15,500
|
||||||
|
特 [0.89]
|
||||||
|
|
||||||
|
73
|
||||||
|
00:00:15,500 --> 00:00:15,619
|
||||||
|
别 [1.00]
|
||||||
|
|
||||||
|
74
|
||||||
|
00:00:15,619 --> 00:00:15,679
|
||||||
|
的 [0.95]
|
||||||
|
|
||||||
|
75
|
||||||
|
00:00:15,679 --> 00:00:15,880
|
||||||
|
新 [0.95]
|
||||||
|
|
||||||
|
76
|
||||||
|
00:00:15,880 --> 00:00:16,120
|
||||||
|
鲜 [1.00]
|
||||||
|
|
||||||
|
77
|
||||||
|
00:00:16,120 --> 00:00:16,339
|
||||||
|
越 [0.94]
|
||||||
|
|
||||||
|
78
|
||||||
|
00:00:16,339 --> 00:00:16,600
|
||||||
|
吃 [1.00]
|
||||||
|
|
||||||
|
79
|
||||||
|
00:00:16,600 --> 00:00:16,780
|
||||||
|
越 [0.99]
|
||||||
|
|
||||||
|
80
|
||||||
|
00:00:16,780 --> 00:00:16,899
|
||||||
|
有 [1.00]
|
||||||
|
|
||||||
|
81
|
||||||
|
00:00:16,899 --> 00:00:17,140
|
||||||
|
滋 [1.00]
|
||||||
|
|
||||||
|
82
|
||||||
|
00:00:17,140 --> 00:00:17,320
|
||||||
|
味 [1.00]
|
||||||
|
|
||||||
|
83
|
||||||
|
00:00:17,320 --> 00:00:17,559
|
||||||
|
到 [0.91]
|
||||||
|
|
||||||
|
84
|
||||||
|
00:00:17,559 --> 00:00:17,699
|
||||||
|
手 [1.00]
|
||||||
|
|
||||||
|
85
|
||||||
|
00:00:17,699 --> 00:00:17,879
|
||||||
|
这样 [0.97]
|
||||||
|
|
||||||
|
86
|
||||||
|
00:00:17,879 --> 00:00:18,079
|
||||||
|
满 [1.00]
|
||||||
|
|
||||||
|
87
|
||||||
|
00:00:18,079 --> 00:00:18,179
|
||||||
|
满 [1.00]
|
||||||
|
|
||||||
|
88
|
||||||
|
00:00:18,179 --> 00:00:18,280
|
||||||
|
的 [0.98]
|
||||||
|
|
||||||
|
89
|
||||||
|
00:00:18,280 --> 00:00:18,339
|
||||||
|
一 [0.56]
|
||||||
|
|
||||||
|
90
|
||||||
|
00:00:18,339 --> 00:00:18,579
|
||||||
|
箱 [0.98]
|
||||||
|
|
||||||
|
91
|
||||||
|
00:00:18,579 --> 00:00:18,899
|
||||||
|
吃 [0.85]
|
||||||
|
|
||||||
|
92
|
||||||
|
00:00:18,899 --> 00:00:19,100
|
||||||
|
起 [0.98]
|
||||||
|
|
||||||
|
93
|
||||||
|
00:00:19,100 --> 00:00:19,179
|
||||||
|
来 [0.98]
|
||||||
|
|
||||||
|
94
|
||||||
|
00:00:19,179 --> 00:00:19,300
|
||||||
|
真的 [0.98]
|
||||||
|
|
||||||
|
95
|
||||||
|
00:00:19,300 --> 00:00:19,460
|
||||||
|
好 [1.00]
|
||||||
|
|
||||||
|
96
|
||||||
|
00:00:19,460 --> 00:00:19,660
|
||||||
|
过 [1.00]
|
||||||
|
|
||||||
|
97
|
||||||
|
00:00:19,660 --> 00:00:19,820
|
||||||
|
瘾 [0.99]
|
||||||
|
|
||||||
|
98
|
||||||
|
00:00:22,199 --> 00:00:22,559
|
||||||
|
哇 [0.55]
|
||||||
|
|
||||||
|
99
|
||||||
|
00:00:22,559 --> 00:00:22,879
|
||||||
|
你 [0.90]
|
||||||
|
|
||||||
|
100
|
||||||
|
00:00:22,879 --> 00:00:23,079
|
||||||
|
看 [1.00]
|
||||||
|
|
||||||
|
101
|
||||||
|
00:00:23,079 --> 00:00:23,239
|
||||||
|
这个 [0.99]
|
||||||
|
|
||||||
|
102
|
||||||
|
00:00:23,239 --> 00:00:23,440
|
||||||
|
汁 [1.00]
|
||||||
|
|
||||||
|
103
|
||||||
|
00:00:23,440 --> 00:00:23,679
|
||||||
|
水 [1.00]
|
||||||
|
|
||||||
|
104
|
||||||
|
00:00:23,679 --> 00:00:24,000
|
||||||
|
真的 [0.99]
|
||||||
|
|
||||||
|
105
|
||||||
|
00:00:24,000 --> 00:00:24,239
|
||||||
|
超 [1.00]
|
||||||
|
|
||||||
|
106
|
||||||
|
00:00:24,239 --> 00:00:24,559
|
||||||
|
级 [0.99]
|
||||||
|
|
||||||
|
107
|
||||||
|
00:00:24,559 --> 00:00:24,699
|
||||||
|
的 [0.97]
|
||||||
|
|
||||||
|
108
|
||||||
|
00:00:24,699 --> 00:00:24,820
|
||||||
|
过 [0.99]
|
||||||
|
|
||||||
|
109
|
||||||
|
00:00:24,820 --> 00:00:24,960
|
||||||
|
瘾 [1.00]
|
||||||
|
|
||||||
|
110
|
||||||
|
00:00:25,039 --> 00:00:25,160
|
||||||
|
超 [1.00]
|
||||||
|
|
||||||
|
111
|
||||||
|
00:00:25,160 --> 00:00:25,320
|
||||||
|
级 [1.00]
|
||||||
|
|
||||||
|
112
|
||||||
|
00:00:25,320 --> 00:00:25,460
|
||||||
|
的 [0.99]
|
||||||
|
|
||||||
|
113
|
||||||
|
00:00:25,460 --> 00:00:25,600
|
||||||
|
满 [1.00]
|
||||||
|
|
||||||
|
114
|
||||||
|
00:00:25,600 --> 00:00:25,839
|
||||||
|
足 [1.00]
|
||||||
|
|
||||||
|
115
|
||||||
|
00:00:25,839 --> 00:00:26,280
|
||||||
|
强 [0.98]
|
||||||
|
|
||||||
|
116
|
||||||
|
00:00:26,280 --> 00:00:26,440
|
||||||
|
烈 [1.00]
|
||||||
|
|
||||||
|
117
|
||||||
|
00:00:26,440 --> 00:00:26,579
|
||||||
|
安 [0.81]
|
||||||
|
|
||||||
|
118
|
||||||
|
00:00:26,579 --> 00:00:26,699
|
||||||
|
逸 [0.80]
|
||||||
|
|
||||||
|
119
|
||||||
|
00:00:26,699 --> 00:00:26,859
|
||||||
|
给 [1.00]
|
||||||
|
|
||||||
|
120
|
||||||
|
00:00:26,859 --> 00:00:27,100
|
||||||
|
爱 [1.00]
|
||||||
|
|
||||||
|
121
|
||||||
|
00:00:27,100 --> 00:00:27,339
|
||||||
|
吃 [1.00]
|
||||||
|
|
||||||
|
122
|
||||||
|
00:00:27,339 --> 00:00:27,539
|
||||||
|
杨 [1.00]
|
||||||
|
|
||||||
|
123
|
||||||
|
00:00:27,539 --> 00:00:27,679
|
||||||
|
梅 [1.00]
|
||||||
|
|
||||||
|
124
|
||||||
|
00:00:27,679 --> 00:00:27,780
|
||||||
|
的 [1.00]
|
||||||
|
|
||||||
|
125
|
||||||
|
00:00:27,780 --> 00:00:27,920
|
||||||
|
每 [1.00]
|
||||||
|
|
||||||
|
126
|
||||||
|
00:00:27,920 --> 00:00:28,039
|
||||||
|
一个 [0.99]
|
||||||
|
|
||||||
|
127
|
||||||
|
00:00:28,039 --> 00:00:28,300
|
||||||
|
人 [1.00]
|
||||||
|
|
||||||
|
128
|
||||||
|
00:00:28,300 --> 00:00:28,719
|
||||||
|
我 [0.99]
|
||||||
|
|
||||||
|
129
|
||||||
|
00:00:28,719 --> 00:00:28,879
|
||||||
|
家 [1.00]
|
||||||
|
|
||||||
|
130
|
||||||
|
00:00:28,879 --> 00:00:29,019
|
||||||
|
里 [1.00]
|
||||||
|
|
||||||
|
131
|
||||||
|
00:00:29,019 --> 00:00:29,219
|
||||||
|
的人 [0.99]
|
||||||
|
|
||||||
|
132
|
||||||
|
00:00:29,219 --> 00:00:29,420
|
||||||
|
都 [1.00]
|
||||||
|
|
||||||
|
133
|
||||||
|
00:00:29,420 --> 00:00:29,500
|
||||||
|
特 [1.00]
|
||||||
|
|
||||||
|
134
|
||||||
|
00:00:29,500 --> 00:00:29,660
|
||||||
|
别 [1.00]
|
||||||
|
|
||||||
|
135
|
||||||
|
00:00:29,660 --> 00:00:29,699
|
||||||
|
的 [0.87]
|
||||||
|
|
||||||
|
136
|
||||||
|
00:00:29,699 --> 00:00:29,800
|
||||||
|
爱 [1.00]
|
||||||
|
|
||||||
|
137
|
||||||
|
00:00:29,800 --> 00:00:30,019
|
||||||
|
吃 [1.00]
|
||||||
|
|
||||||
|
138
|
||||||
|
00:00:30,019 --> 00:00:30,320
|
||||||
|
一次 [0.92]
|
||||||
|
|
||||||
|
139
|
||||||
|
00:00:30,320 --> 00:00:30,519
|
||||||
|
就 [0.99]
|
||||||
|
|
||||||
|
140
|
||||||
|
00:00:30,519 --> 00:00:30,660
|
||||||
|
能 [1.00]
|
||||||
|
|
||||||
|
141
|
||||||
|
00:00:30,660 --> 00:00:30,899
|
||||||
|
掀 [0.93]
|
||||||
|
|
||||||
|
142
|
||||||
|
00:00:30,899 --> 00:00:31,000
|
||||||
|
这 [0.88]
|
||||||
|
|
||||||
|
143
|
||||||
|
00:00:31,000 --> 00:00:31,120
|
||||||
|
一 [1.00]
|
||||||
|
|
||||||
|
144
|
||||||
|
00:00:31,120 --> 00:00:31,260
|
||||||
|
整 [1.00]
|
||||||
|
|
||||||
|
145
|
||||||
|
00:00:31,260 --> 00:00:31,440
|
||||||
|
盒 [0.99]
|
||||||
|
|
||||||
|
146
|
||||||
|
00:00:31,440 --> 00:00:31,620
|
||||||
|
爱 [0.97]
|
||||||
|
|
||||||
|
147
|
||||||
|
00:00:31,620 --> 00:00:31,839
|
||||||
|
吃 [1.00]
|
||||||
|
|
||||||
|
148
|
||||||
|
00:00:31,839 --> 00:00:32,060
|
||||||
|
杨 [0.71]
|
||||||
|
|
||||||
|
149
|
||||||
|
00:00:32,060 --> 00:00:32,200
|
||||||
|
梅 [0.98]
|
||||||
|
|
||||||
|
150
|
||||||
|
00:00:32,200 --> 00:00:32,280
|
||||||
|
的 [1.00]
|
||||||
|
|
||||||
|
151
|
||||||
|
00:00:32,280 --> 00:00:32,399
|
||||||
|
家 [0.96]
|
||||||
|
|
||||||
|
152
|
||||||
|
00:00:32,399 --> 00:00:32,560
|
||||||
|
人 [1.00]
|
||||||
|
|
||||||
|
153
|
||||||
|
00:00:32,560 --> 00:00:32,659
|
||||||
|
们 [1.00]
|
||||||
|
|
||||||
|
154
|
||||||
|
00:00:32,659 --> 00:00:32,939
|
||||||
|
一定要 [0.87]
|
||||||
|
|
||||||
|
155
|
||||||
|
00:00:32,939 --> 00:00:33,159
|
||||||
|
来 [1.00]
|
||||||
|
|
||||||
|
156
|
||||||
|
00:00:33,159 --> 00:00:33,340
|
||||||
|
一 [0.68]
|
||||||
|
|
||||||
|
157
|
||||||
|
00:00:33,340 --> 00:00:33,479
|
||||||
|
箱 [0.62]
|
||||||
|
|
||||||
|
158
|
||||||
|
00:00:33,479 --> 00:00:33,659
|
||||||
|
尝 [0.99]
|
||||||
|
|
||||||
|
159
|
||||||
|
00:00:33,659 --> 00:00:33,780
|
||||||
|
尝 [1.00]
|
||||||
|
|
||||||
|
160
|
||||||
|
00:00:33,780 --> 00:00:33,920
|
||||||
|
看 [1.00]
|
||||||
185
scripts/pipeline.py
Normal file
185
scripts/pipeline.py
Normal file
@ -0,0 +1,185 @@
|
|||||||
|
"""
|
||||||
|
AutoVideo - 视频处理分析一键流程
|
||||||
|
方案: MiMo 2.5 (分镜) + Qwen3-ASR-Flash FileTrans (转录+字级时间戳)
|
||||||
|
"""
|
||||||
|
import subprocess
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import io
|
||||||
|
import time
|
||||||
|
import base64
|
||||||
|
import requests
|
||||||
|
|
||||||
|
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8', errors='replace')
|
||||||
|
|
||||||
|
|
||||||
|
def phase2_preprocess(video_path, output_dir):
|
||||||
|
"""压缩视频 + 提取音频 + 提取关键帧"""
|
||||||
|
name = os.path.splitext(os.path.basename(video_path))[0]
|
||||||
|
compressed = os.path.join(output_dir, f"{name}_compressed.mp4")
|
||||||
|
orig = os.path.getsize(video_path)
|
||||||
|
subprocess.run([
|
||||||
|
"ffmpeg", "-y", "-i", video_path,
|
||||||
|
"-vf", "scale=-2:480", "-r", "15",
|
||||||
|
"-c:v", "libx264", "-preset", "fast", "-crf", "28",
|
||||||
|
"-c:a", "aac", "-b:a", "48k", "-ar", "16000", "-ac", "1",
|
||||||
|
"-movflags", "+faststart", compressed
|
||||||
|
], capture_output=True, check=True)
|
||||||
|
comp = os.path.getsize(compressed)
|
||||||
|
print(f" 压缩: {orig/1024/1024:.1f}MB → {comp/1024/1024:.1f}MB ({comp/orig*100:.0f}%)")
|
||||||
|
|
||||||
|
audio = os.path.join(output_dir, f"{name}_audio.wav")
|
||||||
|
subprocess.run([
|
||||||
|
"ffmpeg", "-y", "-i", video_path,
|
||||||
|
"-vn", "-acodec", "pcm_s16le", "-ar", "16000", "-ac", "1", audio
|
||||||
|
], capture_output=True, check=True)
|
||||||
|
print(f" 音频: {os.path.getsize(audio)/1024:.0f} KB")
|
||||||
|
|
||||||
|
frames_dir = os.path.join(output_dir, f"{name}_frames")
|
||||||
|
os.makedirs(frames_dir, exist_ok=True)
|
||||||
|
subprocess.run([
|
||||||
|
"ffmpeg", "-y", "-i", video_path,
|
||||||
|
"-vf", "fps=2,scale=480:-2", "-q:v", "5",
|
||||||
|
os.path.join(frames_dir, "frame_%04d.jpg")
|
||||||
|
], capture_output=True, check=True)
|
||||||
|
frames = len([f for f in os.listdir(frames_dir) if f.endswith('.jpg')])
|
||||||
|
print(f" 关键帧: {frames} 帧")
|
||||||
|
return compressed, audio
|
||||||
|
|
||||||
|
|
||||||
|
def phase3_mimo_scene(compressed_path):
|
||||||
|
"""MiMo 2.5 分镜分析 (OpenAI 格式)"""
|
||||||
|
import openai
|
||||||
|
with open(compressed_path, "rb") as f:
|
||||||
|
video_b64 = base64.b64encode(f.read()).decode()
|
||||||
|
client = openai.OpenAI(base_url="http://127.0.0.1:15721/v1", api_key="dummy-key")
|
||||||
|
response = client.chat.completions.create(
|
||||||
|
model="mimo-v2.5",
|
||||||
|
messages=[{"role": "user", "content": [
|
||||||
|
{"type": "video_url", "video_url": {"url": f"data:video/mp4;base64,{video_b64}"}},
|
||||||
|
{"type": "text", "text": """请逐镜头分析这个视频，输出JSON：
|
||||||
|
{"video_summary":"概述","scenes":[{"scene_id":1,"start_time":"MM:SS.mmm","end_time":"MM:SS.mmm","duration_sec":数字,"scene_type":"类型","visual_description":"详细视觉描述","camera":"运镜","transition":"转场"}],"timeline":"时间线叙事总结"}"""}
|
||||||
|
]}],
|
||||||
|
max_tokens=8192, temperature=0.1)
|
||||||
|
text = response.choices[0].message.content
|
||||||
|
clean = text.strip()
|
||||||
|
if clean.startswith("```"): clean = clean.split("\n", 1)[1]
|
||||||
|
if clean.endswith("```"): clean = clean.rsplit("```", 1)[0]
|
||||||
|
try:
|
||||||
|
return json.loads(clean.strip())
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
return {"raw_text": text}
|
||||||
|
|
||||||
|
|
||||||
|
def phase4_qwen3_asr(audio_path):
|
||||||
|
"""Qwen3-ASR-Flash FileTrans: 转录 + 字级时间戳 (一步到位)"""
|
||||||
|
import dashscope
|
||||||
|
API_KEY = dashscope.api_key
|
||||||
|
|
||||||
|
# 1. 上传获取临时URL
|
||||||
|
upload = dashscope.Files.upload(file_path=audio_path, purpose="file-extract")
|
||||||
|
file_id = upload.output["uploaded_files"][0]["file_id"]
|
||||||
|
url = dashscope.Files.get(file_id=file_id).output["url"]
|
||||||
|
|
||||||
|
# 2. 提交转录任务
|
||||||
|
resp = requests.post(
|
||||||
|
"https://dashscope.aliyuncs.com/api/v1/services/audio/asr/transcription",
|
||||||
|
headers={"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json",
|
||||||
|
"X-DashScope-Async": "enable"},
|
||||||
|
json={"model": "qwen3-asr-flash-filetrans",
|
||||||
|
"input": {"file_url": url},
|
||||||
|
"parameters": {"channel_id": [0], "enable_words": True, "enable_itn": False, "language": "zh"}}
|
||||||
|
)
|
||||||
|
task_id = resp.json()["output"]["task_id"]
|
||||||
|
|
||||||
|
# 3. 轮询结果
|
||||||
|
for i in range(30):
|
||||||
|
time.sleep(2)
|
||||||
|
result = requests.get(f"https://dashscope.aliyuncs.com/api/v1/tasks/{task_id}",
|
||||||
|
headers={"Authorization": f"Bearer {API_KEY}"}).json()
|
||||||
|
if result["output"]["task_status"] == "SUCCEEDED":
|
||||||
|
break
|
||||||
|
elif result["output"]["task_status"] == "FAILED":
|
||||||
|
raise Exception(f"转录失败: {result['output']}")
|
||||||
|
|
||||||
|
# 4. 获取转录JSON
|
||||||
|
tr_url = result["output"]["result"]["transcription_url"]
|
||||||
|
return requests.get(tr_url).json()
|
||||||
|
|
||||||
|
|
||||||
|
def generate_srt(trans_data, output_path):
|
||||||
|
"""生成 SRT 字幕"""
|
||||||
|
lines, idx = [], 1
|
||||||
|
for ch in trans_data.get("transcripts", []):
|
||||||
|
for sent in ch.get("sentences", []):
|
||||||
|
b, e, t = sent["begin_time"], sent["end_time"], sent["text"]
|
||||||
|
bh, bm, bs, bms = b//3600000, (b%3600000)//60000, (b%60000)//1000, b%1000
|
||||||
|
eh, em, es, ems = e//3600000, (e%3600000)//60000, (e%60000)//1000, e%1000
|
||||||
|
lines.extend([f"{idx}",
|
||||||
|
f"{bh:02d}:{bm:02d}:{bs:02d},{bms:03d} --> {eh:02d}:{em:02d}:{es:02d},{ems:03d}",
|
||||||
|
t, ""])
|
||||||
|
idx += 1
|
||||||
|
with open(output_path, "w", encoding="utf-8") as f:
|
||||||
|
f.write("\n".join(lines))
|
||||||
|
|
||||||
|
|
||||||
|
def print_timestamps(trans_data):
|
||||||
|
"""打印字级时间戳"""
|
||||||
|
for ch in trans_data.get("transcripts", []):
|
||||||
|
total = 0
|
||||||
|
for sent in ch.get("sentences", []):
|
||||||
|
print(f"\n [{sent['begin_time']:>6}ms - {sent['end_time']:>6}ms] {sent['text']}")
|
||||||
|
for w in sent.get("words", []):
|
||||||
|
print(f" [{w['begin_time']:>6}ms - {w['end_time']:>6}ms] {w['text']}{w.get('punctuation','')}")
|
||||||
|
total += 1
|
||||||
|
print(f"\n 总字数: {total}")
|
||||||
|
|
||||||
|
|
||||||
|
def run_pipeline(video_path, output_dir=None):
|
||||||
|
name = os.path.splitext(os.path.basename(video_path))[0]
|
||||||
|
if output_dir is None:
|
||||||
|
output_dir = os.path.join(os.path.dirname(video_path), f"{name}_analysis")
|
||||||
|
os.makedirs(output_dir, exist_ok=True)
|
||||||
|
|
||||||
|
print(f"{'='*60}\nAutoVideo: {name}\n{'='*60}")
|
||||||
|
|
||||||
|
# Phase 2
|
||||||
|
print(f"\n▶ Phase 2: 压缩预处理")
|
||||||
|
compressed, audio = phase2_preprocess(video_path, output_dir)
|
||||||
|
|
||||||
|
# Phase 3
|
||||||
|
print(f"\n▶ Phase 3: MiMo 2.5 分镜分析")
|
||||||
|
try:
|
||||||
|
scene = phase3_mimo_scene(compressed)
|
||||||
|
with open(os.path.join(output_dir, "scene.json"), "w", encoding="utf-8") as f:
|
||||||
|
json.dump(scene, f, ensure_ascii=False, indent=2)
|
||||||
|
for s in scene.get("scenes", []):
|
||||||
|
print(f" [{s.get('start_time','?')}→{s.get('end_time','?')}] {s.get('scene_type','')}")
|
||||||
|
except Exception as e:
|
||||||
|
print(f" 错误: {e}")
|
||||||
|
|
||||||
|
# Phase 4 (转录 + 字级时间戳 一步到位)
|
||||||
|
print(f"\n▶ Phase 4: Qwen3-ASR 转录 + 字级时间戳")
|
||||||
|
try:
|
||||||
|
ts = phase4_qwen3_asr(audio)
|
||||||
|
with open(os.path.join(output_dir, "timestamps.json"), "w", encoding="utf-8") as f:
|
||||||
|
json.dump(ts, f, ensure_ascii=False, indent=2)
|
||||||
|
generate_srt(ts, os.path.join(output_dir, "subtitles.srt"))
|
||||||
|
print_timestamps(ts)
|
||||||
|
except Exception as e:
|
||||||
|
print(f" 错误: {e}")
|
||||||
|
|
||||||
|
# 汇总
|
||||||
|
print(f"\n{'='*60}\n完成! 输出: {output_dir}\n{'='*60}")
|
||||||
|
for f in sorted(os.listdir(output_dir)):
|
||||||
|
fp = os.path.join(output_dir, f)
|
||||||
|
if os.path.isfile(fp):
|
||||||
|
print(f" {f:40s} {os.path.getsize(fp)/1024:>6.1f} KB")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
if len(sys.argv) < 2:
|
||||||
|
print("Usage: python pipeline.py <video_path> [output_dir]")
|
||||||
|
sys.exit(1)
|
||||||
|
run_pipeline(sys.argv[1], sys.argv[2] if len(sys.argv) > 2 else None)
|
||||||
6
test_pipeline.py
Normal file
6
test_pipeline.py
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
"""测试: 视频处理分析流程"""
|
||||||
|
import sys
|
||||||
|
sys.path.insert(0, r"D:\claude\aotovideo\scripts")
|
||||||
|
from pipeline import run_pipeline
|
||||||
|
|
||||||
|
run_pipeline(r"C:\Users\runst\Desktop\test_meimei.mp4")
|
||||||
Loading…
Reference in New Issue
Block a user