f176e9a402
- MiMo 2.5 分镜分析 (OpenAI格式调用) - Qwen3-ASR-Flash FileTrans 转录+字级时间戳 - 一键 pipeline 脚本 - 杨梅/绿豆视频示例输出 - 完整文档和SOP
95 lines
2.8 KiB
Markdown
95 lines
2.8 KiB
Markdown
# 视频处理分析流程 (SOP)
|
|
|
|
## 流程总览
|
|
|
|
```
|
|
视频文件
|
|
│
|
|
▼ Phase 1: 信息采集 ────────── ffprobe
|
|
│
|
|
▼ Phase 2: 压缩预处理 ──────── ffmpeg (压缩+音频+关键帧)
|
|
│
|
|
├─→ Phase 3: 分镜分析 ──────── MiMo 2.5 (OpenAI格式)
|
|
│
|
|
├─→ Phase 4a: 转录文本 ─────── Qwen3-ASR-Flash chat API
|
|
│
|
|
└─→ Phase 4b: 字级时间戳 ───── Qwen3-ASR-Flash FileTrans API
|
|
│
|
|
▼ Phase 5: 输出 ────────────── 报告 + JSON + SRT
|
|
```
|
|
|
|
## Phase 1: 信息采集
|
|
|
|
```bash
|
|
ffprobe -v quiet -print_format json -show_format -show_streams <视频路径>
|
|
```
|
|
|
|
## Phase 2: 压缩预处理
|
|
|
|
```bash
|
|
# 压缩
|
|
ffmpeg -y -i <输入> -vf "scale=-2:480" -c:v libx264 -preset fast -crf 28 \
|
|
-c:a aac -b:a 48k -ar 16000 -ac 1 -movflags +faststart <输出.mp4>
|
|
|
|
# 提取音频
|
|
ffmpeg -y -i <输入> -vn -acodec pcm_s16le -ar 16000 -ac 1 <输出.wav>
|
|
|
|
# 提取关键帧
|
|
ffmpeg -y -i <输入> -vf "fps=2,scale=480:-2" -q:v 5 <输出目录>/frame_%04d.jpg
|
|
```
|
|
|
|
## Phase 3: 分镜分析 (MiMo 2.5)
|
|
|
|
```python
|
|
# 必须用 OpenAI 格式!
|
|
client = openai.OpenAI(base_url="http://127.0.0.1:15721/v1", api_key="dummy-key")
|
|
response = client.chat.completions.create(
|
|
model="mimo-v2.5",
|
|
messages=[{"role": "user", "content": [
|
|
{"type": "video_url", "video_url": {"url": f"data:video/mp4;base64,{b64}"}},
|
|
{"type": "text", "text": "请逐镜头分析这个视频..."}
|
|
]}],
|
|
max_tokens=8192
|
|
)
|
|
```
|
|
|
|
## Phase 4a: 转录文本 (Qwen3 chat API)
|
|
|
|
```python
|
|
response = client.chat.completions.create(
|
|
model="qwen3-asr-flash",
|
|
messages=[{"role": "user", "content": [
|
|
{"type": "input_audio", "input_audio": {"data": f"data:audio/wav;base64,{b64}"}}
|
|
]}],
|
|
max_tokens=4096
|
|
)
|
|
```
|
|
|
|
## Phase 4b: 字级时间戳 (Qwen3 FileTrans API)
|
|
|
|
```python
|
|
# 1. 上传
|
|
upload = dashscope.Files.upload(file_path="audio.wav", purpose="file-extract")
|
|
url = dashscope.Files.get(file_id=upload.output["uploaded_files"][0]["file_id"]).output["url"]
|
|
|
|
# 2. 提交任务
|
|
resp = requests.post("https://dashscope.aliyuncs.com/api/v1/services/audio/asr/transcription",
|
|
headers={"Authorization": f"Bearer {API_KEY}", "X-DashScope-Async": "enable", "Content-Type": "application/json"},
|
|
json={"model": "qwen3-asr-flash-filetrans", "input": {"file_url": url},
|
|
"parameters": {"enable_words": True, "language": "zh"}})
|
|
|
|
# 3. 获取结果
|
|
result = requests.get(f"https://dashscope.aliyuncs.com/api/v1/tasks/{task_id}", ...)
|
|
trans = requests.get(result["output"]["result"]["transcription_url"]).json()
|
|
```
|
|
|
|
## 模型选择
|
|
|
|
| 需求 | 模型 |
|
|
|------|------|
|
|
| 分镜场景 | MiMo 2.5 |
|
|
| 转录文本 | Qwen3-ASR-Flash (chat) |
|
|
| 字级时间戳 | Qwen3-ASR-Flash (FileTrans) |
|
|
| 方言/说话人 | Fun-ASR |
|
|
| 离线私有化 | Paraformer-zh |
|