Video Perception

Drives an ffmpeg MCP server so Claude reads video frames and timestamped transcripts

Works with setup

Test report

Verdict
Works with setup
Score
6.8/10
Tested
Jul 21, 2026
Environment
Claude Code 2.x (agent harness)
Upstream re-checked
Aug 10, 2026 · 6eded2f

I installed claude-video-vision@1.3.2 from npm into a sandboxed HOME, drove the MCP server over stdio JSON-RPC, and confirmed all six tools respond, then generated a 3-minute test video with four labelled chapters and a 1-second "ERROR CODE 7742" flash at 01:37. BASELINE (naive video_watch, fps auto): 100 frames / 487KB of images; it caught the flash at 00:01:37 but silently hit max_frames=100 and never extracted the final 41 seconds of the video. SKILL run (video_info then video_analyze then a planned segment extraction): video_analyze returned the exact cut list 00:00:45, 00:01:30, 00:01:37, 00:01:38, 00:02:15 plus the 00:00:45-00:01:30 silence interval for zero image tokens — including the 02:15 cut baseline never saw — but the prescribed segment extraction returned only 34KB of frames labelled 00:00:10, 00:01:48, 00:01:41, 00:02:21, 00:02:34 and 00:03:54 (past the 3:00 duration) and missed the error flash entirely. Root cause I read in dist/extractors/frames.js: -ss is passed before -i while -to is passed after, so every window request runs to EOF; I reproduced it on video_watch segments, video_watch start_time/end_time, and video_detail in a clean HOME. Transcription also returns an empty array with no warning when no whisper/API backend is configured.

Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works

  • Installs cleanly 4/5
  • Triggers reliably 5/5
  • Output vs. baseline 5/10
  • Docs & honesty 3/5

What Video Perception does

Wires Claude Code into the claude-video-vision MCP server so it can read local video files (.mp4, .mov, .mkv, .webm) and YouTube URLs as image frames plus timestamped audio transcription. The skill prescribes a workflow: fetch metadata, run ffmpeg structural analysis (scene changes, silence intervals, motion) before extracting any frames, then pull frames at adaptive FPS around the moments that matter. Triggers when a video file path, a YouTube link, or a request to watch, review, or summarize a video appears in the conversation.

How to install Video Perception

git clone --depth 1 https://github.com/jordanrendric/claude-video-vision.git /tmp/video-perception-src
mkdir -p ~/.claude/skills
cp -R /tmp/video-perception-src/skills/video-perception ~/.claude/skills/video-perception
# The skill is only guidance — it is USELESS without the MCP server that provides
# video_info / video_analyze / video_watch / video_detail / video_configure / video_setup.
#
# Register the MCP server (needs Node 20+ and ffmpeg on PATH):
#   claude mcp add claude-video-vision -- npx -y claude-video-vision@latest
#
# Or install the whole plugin instead of this skill, inside Claude Code, one at a time:
#   /plugin marketplace add https://github.com/jordanrendric/claude-video-vision
#   /plugin install claude-video-vision
#   /setup-video-vision
#
# Required deps:
#   brew install ffmpeg            # frame extraction (mandatory)
#   brew install yt-dlp            # only for YouTube URLs
# Audio transcription needs a backend, otherwise transcription silently returns []:
#   export GEMINI_API_KEY=...      # free tier, backend "gemini-api"
#   brew install whisper-cpp       # fully local, backend "local"
#   export OPENAI_API_KEY=...      # paid, backend "openai"
# Config lives at ~/.claude-video-vision/config.json (set via the video_configure tool).
#
# Known defect in the published v1.3.2 server: any time-bounded request over-extracts.
# ffmpeg is called with -ss before -i but -to after -i, so asking for 01:36-01:39
# returns ~100 frames spanning 01:36-02:25. Prefer whole-video video_watch and lean on
# video_analyze for timestamps until this is fixed.

Skills live in ~/.claude/skills/ (global) or .claude/skills/ (per-project). Restart Claude Code after installing.

Commands — how to trigger Video Perception

  • /video-perception Drives an ffmpeg MCP server so Claude reads video frames and timestamped transcripts

It also activates on plain-language prompts like these:

  • Watch this video and summarize what happens
  • Analyze this YouTube link for key moments
  • Review this MP4 and describe the scenes

Frequently asked questions

Is the Video Perception skill free?
Yes. The skill itself is free from jordanrendric/claude-video-vision. SkillProof publishes the install command and an independent test verdict at no cost.
Does Video Perception work with Claude Code?
We tested it with Claude Code 2.x (agent harness) on Jul 21, 2026. Verdict: Works with setup. I installed claude-video-vision@1.3.2 from npm into a sandboxed HOME, drove the MCP server over stdio JSON-RPC, and confirmed all six tools respond, then generated a 3-minute test video with four labelled chapters and a 1-second "ERROR CODE 7742" flash at 01:37. BASELINE (naive video_watch, fps auto): 100 frames / 487KB of images; it caught the flash at 00:01:37 but silently hit max_frames=100 and never extracted the final 41 seconds of the video. SKILL run (video_info then video_analyze then a planned segment extraction): video_analyze returned the exact cut list 00:00:45, 00:01:30, 00:01:37, 00:01:38, 00:02:15 plus the 00:00:45-00:01:30 silence interval for zero image tokens — including the 02:15 cut baseline never saw — but the prescribed segment extraction returned only 34KB of frames labelled 00:00:10, 00:01:48, 00:01:41, 00:02:21, 00:02:34 and 00:03:54 (past the 3:00 duration) and missed the error flash entirely. Root cause I read in dist/extractors/frames.js: -ss is passed before -i while -to is passed after, so every window request runs to EOF; I reproduced it on video_watch segments, video_watch start_time/end_time, and video_detail in a clean HOME. Transcription also returns an empty array with no warning when no whisper/API backend is configured.
What is the Video Perception SkillProof Score?
6.8/10 — installs cleanly 4/5, triggers reliably 5/5, output vs. baseline 5/10, docs & honesty 3/5.
How do I install Video Perception?
Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
Can I use Video Perception with Cursor, Copilot, Gemini CLI, Codex or other AI tools?
The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.