feat(agent-core): guide AI to use ReadMediaFile for video analysis instead of manual frame extraction

.changeset/video-readmediafile-prompt.md

@@ -0,0 +1,7 @@
+---
+"@moonshot-ai/agent-core": patch
+---
+
+feat(agent-core): guide AI to use ReadMediaFile for video analysis instead of manual frame extraction
+
+Adds explicit guidance in the system prompt and ReadMediaFile tool description to prefer the `ReadMediaFile` tool over writing Python/ffmpeg scripts when analyzing video content. This prevents inefficient manual frame extraction and leverages built-in multimodal capabilities.

packages/agent-core/src/profile/default/system.md

@@ -58,7 +58,7 @@ The user may ask you to research on certain topics, process or generate certain
 - Understand the user's requirements thoroughly, ask for clarification before you start if needed.
 - Make plans before doing deep or wide research, to ensure you are always on track.
 - Search on the Internet if possible, with carefully-designed search queries to improve efficiency and accuracy.
-- Use proper tools or shell commands or Python packages to process or generate images, videos, PDFs, docs, spreadsheets, presentations, or other multimedia files. Detect if there are already such tools in the environment. If you have to install third-party tools/packages, you MUST ensure that they are installed in a virtual/isolated environment.
+- Use proper tools or shell commands or Python packages to process or generate images, videos, PDFs, docs, spreadsheets, presentations, or other multimedia files. **When the user provides a video file for analysis, prefer using the `ReadMediaFile` tool to read it directly rather than writing Python scripts or ffmpeg commands to extract frames manually.** Detect if there are already such tools in the environment. If you have to install third-party tools/packages, you MUST ensure that they are installed in a virtual/isolated environment.
 - Once you generate or edit any images, videos or other media files, try to read it again before proceed, to ensure that the content is as expected.
 - Avoid installing or deleting anything to/from outside of the current working directory. If you have to do so, ask the user for confirmation.
 

packages/agent-core/src/tools/builtin/file/read-media.md

@@ -7,6 +7,7 @@ Read media content from a file.
 - If automatic compression cannot safely produce an image within model limits, the tool returns an error and does not send the original image. Follow the error: use Bash or an available image-processing tool to create a smaller copy, then read that copy. Do not retry the unchanged file.
 - The system will notify you when there is anything wrong when reading the file.
 - This tool is a tool that you typically want to use in parallel. Always read multiple files in one response when possible.
+- For video files, this is the preferred way to analyze video content. Do not write Python scripts or ffmpeg commands to extract frames manually — use this tool to read the video directly.
 - This tool can only read image or video files. To read text files, use the Read tool. To list directories, use `ls` via Bash for a known directory, or Glob for pattern search.
 - If the file doesn't exist or path is invalid, an error will be returned.
 - The maximum size that can be read is {{ MAX_MEDIA_MEGABYTES }}MB. An error will be returned if the file is larger than this limit.