Skip to content

feat(pptx): bounded OOXML semantic ingestion with media and cache safety #2136

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
skilfoy wants to merge 1 commit into
base: v8
Choose a base branch
from
Open

feat(pptx): bounded OOXML semantic ingestion with media and cache safety #2136

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,7 +337,7 @@ To remove graphify from all platforms at once: `graphify uninstall` (add `--purg
| MCP configs | `.mcp.json` `mcp.json` `mcp_servers.json` `claude_desktop_config.json` — extracts server nodes, package refs, env var requirements |
| Package manifests | `apm.yml` `pyproject.toml` `go.mod` `pom.xml` — one canonical package node per package (by name) plus `depends_on` edges, so a package referenced from many manifests is a single hub |
| Docs | `.md .mdx .qmd .html .txt .rst .yaml .yml` (markdown `[text](./other.md)` links and `[[wikilinks]]` become `references` edges between docs) |
| Office | `.docx .xlsx` (requires `uv tool install graphifyy[office]`) |
| Office | `.pptx` (built in); `.docx .xlsx` (require `uv tool install graphifyy[office]`) |
| Google Workspace | `.gdoc .gsheet .gslides` (opt-in; requires `gws` auth and `--google-workspace`; Sheets need `uv tool install graphifyy[google]`) |
| PDFs | `.pdf` |
| Images | `.png .jpg .webp .gif` |
Expand All @@ -346,6 +346,14 @@ To remove graphify from all platforms at once: `graphify uninstall` (add `--purg

Code is extracted **locally with no API calls** (AST via tree-sitter). Everything else goes through your AI assistant's model API.

PowerPoint extraction preserves slide order, sections, hidden-slide state, text
and bullets, speaker notes, comments, tables, chart data, SmartArt/native diagram
relationships, hyperlinks, alt text, layout/master evidence, custom XML, and
embedded package objects. Embedded images are sent through the normal vision
path. Embedded audio and video are extracted with slide provenance and transcribed
locally when `graphifyy[video]` is installed; external media links are recorded but
never fetched automatically.

Google Drive for desktop `.gdoc`, `.gsheet`, and `.gslides` files are shortcut
pointers, not document content. To include native Google Docs, Sheets, and Slides
in a headless extraction, install and authenticate the
Expand Down
7 changes: 5 additions & 2 deletions docs/how-it-works.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,11 @@ Video and audio files are transcribed with faster-whisper. To focus the transcri
Claude runs in parallel over markdown, PDFs, images, and transcripts. Each subagent reads a batch of files and outputs a JSON fragment: nodes, edges, and any group relationships. The fragments are merged into a single graph.

Before Pass 3, optional converters turn supported pointer/binary formats into
Markdown sidecars under `graphify-out/converted/`. Office files (`.docx`,
`.xlsx`) use the `[office]` extra. Google Workspace shortcuts (`.gdoc`,
Markdown sidecars under `graphify-out/converted/`. PowerPoint (`.pptx`) uses a
built-in bounded OOXML reader that also emits embedded images and media into the
vision/transcription passes; embedded audio/video needs the `[video]` extra.
Other Office files (`.docx`, `.xlsx`) use the `[office]` extra. Google
Workspace shortcuts (`.gdoc`,
`.gsheet`, `.gslides`) are opt-in with `--google-workspace` or
`GRAPHIFY_GOOGLE_WORKSPACE=1` and require an authenticated `gws` CLI.

Expand Down
1 change: 1 addition & 0 deletions graphify/__main__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -607,6 +607,7 @@ def _run_cli() -> None:
print(" --token-budget N per-chunk token cap for semantic extraction (default: 60000)")
print(" --max-concurrency N parallel semantic chunks in flight (default: 4; set 1 for local LLMs)")
print(" --api-timeout S per-request timeout in seconds for the LLM client (default: 600)")
print(" --whisper-model M local faster-whisper model for video/audio (default: base)")
print(" --out DIR, --output DIR output dir (default: <path>); writes <DIR>/graphify-out/")
print(" --google-workspace export .gdoc/.gsheet/.gslides shortcuts via gws before extraction")
print(" --no-gitignore ignore .gitignore and .git/info/exclude (prioritizes .graphifyignore)")
Expand Down
85 changes: 78 additions & 7 deletions graphify/cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2464,7 +2464,7 @@ def _to_simple(g: "_nx.Graph") -> "_nx.Graph":
"[--model M] [--mode deep] [--out DIR|--output DIR] [--google-workspace] [--no-cluster] "
"[--no-gitignore] [--code-only] "
"[--max-workers N] [--token-budget N] [--max-concurrency N] "
"[--api-timeout S] [--postgres DSN] [--cargo] [--allow-partial] [--timing]",
"[--api-timeout S] [--whisper-model M] [--postgres DSN] [--cargo] [--allow-partial] [--timing]",
file=sys.stderr,
)
sys.exit(1)
Expand Down Expand Up @@ -2498,6 +2498,7 @@ def _to_simple(g: "_nx.Graph") -> "_nx.Graph":
cli_token_budget: int | None = None
cli_max_concurrency: int | None = None
cli_api_timeout: float | None = None
cli_whisper_model: str | None = None
# Clustering tuning knobs
cli_resolution: float = 1.0
cli_exclude_hubs: float | None = None
Expand Down Expand Up @@ -2582,6 +2583,10 @@ def _parse_float(name: str, raw: str) -> float:
cli_api_timeout = _parse_float("--api-timeout", args[i + 1]); i += 2
elif a.startswith("--api-timeout="):
cli_api_timeout = _parse_float("--api-timeout", a.split("=", 1)[1]); i += 1
elif a == "--whisper-model" and i + 1 < len(args):
cli_whisper_model = args[i + 1]; i += 2
elif a.startswith("--whisper-model="):
cli_whisper_model = a.split("=", 1)[1]; i += 1
elif a == "--resolution" and i + 1 < len(args):
cli_resolution = _parse_float("--resolution", args[i + 1]); i += 2
elif a.startswith("--resolution="):
Expand Down Expand Up @@ -2638,6 +2643,10 @@ def _parse_float(name: str, raw: str) -> float:
# the skill.md pipeline.
out_root = (out_dir.resolve() if out_dir else target)
graphify_out = out_root / _GRAPHIFY_OUT
if cli_whisper_model:
# Set before incremental detection: transcript cache identity includes
# the model, so liveness must be computed under the requested model.
os.environ["GRAPHIFY_WHISPER_MODEL"] = cli_whisper_model
graphify_out.mkdir(parents=True, exist_ok=True)
# Persist corpus-shaping options so later update/watch/hook rebuilds
# use the same file set as the initial extraction (#1886).
Expand Down Expand Up @@ -2698,6 +2707,7 @@ def _parse_float(name: str, raw: str) -> float:
doc_files = []
paper_files = []
image_files = []
video_files = []
deleted_files = []
excluded_files = []
graph_stale_sources = []
Expand All @@ -2711,13 +2721,15 @@ def _parse_float(name: str, raw: str) -> float:
google_workspace=google_workspace or None,
extra_excludes=_effective_excludes or None,
gitignore=_effective_gitignore,
transcript_root=graphify_out / "transcripts",
)
files_by_type = detection.get("files", {})
new_by_type = detection.get("new_files", {})
code_files = [Path(p) for p in new_by_type.get("code", [])]
doc_files = [Path(p) for p in new_by_type.get("document", [])]
paper_files = [Path(p) for p in new_by_type.get("paper", [])]
image_files = [Path(p) for p in new_by_type.get("image", [])]
video_files = [Path(p) for p in new_by_type.get("video", [])]
deleted_files = list(detection.get("deleted_files", []))
excluded_files = list(detection.get("excluded_files", []))
unchanged_total = sum(len(v) for v in detection.get("unchanged_files", {}).values())
Expand Down Expand Up @@ -2745,26 +2757,55 @@ def _parse_float(name: str, raw: str) -> float:
doc_files = [Path(p) for p in files_by_type.get("document", [])]
paper_files = [Path(p) for p in files_by_type.get("paper", [])]
image_files = [Path(p) for p in files_by_type.get("image", [])]
video_files = [Path(p) for p in files_by_type.get("video", [])]
deleted_files = []
excluded_files = []
graph_stale_sources = []
unchanged_total = 0

if deep_mode and incremental_mode and not code_only:
# Deep mode widens all semantic sources below, so transcribe all live
# media and let Whisper's derivative cache decide whether work is due.
video_files = [Path(p) for p in files_by_type.get("video", [])]

transcript_files: list[Path] = []
media_files_due = list(video_files)
if not code_only and video_files:
from graphify.transcribe import transcribe_all

transcript_files = transcribe_all(
video_files,
output_dir=graphify_out / "transcripts",
force=force,
)
doc_files.extend(transcript_files)
files_by_type.setdefault("document", []).extend(
str(path) for path in transcript_files
)
if len(transcript_files) != len(video_files):
print(
f"[graphify extract] warning: transcribed {len(transcript_files)} of "
f"{len(video_files)} video/audio file(s); install graphifyy[video] "
"and check media format errors for full semantic extraction",
file=sys.stderr,
)

semantic_files = doc_files + paper_files + image_files
# --code-only: index code (pure local AST, no key) and skip the semantic
# (doc/paper/image) pass entirely, so a mixed repo doesn't hard-fail when no
# LLM backend is configured (#1734). Report what was skipped rather than
# silently dropping it.
if code_only and semantic_files:
if code_only and (semantic_files or video_files):
print(
f"[graphify extract] --code-only: skipping {len(semantic_files)} "
f"[graphify extract] --code-only: skipping {len(semantic_files) + len(video_files)} "
f"non-code file(s) ({len(doc_files)} docs, {len(paper_files)} papers, "
f"{len(image_files)} images) — no LLM extraction"
f"{len(image_files)} images, {len(video_files)} video/audio) — no LLM extraction"
)
semantic_files = []
doc_files = []
paper_files = []
image_files = []
video_files = []
if deep_mode and incremental_mode and not code_only:
# Deep mode reads/writes its own cache namespace
# (cache/semantic-deep/), so the manifest's changed-file gate is
Expand Down Expand Up @@ -2796,15 +2837,16 @@ def _parse_float(name: str, raw: str) -> float:
_excl_note = f"; {len(excluded_files)} excluded" if excluded_files else ""
print(
f"[graphify extract] {len(code_files)} code, {len(doc_files)} docs, "
f"{len(paper_files)} papers, {len(image_files)} images changed; "
f"{len(paper_files)} papers, {len(image_files)} images, "
f"{len(video_files)} video/audio changed; "
f"{unchanged_total} unchanged; {len(deleted_files)} deleted"
f"{_excl_note}"
)
else:
print(
f"[graphify extract] found {len(code_files)} code, "
f"{len(doc_files)} docs, {len(paper_files)} papers, "
f"{len(image_files)} images"
f"{len(image_files)} images, {len(video_files)} video/audio"
)
# Surface files that were seen but not classified (extensionless non-shebang
# project files like Dockerfile/Makefile, or unsupported extensions), so they
Expand Down Expand Up @@ -3181,6 +3223,33 @@ def _progress(idx: int, total: int, _result: dict) -> None:
# absolute file lists.
_manifest_files = _stamped_manifest_files(files_by_type, sem_result, target,
partial_source_files=_partial_semantic_files)
# A media file is semantically complete only when its exact content/
# model/prompt-addressed transcript produced semantic output. Merely
# attempting transcription must not stamp corrupt media or a missing
# faster-whisper installation as successful forever.
_stamped_documents = set(_manifest_files.get("document", []))
_failed_media: set[str] = set()
if media_files_due:
from graphify.transcribe import transcript_path_for as _transcript_path_for

for _media in media_files_due:
try:
_expected_transcript = str(
_transcript_path_for(
_media,
output_dir=graphify_out / "transcripts",
)
)
except OSError:
_expected_transcript = ""
if _expected_transcript not in _stamped_documents:
_failed_media.add(str(_media))
if _failed_media:
_manifest_files["video"] = [
path
for path in _manifest_files.get("video", [])
if str(path) not in _failed_media
]

# Files dispatched this run but dropped by _stamped_manifest_files
# above (failed chunk, LLM omission, or any future exclusion) still
Expand All @@ -3196,7 +3265,9 @@ def _progress(idx: int, total: int, _result: dict) -> None:
_stamped_semantic = {
f for _flist in _manifest_files.values() for f in _flist
}
_cleared_semantic = {str(p) for p in semantic_files} - _stamped_semantic
_cleared_semantic = (
{str(p) for p in semantic_files} - _stamped_semantic
) | _failed_media

# Full-scan manifest saves prune rows for in-root files that left the
# scan corpus but still exist on disk (#1908). The corpus must be the
Expand Down
Loading