Skip to content

feat(posthog-integration): detect existing PostHog, surface the other tricks - #1066

Open
joshtronic wants to merge 7 commits into
PostHog:mainfrom
joshtronic:feat/detect-existing-setup
Open

feat(posthog-integration): detect existing PostHog, surface the other tricks#1066
joshtronic wants to merge 7 commits into
PostHog:mainfrom
joshtronic:feat/detect-existing-setup

Conversation

@joshtronic

Copy link
Copy Markdown

Problem

If you run the wizard on a project that already has PostHog you need to walk through the full integration flow again. The original issue quotes a user who bailed out of fear it would overwrite their setup. I also ran into this issue and fortunately, in my testing re-runs mostly don't do anything (destructive or otherwise). No-op still takes ~8 minutes due to auth, skill installation, and running of the agent. Adjacent problem is that the additional wizard tools (or "tricks") aren't made apparent to the user, first run or subsequent runs.

Part of #616.

Changes

  • Detection step scans package.json files for PostHog SDK using the existing findPackageJsons() (same helper revenue-analytics detection uses, monorepo-aware) and sets session.posthogSdkDetected
  • Intro view shows PostHog: detected in package.json under the framework line. Primary action (when detected) reads "Continue anyway" (borrowed from the health-check view) and still routes into the normal (full) flow.
  • New "More wizard tricks" menu item opens a registry-driven list of the subcommands, their descriptions, and how to run them
  • posthog_sdk_detected added to the standard session properties so existing events can be filtered by it

Regarding the weak signal: the notes on the issue call out that a dependency isn't proof of a working install (agreed 💯) which is why nothing gates or routes on this. A false positive costs a slightly more cautious button label. Init and/or env-key checks or post-auth server-side signals can tighten it further before anything ever routes on the flag.

On the proposed menu, the issue's UX launches the selected command, which needs an in-session program handoff that doesn't exist today. That seemed like maybe something that would be included in the v3 wizard's navigation. This ships the groundwork, but the telemetry already baked to track view changes will give a good signal on when to iterate down the road. I also left the intro's tagline untouched primarily because I couldn't come up with anything that had enough "WOW" as the "2 hours in 8 minutes" line (srsly, so good).

image image

Test plan

  • Contract test, written red-first, for detectExistingPostHog. Covers 6 cases: no package.json, dependencies without PostHog SDK, dependencies, devDependencies, nested monorepo packages, invalid installation directory (shouldn't throw)
  • Store setter test alongside the existing session-setter tests with a sessionProperties test pinning the new property
  • Running pnpm build && pnpm test && pnpm fix all came back clean
  • Live runs against a few projects via pnpm try. Projects that were already set up showed the new row, relabel, and tricks view (screenshots above). Clean project showed identical to upstream's main. Nothing gates on the flag, so --ci/headless usage is unaffected

LLM context

Friendly robots, specifically Claude Code, were utilized to help me better understand an unfamiliar codebase. Tests were generated (with my direction) with attribution on those commits. Code changes were guided, but handcrafted to help knock some ring rust off (mostly, as I think Claude committed one of my changes when I wasn't looking 👀 ). All verification runs were done by a human. Robots also played janitor and helped me identify typos.

@joshtronic
joshtronic requested a review from a team as a code owner August 7, 2026 02:43
@joshtronic joshtronic changed the title Feat/detect existing setup feat(posthog-integration): detect existing PostHog, surface the other tricks Aug 7, 2026
@gewenyu99

Copy link
Copy Markdown
Collaborator

Hey, this is incredible! I'm gonna nudge you with some more context:

  • Deterministic detection is what whacking infinite moles with a finite number of sticks feel like. We believe a haiku/GPT luna agent can do this better 10/10 times for cheaper. We actually already have this code around and are soon swapping over to it. This is a great idea, and I think we should roll this in with that bit of detection code.
  • Another migration in flight is the orchestration model for the wizard. If you look carefully at live runs today, we actually enqueue tasks dynamically, albeit for now a fixed list. There's probably more magic to be had this way, if you'd like to play around, go for it! If not, I'd be happy to take this contribution forward once you're happy with the detection related changes.

See if this gives you more ideas ;) Apologies for the highly out of dates docs around these things. They're changing like every other week.

joshtronic and others added 6 commits August 10, 2026 14:58
Red-first tests for detectExistingPosthog, ahead of the implementation:
setter always fires (including the negative case), both dependency
sections count, nested monorepo packages are found, and an invalid
install dir reports false instead of throwing.

Co-Authored-By: Claude <noreply@anthropic.com>
Scan found `package.json` files for a declared PostHog SDK and store the
verdict in the session as `posthogSdkDetected`. Dependency-level signal
only, not a verified (or complete) install. Best-effort: scan failures
report false and never block detection.

Pre-work for PostHog#616. Nothing reads the flag yet. Wiring up the intro
screen is next up.

Co-Authored-By: Claude <noreply@anthropic.com>
When the detection step finds an existing PostHog SDK, the intro shows
`PostHog: detected in package.json` row, and the primary action reads
`Continue anyway` (similar phrasing as the health check screen).
Continue still routes into the normal flow as this is a disclosure and
not an actual gate.

Part of PostHog#616
If an existing PostHog SDK is detected (via `package.json`), there's a
good opportunity to raise awareness to the other wizard commands (or
"tricks"). New menu option lists out every subcommand: nam, description,
and how to invoke it directly later. No launching of the subcommands as
I wasn't sure how much of this was being addressed in the v3 TUI
improvements already. The list reads from `getSubcommandPrograms()` so
future additions will appear automagically.

Part of PostHog#616, addressing the discoverability concern in the notes.
Red-first: sessionProperties() should carry the detect step's verdict so
every capture that spreads it becomes sliceable by whether the run
started on an already-instrumented project.

Co-Authored-By: Claude <noreply@anthropic.com>
Add the detect step's verdict to `sessionProperties()`. Any event that
includes the standard session properties (setup confirmed, outro,
aborts) can be filtered by whether the run started on an
already-integrated project. Answers how often PostHog#616 actually happens
without introducing a new event.
@joshtronic
joshtronic force-pushed the feat/detect-existing-setup branch from 245925a to c050ebe Compare August 10, 2026 19:58
Prep for the agentic detection swap mentioned in the PR review. The flag
comment can now outlive the current mechanism of detection. Also fixed a
small typo.
@joshtronic

Copy link
Copy Markdown
Author

Hey @gewenyu99, thanks for the additional context!

Totally tracks on the deterministic detection being mole-whacking nightmare fuel. The manifest scan was always meant as a cheap first signal, with the final answer being the agentic plumbing that sounds in flight on your end.

Fortunately, the detection and disclosure are decoupled already. detectExistingPostHog is the only thing that knows how the verdict happened, as it just sets posthogSdkDetected and everything downstream works off the flag. Your agentic detection can set the same flag and the rest should carry over as-is.

I just pushed a small change so the flag's comment describes the outcome rather than the mechanism. One less thing to change when the agentic detection lands.

I'm happy with this PR as-is. You can consider the detector disposable and the rest is yours to take forward whenever. I may poke at the dynamic task enqueuing this week out of curiosity, but don't let me block anything.

Appreciate you taking the time to put eyes on this PR. 🙇

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants