feat(analytics): gate Web Analytics behind a webAnalytics rollout flag - #386
Conversation
Web Analytics shipped visible to every organization. Put it behind the existing per-org rollout mechanism instead: `OrganizationFeatureFlags`, decoded from Clerk organization public metadata, which already carried `aiAutoTriage`. Enable per org by setting `webanalytics: true` in that org's Clerk public metadata (keys are lowercase-squashed there, camelCase in code — the schema's `encodeKeys` owns the mapping). All three entry points are gated, because closing fewer than all of them does not hide anything: - the sidebar row — `navGroups(flags)`, filtered - ⌘K — `paletteNavItems(flags)`, which derives from `navGroups`, so a flagged-off page is not findable by typing its name either - the route — `/analytics` renders the router's own `NotFoundError`, so an unflagged org sees exactly what a nonexistent route would give it The fourth was a gap in the previous commit: the Analytics button in the Session Replays header was still rendered, so an unflagged org could click straight into "Page not found". It is gated on the same flag now. Two supporting changes. `useOrganizationFeatureFlags` is new and is the only way consumers read flags, so the two rules that make a flag safe live in one place: fail closed while Clerk metadata loads (a surface that flashes in and then disappears is worse than one that arrives late), and fail open when `!isClerkAuthEnabled`, since a self-hosted deployment has no Clerk to read and would otherwise have flagged features hidden permanently. `settings-nav` moves onto that hook, replacing its inline decode — it had already made the same self-hosted call for `aiAutoTriage`, and this stops the two from drifting. Client-side only, matching `aiAutoTriage`: the API does not read Clerk org metadata, so this hides the surface rather than protecting data. The data was never exposed — every warehouse query is org-scoped through `CurrentTenant`.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| export function useOrganizationFeatureFlags(): OrganizationFeatureFlags { | ||
| const { organization } = useOrganization() | ||
| if (!isClerkAuthEnabled) return ENABLED_ORGANIZATION_FEATURE_FLAGS | ||
| return organizationFeatureFlagsFrom(organization?.publicMetadata) | ||
| } |
There was a problem hiding this comment.
🔴 Self-hosted deployments crash on every dashboard page
The organization's rollout settings are looked up through Clerk (useOrganization() at apps/web/src/hooks/use-organization-feature-flags.ts:30) before the code checks whether Clerk is even in use, so self-hosted installs — which never load Clerk — hit an error instead of the sidebar.
Impact: Anyone running Maple themselves sees a blank/error screen on the main dashboard, command palette, replays and analytics pages.
Clerk hooks assert a surrounding provider; self-hosted builds render without one
apps/web/src/main.tsx:178-194 mounts ClerkProvider only when isClerkAuthEnabled; the self-hosted branch renders SelfHostedInnerApp with no provider. Clerk v5 hooks (@clerk/clerk-react ^5.61.3) assert they are wrapped by ClerkProvider and throw otherwise — which is exactly why the codebase gates every other call at a hook-free boundary: apps/web/src/components/dashboard/org-switcher.tsx:45-52 picks ClerkOrgSwitcher vs SelfHostedOrgSwitcher before any hook runs, and apps/web/src/routes/select-plan.tsx:27-33 documents the same pattern ("Clerk hooks below require ClerkProvider, which is absent when auth is disabled (self-hosted). Gate at this hook-free boundary").
The new hook inverts that order: useOrganization() runs first and the !isClerkAuthEnabled early return comes after, so the throw happens before the self-hosted fallback can be returned. The hook is now called from apps/web/src/components/dashboard/app-sidebar.tsx:542 (rendered on every dashboard page), the command palette, apps/web/src/routes/replays/index.tsx:128 and apps/web/src/routes/analytics/index.tsx:70.
The fix is to keep the Clerk read in a component/hook that only runs in Clerk mode, or to branch on isClerkAuthEnabled before touching useOrganization (e.g. split into useClerkOrganizationFeatureFlags used behind a gated boundary).
Prompt for agents
useOrganizationFeatureFlags calls useOrganization() unconditionally and only afterwards checks isClerkAuthEnabled. In self-hosted builds apps/web/src/main.tsx renders the app without ClerkProvider, and Clerk v5 hooks throw when no provider is present (see the existing gating patterns in apps/web/src/components/dashboard/org-switcher.tsx and apps/web/src/routes/select-plan.tsx, which explicitly branch at a hook-free boundary). Because the new hook is now used by AppSidebar, the command palette, the replays route and the analytics route, self-hosted deployments would throw on essentially every dashboard page. Restructure so the Clerk hook is only evaluated when isClerkAuthEnabled is true — e.g. keep a Clerk-only inner hook/component and have consumers pick between it and the fixed ENABLED_ORGANIZATION_FEATURE_FLAGS value, or otherwise avoid invoking useOrganization in the self-hosted branch.
Was this helpful? React with 👍 or 👎 to provide feedback.
| const featureFlags = useOrganizationFeatureFlags() | ||
| if (!featureFlags.webAnalytics) return <NotFoundError /> | ||
|
|
||
| return <WebAnalyticsPageContent /> |
There was a problem hiding this comment.
🟡 Analytics page briefly shows "Page not found" for organizations that have the feature enabled
The analytics page decides the feature is off while the organization's settings are still being fetched (if (!featureFlags.webAnalytics) at apps/web/src/routes/analytics/index.tsx:71), so an entitled organization sees a "Page not found" screen before the real page appears.
Impact: Users with Web Analytics enabled get a confusing not-found flash on every fresh load or reload of the page.
Loading state is indistinguishable from "flag off"
useOrganizationFeatureFlags (apps/web/src/hooks/use-organization-feature-flags.ts:29-33) returns all flags false while useOrganization() has not resolved (organization is undefined). Failing closed is fine for hiding a sidebar row (nothing is rendered), but on the route itself it renders a definite negative answer — NotFoundError — during the loading window, then swaps to the real page once Clerk resolves. Using Clerk's isLoaded (or the organization-loaded state) to render a neutral loading state before deciding would avoid the flash.
Prompt for agents
WebAnalyticsPage in apps/web/src/routes/analytics/index.tsx renders NotFoundError as soon as featureFlags.webAnalytics is false, but useOrganizationFeatureFlags returns all-false while Clerk's organization data is still loading. For an org that actually has the flag on, this shows 'Page not found' first and then the real page. Consider exposing a loading/isLoaded signal from useOrganizationFeatureFlags (Clerk's useOrganization provides isLoaded) and rendering a skeleton/boot state until the flags are known, only rendering NotFoundError once the flags have actually resolved to disabled.
Was this helpful? React with 👍 or 👎 to provide feedback.
…t flash not-found
Both issues Devin flagged on the flag-gating commit were real.
**Self-hosted crashed on every dashboard page.** `useOrganizationFeatureFlags`
called `useOrganization()` and only then checked `isClerkAuthEnabled` — too late,
because the hook has already run. `apps/web/src/main.tsx` mounts `ClerkProvider`
only in Clerk mode and Clerk v5 hooks throw without one, which is precisely why
`org-switcher.tsx` and `select-plan.tsx` branch at a hook-free boundary
("Clerk hooks below require ClerkProvider, which is absent when auth is disabled
(self-hosted). Gate at this hook-free boundary"). The new hook broke that rule and
then got called from `AppSidebar`, so the blast radius was every page rather than
one component.
The implementation is now chosen at module scope off the build-time constant: each
variant calls its hooks unconditionally, hook order is fixed for the lifetime of
the bundle, and the self-hosted build never reaches the Clerk variant. This also
makes `settings-nav` safer than it was before this branch — it used to call
`useOrganization()` unconditionally itself.
**An entitled org saw "Page not found" flash before the page.** Flags fail closed
while Clerk resolves, which is right for a nav row (nothing renders either way)
but wrong for a route, where "off" becomes a visible verdict. The state now
carries `isLoaded`, and the route renders nothing until the flags are the org's
real answer instead of asserting not-found during the load window. The sidebar and
chrome come from the layout, so that window is a brief empty content area, not a
blank app.
`isLoaded` is documented on the type rather than at the call site, since every
future consumer that turns a disabled flag into a visible conclusion needs the
same guard.
🍁 Maple PR previewNote Preview resources were removed when this pull request closed. Final commit |
1 participant
Web Analytics shipped visible to every organization. This puts it behind the rollout mechanism that already exists —
OrganizationFeatureFlagsin organization-feature-flags.ts, decoded from Clerk organization public metadata, which already carriedaiAutoTriage.Turning it on
Set
webanalytics: truein the org's Clerk public metadata:{ "webanalytics": true }Keys are lowercase-squashed in Clerk and camelCase in code; the schema's
encodeKeysowns that mapping. Only the literal booleantrueenables — the string"true", which is what hand-editing the Clerk dashboard tends to produce, reads as off (there's a test for it).All four entry points
Closing fewer than all of them doesn't hide anything:
navGroups(flags)filters itpaletteNavItems(flags)— derives fromnavGroups, so it isn't findable by name either/analyticsrenders the router's ownNotFoundError, so an unflagged org sees what a nonexistent route givesSupporting changes
useOrganizationFeatureFlagsis new and is the only way consumers read flags, so the two rules that make a flag safe live in one place rather than being re-derived per call site:useOrganization()returnsundefinedin that window, and a surface that flashes in then disappears is worse than one arriving a beat late.!isClerkAuthEnabled— a self-hosted deployment has no Clerk to read metadata from, and treating that as "all flags off" would hide flagged features from self-hosters permanently.settings-navhad already made this exact call foraiAutoTriage.settings-navmoves onto the hook, replacing its inlineorganizationFeatureFlagsFrom(organization?.publicMetadata), so the two can't drift.Scope
Client-side only, matching
aiAutoTriage. The API does not read Clerk org metadata, so this hides the surface, not the data — and the data was never exposed: every warehouse query is org-scoped throughCurrentTenant. Server-side enforcement doesn't exist foraiAutoTriageeither and would mean adding a field toTenantSchemaplus a Clerk fetch inapps/api; worth doing if you want it, but it's a separate change.Verification
useEffectones in other files)"true"-string case/analyticsrenders "Page not found"; ⌘K search for "analytics" returns nothing; zero/analyticsanchors anywhere in the DOMNeed help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.