Skip to content

feat(ui-test): Playwright ui tests foundation stage 0#2180

Closed
cjlg-soloio wants to merge 13 commits into
kagent-dev:mainfrom
cjlg-soloio:feat(ui)--Playwright-tests-foundation-stage-1
Closed

feat(ui-test): Playwright ui tests foundation stage 0#2180
cjlg-soloio wants to merge 13 commits into
kagent-dev:mainfrom
cjlg-soloio:feat(ui)--Playwright-tests-foundation-stage-1

Conversation

@cjlg-soloio

@cjlg-soloio cjlg-soloio commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Description

Adds the Stage 0 foundation for Playwright page-level E2E testing of the kagent UI, running against a mocked backend with no cluster required.

The suite targets the one gap nothing else covers — multi-step user flows across components:

Tool Covers
Storybook / Chromatic Component render states
Jest Logic
Cypress Page-load smoke
Playwright (this PR) Cross-component user flows

The core design challenge: the UI fetches data server-side via Next.js server actions, so browser-level page.route("**/api/**") intercepts nothing. Instead, this PR boots a dependency-free stub backend and redirects Next's server-side fetch to it via BACKEND_INTERNAL_URL — which getBackendUrl() in ui/src/lib/utils.ts already honors. Playwright's webServer block starts both the stub (:8899) and next dev (:8001) automatically.

This is foundation-only: config, stub backend, shared fixture, CI workflow, and a single smoke test. The helper/driver library and per-test scenarios land in Stage 1; feature flows in Stage 3 (see ui/playwright/README.md).

Files

New

File Purpose
ui/playwright.config.ts Boots stub + dev server; chromium project; CI-aware retries/workers; trace & screenshot on failure
ui/playwright/mocks/server.mjs Dependency-free stub backend (Node http only); serves happy-path payloads mirroring src/types/index.ts; exposes /__mock/health & /__mock/reset
ui/playwright/mocks/data.ts Typed spec-side data builders mirroring the stub's shapes
ui/playwright/fixtures/test.ts Shared test/expect fixture; resets stub per test and bypasses first-run onboarding
ui/playwright/tests/smoke.spec.ts Stage 0 smoke test — asserts the Agents list renders against the mock
ui/playwright/tsconfig.json Extends UI tsconfig with node + Playwright types
ui/playwright/README.md Test strategy, mocking rationale, conventions, roadmap
.github/workflows/ui-playwright.yaml Standalone CI workflow (paths-filtered to ui/**); installs chromium, runs test:pw, uploads report

Modified

  • ui/package.json@playwright/test dev dep + test:pw / test:pw:ui / test:pw:debug scripts
  • ui/package-lock.json — lockfile updates
  • ui/.gitignore — ignores playwright-report, playwright/test-results, playwright/.cache

Testing

cd ui
npm install
npx playwright install chromium   # first time only
npm run test:pw

Playwright starts the stub backend and dev server automatically. The smoke test passes when the home page renders the Agents, no "Error Encountered" state appears, and no uncaught page errors occur. CI runs the same on any ui/** change.

Change type

test / ci — new test infrastructure and CI workflow. No product or runtime code changed.

Changelog

Stage 1 of 3 playwright tests for ui

Copilot AI review requested due to automatic review settings July 8, 2026 16:06
@cjlg-soloio
cjlg-soloio requested a review from peterj as a code owner July 8, 2026 16:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces a foundational Playwright E2E test setup for the UI, including a dependency-free stub backend and CI workflow, to enable end-to-end coverage of multi-step user flows against mocked API responses.

Changes:

  • Added Playwright configuration, fixtures, and an initial smoke spec to validate the app loads against a stubbed backend.
  • Implemented a standalone Node HTTP stub backend plus typed mock builders for specs.
  • Wired Playwright into UI scripts, gitignore, and a dedicated GitHub Actions workflow.

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
ui/playwright/tsconfig.json Adds Playwright-local TS config extending the UI tsconfig.
ui/playwright/tests/smoke.spec.ts Adds initial Stage 0 smoke test asserting Agents page loads without error state/page errors.
ui/playwright/README.md Documents Playwright scope, mocking approach, layout, and conventions.
ui/playwright/mocks/server.mjs Adds dependency-free stub backend serving /api/* responses used by server actions.
ui/playwright/mocks/data.ts Adds typed mock builders for assertions and future scenario plumbing.
ui/playwright/fixtures/test.ts Adds shared fixture to reset stub and bypass onboarding via localStorage init script.
ui/playwright.config.ts Adds Playwright runner config including webServer boot of stub + next dev.
ui/package.json Adds Playwright test scripts and @playwright/test dev dependency.
ui/package-lock.json Locks @playwright/test and updates Playwright packages.
ui/.gitignore Ignores Playwright report/output directories.
.github/workflows/ui-playwright.yaml Adds CI workflow to run Playwright tests on UI changes and upload reports.
Files not reviewed (1)
  • ui/package-lock.json: Generated file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread ui/playwright.config.ts Outdated
Comment thread ui/playwright/mocks/server.mjs Outdated
Comment thread ui/playwright/fixtures/test.ts Outdated
Comment thread .github/workflows/ui-playwright.yaml Outdated
@cjlg-soloio cjlg-soloio changed the title Merge branch 'main' of https://github.com/kagent-dev/kagent into feat(ui)--Playwright-tests-foundation-stage-1 Playwright ui tests foundation stage 0 Jul 8, 2026
@cjlg-soloio cjlg-soloio changed the title Playwright ui tests foundation stage 0 feat(ui-test): Playwright ui tests foundation stage 0 Jul 8, 2026
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Jul 8, 2026
@cjlg-soloio
cjlg-soloio force-pushed the feat(ui)--Playwright-tests-foundation-stage-1 branch from 4bed054 to 4e52362 Compare July 8, 2026 18:42
Comment thread ui/playwright.config.ts
@mesutoezdil

Copy link
Copy Markdown
Contributor

dco is red

@cjlg-soloio
cjlg-soloio force-pushed the feat(ui)--Playwright-tests-foundation-stage-1 branch 2 times, most recently from 08b3cfc to 5d1b617 Compare July 8, 2026 20:09
Comment thread ui/playwright/tsconfig.json Outdated
Comment thread ui/playwright.config.ts
Comment thread ui/playwright.config.ts Outdated
Comment thread ui/playwright/mocks/data.ts Outdated

@Charlesthebird Charlesthebird left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, just a few requests before merging:

  • The package-lock diff due to node version not matching .nvmrc should be resolved.
  • Cypress can be removed here.
  • There should be a playwright video output.

Comment thread .github/workflows/ui-playwright.yaml Outdated
Comment thread .github/workflows/ui-playwright.yaml Outdated
Comment thread ui/playwright/mocks/data.ts
Comment thread ui/playwright/README.md Outdated
cjlg-soloio and others added 6 commits July 13, 2026 15:12
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: cjlg-soloio <carlos.logrono@solo.io>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
- Force single Playwright worker unconditionally (shared stub + Next server)
- Default req.method/req.url before parsing in the stub server
- Fail fast on stub reset failure in CI; stay non-fatal locally
- Drop unused pull-requests:write permission from the workflow

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
Pairs fullyParallel with workers: 1 so both stay off until per-test data
isolation lands, preventing tests from silently racing against the shared
stub backend if the worker count is raised.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
- Pin STUB_PORT on the stub webServer so a shell-set STUB_PORT can't misbind it
- Stop reusing the dev server so BACKEND_INTERNAL_URL always applies (no silent stub bypass)
- Type ToolServerListEntry as the real RemoteMCPServerResponse for compile-time drift detection
- Drop no-op **/*.mjs from the playwright tsconfig include

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio
cjlg-soloio force-pushed the feat(ui)--Playwright-tests-foundation-stage-1 branch from 081fdd1 to a0a9d05 Compare July 13, 2026 19:14
cjlg-soloio and others added 2 commits July 13, 2026 15:48
…ght video

Resolve Charlesthebird's review on PR kagent-dev#2180:
- Playwright workflow now reads node-version-file: ui/.nvmrc instead of a
  hardcoded major version.
- Remove Cypress entirely: delete cypress.config.ts and cypress/e2e/smoke.cy.ts,
  drop the cypress and now-unused start-server-and-test devDeps plus the
  test:e2e/test:e2e:cypress scripts, and remove the dead CYPRESS_INSTALL_BINARY
  env from the ui-playwright, ci, and ui-chromatic workflows and the Dockerfile.
- Enable Playwright video recording (video: "on"); artifacts already upload via
  the existing test-results step.
- De-Cypress the Playwright README (Jest, Vitest, Storybook untouched).

data.ts is intentionally kept — it is consumed by the stacked stage-2 PR.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

@Charlesthebird Charlesthebird left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates!

Comment thread ui/package.json
Comment thread ui/package-lock.json
Restore a conventional test:e2e entrypoint pointing at the Playwright
suite, per review feedback on kagent-dev#2180.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
peterj
peterj previously approved these changes Jul 15, 2026
@peterj

peterj commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio
cjlg-soloio deleted the feat(ui)--Playwright-tests-foundation-stage-1 branch July 15, 2026 14:25
@cjlg-soloio cjlg-soloio reopened this Jul 15, 2026
@cjlg-soloio

Copy link
Copy Markdown
Contributor Author

Superseded by #2262. This PR's branch name contained parentheses (feat(ui)--...) which broke the Docker buildx cache-scope shell interpolation in CI (/bin/sh: Syntax error: "(" unexpected), failing all build/e2e/upgrade jobs. Reopened from a paren-free branch (cjlg-soloio/playwrightTestsStage1) in #2262 with the same changes plus the ui-tests lint fix.

Charlesthebird added a commit that referenced this pull request Jul 16, 2026
### Description



---
*🤖 written by Claude (start)*

Follow-up to a review comment on #2180: pins the npm version via the
`packageManager` field so `npm install` is reproducible and
`ui/package-lock.json` stops picking up metadata churn across npm
versions.

`ui/.nvmrc` pins Node **24.13.0**, which bundles **npm 11.6.2** — so the
field is set to `npm@11.6.2`, consistent with the Node version CI
already reads from `.nvmrc`.

### Files

**Modified**
- `ui/package.json` — adds top-level `"packageManager": "npm@11.6.2"`
- `ui/package-lock.json` — regenerated under npm 11.6.2 to match the
pin; metadata-only diff (`peer`/`optional` flags), no dependency version
changes

### Testing

```
cd ui
corepack enable          # so the pin is enforced locally
npm --version            # -> 11.6.2
npm pkg get packageManager   # -> "npm@11.6.2"
```

### Change type

chore / cleanup — dev tooling only. No product or runtime code changed.

### Changelog

`NONE`

---
*🤖 written by Claude (end)*

---------

Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
Signed-off-by: Nicholas Bucher <behappy54321@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Nicholas Bucher <behappy54321@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants