feat(pglite): pgpm→PGlite plugin — pg-cache + pgsql-client driver seams, @pgpmjs/pglite-adapter, pglite-test

.github/workflows/run-tests.yaml

@@ -94,6 +94,8 @@ jobs:
             packages: 'packages/postmaster packages/smtppostmaster packages/csv-to-pg packages/cli postgres/pgsql-client postgres/pg-ast'
           - batch: graphql
             packages: 'graphql/query graphql/codegen'
+          - batch: pglite
+            packages: 'postgres/pglite-adapter postgres/pglite-test examples/pglite-socket'
 
     steps:
       - name: Download workspace

examples/pglite-socket/CHANGELOG.md

@@ -0,0 +1,4 @@
+# Change Log
+
+All notable changes to this project will be documented in this file.
+See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.

examples/pglite-socket/README.md

@@ -0,0 +1,58 @@
+# pgpm → PGlite
+
+<p align="center" width="100%">
+  <img height="250" src="https://raw.githubusercontent.com/constructive-io/constructive/refs/heads/main/assets/outline-logo.svg" />
+</p>
+
+<p align="center" width="100%">
+  <a href="https://github.com/constructive-io/constructive/actions/workflows/run-tests.yaml">
+    <img height="20" src="https://github.com/constructive-io/constructive/actions/workflows/run-tests.yaml/badge.svg" />
+  </a>
+  <a href="https://github.com/constructive-io/constructive/blob/main/LICENSE">
+    <img height="20" src="https://img.shields.io/badge/license-MIT-blue.svg"/>
+  </a>
+</p>
+
+An end-to-end demonstration that the **unmodified** `@pgpmjs/core` migrate engine can deploy real migrations into [**PGlite**](https://github.com/electric-sql/pglite) — ElectricSQL's WASM build of Postgres ("SQLite for Postgres"): embedded, in-process, with **no Postgres server or service container**.
+
+It proves the wire-protocol path (pgpm ↔ PGlite over a socket). The **in-process driver** path — `@pgpmjs/pglite-adapter` + `pglite-test`, with no socket — is the productized version. Both run in the same no-services `unit-tests (pglite …)` job in [`run-tests.yaml`](../../.github/workflows/run-tests.yaml).
+
+## Run
+
+```sh
+cd examples/pglite-socket
+pnpm test   # jest: boots PGlite, runs pgpm deploy → verify → data → revert, asserts
+```
+
+It's a normal workspace package, so `pnpm install` / `pnpm build` at the repo root wires it up. The test needs **no Postgres service** — PGlite runs in-process.
+
+## How it works
+
+The jest test boots a PGlite instance, exposes it over a socket with the official `pg-gateway` shim (`@electric-sql/pglite-socket`), and points the `@pgpmjs/core` `PgpmMigrate` engine at it as if it were ordinary Postgres:
+
+```
+pgpm (@pgpmjs/core, unmodified)  ──node-pg / TCP──▶  PGLiteSocketServer  ──▶  PGlite (WASM)
+```
+
+It then asserts a full lifecycle against `__fixtures__/` (a 4-change pgpm plan ending in a `pgvector` column):
+
+* **initialize** — bootstraps the `pgpm_migrate` schema (tables + PL/pgSQL procs)
+* **deploy** — `schema → table → index → embedding`
+* **verify** — all four verified
+* **data round-trip** — a pgvector nearest-neighbor query returns the seeded row
+* **revert** — everything reverted, registry emptied
+
+## The two accommodations this pins down
+
+The engine runs against PGlite **today** with two call-site accommodations; the productized in-process driver folds these into the adapter layer:
+
+1. **Single-connection / transaction affinity.** PGlite serializes everything and, while a transaction is open, pins the engine to that one connection. `deploy({ useTransaction: true })` opens `BEGIN` on one pooled connection and calls `isDeployed()` on a *second* — which can never run on PGlite → deadlock. This demo uses `useTransaction: false`; the in-process adapter avoids it entirely because both seams funnel to PGlite's single session. (The socket shim also defaults to `maxConnections: 1`; we raise it — PGlite still serializes, so multiplexing is safe.)
+
+2. **Extensions are provisioned out-of-band.** pgpm's `cleanSql` deliberately strips `CREATE EXTENSION` (and `BEGIN`/`COMMIT`) from migrations — extensions come from the environment (the `postgres-plus` image / `pgsql-test`'s `installExtensions()`), not migrations. So the demo creates the extension at PGlite bootstrap, and PGlite additionally requires the extension registered in JS at construction (`PGlite.create({ extensions: { vector } })`).
+
+Extension availability — not pgpm — is the functional boundary. Bundled/available: `pg_trgm`, `citext`, `ltree`, `uuid_ossp`, `pgvector`, … Not available (no WASM / no background workers): `pg_cron`, `pg_partman`, the BM25 search ext; PostGIS is experimental. A module needing those simply won't deploy to PGlite.
+
+## See also
+
+* [`@pgpmjs/pglite-adapter`](../../postgres/pglite-adapter) — the productized in-process driver (no socket): registers a PGlite instance as the `pg-cache` pool factory.
+* [`pglite-test`](../../postgres/pglite-test) — a drop-in `getConnections()` on in-process PGlite for writing tests.

examples/pglite-socket/__fixtures__/deploy/embedding.sql

@@ -0,0 +1,9 @@
+-- Deploy test-simple:embedding to pg
+-- requires: table
+
+BEGIN;
+
+CREATE EXTENSION IF NOT EXISTS vector WITH SCHEMA public;
+ALTER TABLE test_app.users ADD COLUMN embedding public.vector(3);
+
+COMMIT;

examples/pglite-socket/__fixtures__/deploy/index.sql

@@ -0,0 +1,9 @@
+-- Deploy test-simple:index to pg
+-- requires: table
+
+BEGIN;
+
+CREATE INDEX idx_users_email ON test_app.users(email);
+CREATE INDEX idx_users_created_at ON test_app.users(created_at);
+
+COMMIT;

examples/pglite-socket/__fixtures__/deploy/schema.sql

@@ -0,0 +1,7 @@
+-- Deploy test-simple:schema to pg
+
+BEGIN;
+
+CREATE SCHEMA test_app;
+
+COMMIT;

examples/pglite-socket/__fixtures__/deploy/table.sql

@@ -0,0 +1,13 @@
+-- Deploy test-simple:table to pg
+-- requires: schema
+
+BEGIN;
+
+CREATE TABLE test_app.users (
+    id SERIAL PRIMARY KEY,
+    name TEXT NOT NULL,
+    email TEXT UNIQUE NOT NULL,
+    created_at TIMESTAMPTZ DEFAULT NOW()
+);
+
+COMMIT;

examples/pglite-socket/__fixtures__/pgpm.plan

@@ -0,0 +1,8 @@
+%syntax-version=1.0.0
+%project=test-simple
+%uri=https://github.com/test/simple
+
+schema 2024-01-01T00:00:00Z test <test@example.com> # Create schema
+table [schema] 2024-01-02T00:00:00Z test <test@example.com> # Create table
+index [table] 2024-01-03T00:00:00Z test <test@example.com> # Create index
+embedding [table] 2024-01-04T00:00:00Z test <test@example.com> # Add pgvector column

examples/pglite-socket/__fixtures__/revert/embedding.sql

@@ -0,0 +1,4 @@
+-- Revert test-simple:embedding from pg
+BEGIN;
+ALTER TABLE test_app.users DROP COLUMN embedding;
+COMMIT;

examples/pglite-socket/__fixtures__/revert/index.sql

@@ -0,0 +1,8 @@
+-- Revert test-simple:index from pg
+
+BEGIN;
+
+DROP INDEX test_app.idx_users_email;
+DROP INDEX test_app.idx_users_created_at;
+
+COMMIT;

examples/pglite-socket/__fixtures__/revert/schema.sql

@@ -0,0 +1,7 @@
+-- Revert test-simple:schema from pg
+
+BEGIN;
+
+DROP SCHEMA test_app CASCADE;
+
+COMMIT;

examples/pglite-socket/__fixtures__/revert/table.sql

@@ -0,0 +1,7 @@
+-- Revert test-simple:table from pg
+
+BEGIN;
+
+DROP TABLE test_app.users;
+
+COMMIT;

examples/pglite-socket/__fixtures__/verify/embedding.sql

@@ -0,0 +1,4 @@
+-- Verify test-simple:embedding on pg
+BEGIN;
+SELECT embedding::public.vector FROM test_app.users WHERE false;
+COMMIT;

examples/pglite-socket/__fixtures__/verify/index.sql

@@ -0,0 +1,7 @@
+-- Verify test-simple:index on pg
+
+SELECT 1/COUNT(*) FROM pg_indexes 
+WHERE schemaname = 'test_app' 
+AND tablename = 'users' 
+AND indexname IN ('idx_users_email', 'idx_users_created_at')
+HAVING COUNT(*) = 2;

examples/pglite-socket/__fixtures__/verify/schema.sql

@@ -0,0 +1,3 @@
+-- Verify test-simple:schema on pg
+
+SELECT 1/COUNT(*) FROM information_schema.schemata WHERE schema_name = 'test_app';

examples/pglite-socket/__fixtures__/verify/table.sql

@@ -0,0 +1,4 @@
+-- Verify test-simple:table on pg
+
+SELECT 1/COUNT(*) FROM information_schema.tables 
+WHERE table_schema = 'test_app' AND table_name = 'users';

examples/pglite-socket/__tests__/socket.e2e.test.ts

@@ -0,0 +1,118 @@
+// End-to-end example: the UNMODIFIED pgpm migrate engine deploying into PGlite
+// over the pg-gateway socket shim.
+//
+// PGlite is ElectricSQL's WASM build of Postgres ("SQLite for Postgres") — it runs
+// in-process with no server and no Postgres service container. pgpm's engine only
+// needs a node-`pg`-shaped client (`query` / `connect`), so we expose the PGlite
+// instance over a socket with the official pg-gateway shim and point pgpm at it.
+//
+// This proves the wire-protocol path. The in-process driver path (no socket) is
+// covered by @pgpmjs/pglite-adapter + pglite-test.
+import { PGlite } from '@electric-sql/pglite';
+import { vector } from '@electric-sql/pglite-pgvector';
+import { PGLiteSocketServer } from '@electric-sql/pglite-socket';
+import { PgpmMigrate } from '@pgpmjs/core';
+import { join } from 'path';
+import pg from 'pg';
+import { teardownPgPools } from 'pg-cache';
+
+const HOST = '127.0.0.1';
+const PORT = Number(process.env.PGLITE_PORT || 5544);
+const MODULE = join(__dirname, '..', '__fixtures__');
+const pgConfig = { host: HOST, port: PORT, user: 'postgres', password: 'x', database: 'postgres' };
+
+describe('pglite-socket: unmodified pgpm engine deploys into PGlite over the socket shim', () => {
+  let db: PGlite;
+  let server: PGLiteSocketServer;
+  let client: pg.Client;
+
+  beforeAll(async () => {
+    db = await PGlite.create({ extensions: { vector } });
+    await db.waitReady;
+    // Extensions are provisioned OUT-OF-BAND: pgpm's cleanSql strips CREATE EXTENSION
+    // from migrations, exactly like pgsql-test's admin.installExtensions() / the
+    // postgres-plus image. PGlite additionally requires the ext registered in JS above.
+    await db.exec('CREATE EXTENSION IF NOT EXISTS vector;');
+
+    // PGlite serializes all queries onto one engine; the socket shim caps concurrent
+    // connections at 1 by default. Raise it so a pooled client works (still serialized).
+    server = new PGLiteSocketServer({ db, host: HOST, port: PORT, maxConnections: 10 });
+    await server.start();
+
+    client = new pg.Client(pgConfig);
+    await client.connect();
+  });
+
+  afterAll(async () => {
+    // PgpmMigrate opens a pool through pg-cache; close it before the socket dies.
+    await teardownPgPools();
+    await client?.end();
+    await server?.stop();
+    await db?.close();
+  });
+
+  it('bootstraps the pgpm_migrate schema in PGlite', async () => {
+    const migrate = new PgpmMigrate(pgConfig);
+    await migrate.initialize();
+    const res = await client.query<{ schema_name: string }>(
+      "SELECT schema_name FROM information_schema.schemata WHERE schema_name = 'pgpm_migrate'"
+    );
+    expect(res.rows).toHaveLength(1);
+  });
+
+  it('deploys the plan change-by-change', async () => {
+    const migrate = new PgpmMigrate(pgConfig);
+    // useTransaction:false because over the socket PGlite pins the engine to one
+    // connection while a transaction is open; the engine's deploy path mixes a txn
+    // client with pool queries, which deadlocks a single-connection backend. The
+    // in-process @pgpmjs/pglite-adapter avoids this (both seams share one session).
+    const deployed = await migrate.deploy({ modulePath: MODULE, useTransaction: false });
+    expect(deployed.deployed).toEqual(['schema', 'table', 'index', 'embedding']);
+    expect(deployed.failed).toBeUndefined();
+  });
+
+  it('verifies every deployed change', async () => {
+    const migrate = new PgpmMigrate(pgConfig);
+    const verified = await migrate.verify({ modulePath: MODULE });
+    expect(verified.verified).toEqual(['schema', 'table', 'index', 'embedding']);
+    expect(verified.failed).toEqual([]);
+  });
+
+  it('runs a pgvector nearest-neighbor query through the pgpm-deployed schema', async () => {
+    await client.query(
+      'INSERT INTO test_app.users(name, email, embedding) VALUES ($1, $2, $3)',
+      ['ada', 'ada@example.com', '[0.1,0.2,0.3]']
+    );
+    const nearest = (
+      await client.query<{ name: string }>(
+        'SELECT name FROM test_app.users ORDER BY embedding <-> $1 LIMIT 1',
+        ['[0.1,0.2,0.3]']
+      )
+    ).rows;
+    expect(nearest).toEqual([{ name: 'ada' }]);
+  });
+
+  it('records deployed changes in the pgpm registry', async () => {
+    const changes = (
+      await client.query<{ change_name: string }>(
+        'SELECT change_name FROM pgpm_migrate.changes ORDER BY 1'
+      )
+    ).rows.map((r) => r.change_name);
+    expect(changes).toEqual(['embedding', 'index', 'schema', 'table']);
+  });
+
+  it('reverts the full plan and empties the registry', async () => {
+    const migrate = new PgpmMigrate(pgConfig);
+    await migrate.revert({ modulePath: MODULE, useTransaction: false });
+    const changes = (
+      await client.query<{ change_name: string }>(
+        'SELECT change_name FROM pgpm_migrate.changes ORDER BY 1'
+      )
+    ).rows.map((r) => r.change_name);
+    expect(changes).toEqual([]);
+    const gone = await client.query<{ t: string | null }>(
+      "SELECT to_regclass('test_app.users') AS t"
+    );
+    expect(gone.rows[0].t).toBeNull();
+  });
+});

examples/pglite-socket/jest.config.js

@@ -0,0 +1,18 @@
+/** @type {import('ts-jest').JestConfigWithTsJest} */
+module.exports = {
+  preset: 'ts-jest',
+  testEnvironment: 'node',
+  transform: {
+    '^.+\\.tsx?$': [
+      'ts-jest',
+      {
+        babelConfig: false,
+        tsconfig: 'tsconfig.json',
+      },
+    ],
+  },
+  transformIgnorePatterns: [`/node_modules/*`],
+  testRegex: '(/__tests__/.*|(\\.|/)(test|spec))\\.(jsx?|tsx?)$',
+  moduleFileExtensions: ['ts', 'tsx', 'js', 'jsx', 'json', 'node'],
+  modulePathIgnorePatterns: ['dist/*']
+};

examples/pglite-socket/package.json

@@ -0,0 +1,37 @@
+{
+  "name": "@constructive-io/examples-pglite-socket",
+  "version": "0.0.1",
+  "private": true,
+  "description": "Example: the pgpm migrate engine deploying into PGlite (WASM Postgres) over the pg-gateway socket shim — no Postgres service required.",
+  "homepage": "https://github.com/constructive-io/constructive",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/constructive-io/constructive"
+  },
+  "keywords": [
+    "postgres",
+    "postgresql",
+    "pglite",
+    "wasm",
+    "pgpm",
+    "migrations",
+    "socket",
+    "pg-gateway",
+    "example"
+  ],
+  "scripts": {
+    "lint": "eslint . --fix",
+    "test": "NODE_OPTIONS=--experimental-vm-modules jest --passWithNoTests",
+    "test:watch": "NODE_OPTIONS=--experimental-vm-modules jest --watch"
+  },
+  "devDependencies": {
+    "@electric-sql/pglite": "0.5.4",
+    "@electric-sql/pglite-pgvector": "0.0.5",
+    "@electric-sql/pglite-socket": "0.2.7",
+    "@pgpmjs/core": "workspace:^",
+    "@types/pg": "^8.20.0",
+    "pg": "^8.21.0",
+    "pg-cache": "workspace:^"
+  }
+}

examples/pglite-socket/tsconfig.json

@@ -0,0 +1,9 @@
+{
+  "extends": "../../tsconfig.json",
+  "compilerOptions": {
+    "noEmit": true,
+    "rootDir": "."
+  },
+  "include": ["__tests__/**/*.ts"],
+  "exclude": ["node_modules"]
+}