Skip to content

fix: confine all objects to the apalis schema (#86)#89

Merged
geofmureithi merged 4 commits into
apalis-dev:mainfrom
tysen:fix/confine-to-apalis-schema
Jul 7, 2026
Merged

fix: confine all objects to the apalis schema (#86)#89
geofmureithi merged 4 commits into
apalis-dev:mainfrom
tysen:fix/confine-to-apalis-schema

Conversation

@tysen

@tysen tysen commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Fixes #86

Every object apalis creates now lives in the apalis schema:

  • generate_ulid() is created as apalis.generate_ulid() with a pinned SET search_path = apalis, public, so its bare gen_random_bytes() call resolves whether pgcrypto is installed into apalis (fresh DBs) or already exists in public.
  • pgcrypto is installed with SCHEMA apalis on fresh databases.
  • The sqlx migrations table moves to apalis._sqlx_migrations via a new sqlx.toml (create-schemas + table-name), which requires sqlx 0.9. The first migration's CREATE SCHEMA is now IF NOT EXISTS to coexist with create-schemas.

Also bumps sqlx 0.8 -> 0.9 and remaps the runtime/TLS cargo features (0.9 removed the combined runtime-*-tls flags), regenerates the .sqlx offline cache, and updates deny.toml / cargo-vet exemptions for the new dependency tree.

BREAKING CHANGE: existing pre-1.0 deployments track migrations in public._sqlx_migrations; after this change sqlx looks in apalis._sqlx_migrations and will try to re-run every migration. Recreate the apalis schema (and drop public._sqlx_migrations) when upgrading.

@tysen
tysen requested a review from geofmureithi as a code owner June 4, 2026 17:06
@geofmureithi

geofmureithi commented Jun 4, 2026 via email

Copy link
Copy Markdown
Member

Move the objects apalis creates out of `public`

- generate_ulid() becomes apalis.generate_ulid() and no longer depends on
  pgcrypto -- its 10 random bytes come from core gen_random_uuid() instead
  of pgcrypto's gen_random_bytes(). A new forward migration creates it,
  repoints the legacy (driver-unused) apalis.push_job at it, and drops
  public.generate_ulid.
- The sqlx migrations table moves from public._sqlx_migrations to
  apalis._sqlx_migrations via a new sqlx.toml (requires sqlx 0.9). This
  also isolates apalis's migration history from a user's own sqlx
  migrations, which previously collided over the shared default table name.
- pgcrypto is no longer used; it is left where an earlier version installed
  it and documented as droppable.

The upgrade is automatic and non-breaking. PostgresStorage::setup()
relocates an existing apalis-owned public._sqlx_migrations into the apalis
schema and re-stamps checksums (read from the embedded migrator) before
running migrations, so existing deployments migrate with no manual steps
and nothing is re-run. The relocation is guarded so it never adopts a
user's own public._sqlx_migrations sharing the default name. The only
edited migration is the first one (CREATE SCHEMA -> CREATE SCHEMA IF NOT
EXISTS, so create-schemas can pre-create the schema on fresh installs);
its checksum is healed by the same transition.

Also bumps sqlx 0.8 -> 0.9 (remapping the runtime/TLS cargo features that
0.9 split apart) and updates deny.toml / cargo-vet for the new dep tree.
@tysen
tysen force-pushed the fix/confine-to-apalis-schema branch from b4c57bb to 513c73b Compare June 5, 2026 19:55
Comment thread src/lib.rs Outdated
Comment thread migrations/20220530084123_jobs_workers.sql
@tysen

tysen commented Jun 7, 2026

Copy link
Copy Markdown
Contributor Author

The accidental pollution of the public schema with generate_ulid and pgcrypto is unfortunate but mostly harmless. Creating and using _sqlx_migrations in public is a real problem that has bitten users many times (see apalis-dev/apalis#81, apalis-dev/apalis#177, apalis-dev/apalis#439, #64).

I think you have an opportunity with your impending 1.0 release to make a breaking change and clean this up - this PR does so. The 0.9 version of sqlx lets us easily configure _sqlx_migrations to live in apalis. It is fortuitous that sqlx has added this capability right around now!

We have to edit the first migration because of the missing IF NOT EXISTS - because we would be creating the apalis schema outside of (and before) the sqlx migrations, it would fail otherwise.

We can't use an sqlx migration to move _sqlx_migrations into the apalis schema because sqlx creates and reads that table at the start of Migrator::run, before any migration in the set executes — so a migration can never relocate the table that's tracking it (it would run too late, and against the freshly-created table in the new location). Instead, PostgresStorage::setup() does the one-time move, relocating an existing public._sqlx_migrations into apalis and re-stamping the one edited migration's checksum, before handing off to the migrator. I believe this will work for new and existing users.

@geofmureithi

Copy link
Copy Markdown
Member

Hey @tysen

Fresh installs via sqlx-cli already work, it reads sqlx.toml (create-schemas + table-name),

Have you run your fixes via sqlx migrate run? I tried and got some errors:

sqlx migrate run
Found `sqlx.toml` in current directory; reading...
error: while executing migration 20220530084123: error returned from database: function "notify_new_jobs" already exists with same argument types at line 374

@geofmureithi

geofmureithi commented Jun 22, 2026

Copy link
Copy Markdown
Member

@tysen After playing around with the PR I think most of the code is ok.

I recommend we dont include the relocation functions.

I suggest we let the migrations fail and the user to manually upgrade their database. The docs would highlight that the manual update is needed:

When a user encounters:

called `Result::unwrap()` on an `Err` value: Migrate(VersionMismatch(20220530084123))

They can manually call:

ALTER TABLE public._sqlx_migrations SET SCHEMA apalis;

UPDATE apalis._sqlx_migrations SET checksum = ... WHERE version = 20220530084123;

Since this needs to be run once, it would be part of docs (Upgrading to V1) and not included in the code.

I need to take one more look and try out on several existing databases but those are my two cents.

Let me know your thoughts

Remove the auto relocation + checksum re-stamp from setup() and the
public relocate_legacy_migrations_table / reconcile_migration_checksums helpers.
setup() now just runs migrations.

Docs now cover the two failure modes (function-already-exists on a blind
upgrade, and the empty apalis._sqlx_migrations that then blocks the
ALTER TABLE).
@tysen

tysen commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

That sounds like a good idea to me. I've updated the code.

@codecov

codecov Bot commented Jun 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@geofmureithi
geofmureithi merged commit b6983a1 into apalis-dev:main Jul 7, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Confine all objects to the apalis schema (generate_ulid, _sqlx_migrations, pgcrypto currently leak into public)

2 participants