From 8e145f2dcbb6188fc37b1221d5ecd27275c1880d Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 14:36:00 -0400 Subject: [PATCH 01/11] =?UTF-8?q?docs:=20design=20for=20simplifying=20veti?= =?UTF-8?q?ver=E2=86=94Connect=20integration=20testing?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adopt posit-dev/with-connect in both repos, move vetiver's Connect tests fully into vetiver, strip vetiver-specific test baggage from rsconnect-python (re-homing its own system-caches integration test), and relabel the deploy_python_fastapi compatibility shim rather than removing it. --- ...iver-connect-test-simplification-design.md | 223 ++++++++++++++++++ 1 file changed, 223 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md diff --git a/docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md b/docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md new file mode 100644 index 00000000..feda0813 --- /dev/null +++ b/docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md @@ -0,0 +1,223 @@ +# Simplifying the vetiver ↔ Connect integration-testing relationship + +**Date:** 2026-06-30 +**Status:** Approved design, pending implementation plan +**Repos affected:** `vetiver-python`, `rsconnect-python` + +## Problem + +The integration-testing relationship between `vetiver-python` and +`rsconnect-python` is both inverted and duplicated: + +- The real dependency is one-way: `vetiver-python` declares + `rsconnect-python>=1.11.0` as a production dependency and calls + `rsconnect.actions.deploy_python_fastapi` / `rsconnect.api.RSConnectServer` / + `RSConnectClient` directly. +- Despite not depending on vetiver, `rsconnect-python` carries vetiver-specific + test baggage: a `vetiver-testing/` harness, `tests/test_vetiver_pins.py`, a + `--vetiver` pytest marker, and a nightly `test-dev-connect` CI job that installs + vetiver and deploys a model. +- Both repos independently maintain a near-identical bespoke Connect-in-Docker + harness (`docker-compose.yml`, `setup-rsconnect/` with `dump_api_keys.py`, + `add-users.sh`, `users.txt`, `rstudio-connect.gcfg`). + +## Goal + +vetiver owns and runs all vetiver↔Connect integration tests in its own repo; +`rsconnect-python` carries no vetiver-specific test code. Both repos use +[`with-connect`](https://github.com/posit-dev/with-connect) — the preferred +shared tool for running Connect in Docker during tests — instead of a hand-rolled +harness. + +## Target architecture + +``` +rsconnect-python vetiver-python + ├ tests/test_main_system_caches.py ├ vetiver/tests/test_rsconnect.py + │ (own integration test; kept) │ (reads env creds, single admin user) + ├ CI: own with-connect job; ├ CI: posit-dev/with-connect Action (pinned) + │ creates publisher user via admin API ├ local: `make test-rsc` → uvx with-connect + └ actions.py compatibility shim └ deletes docker-compose.yml, + (deploy_python_fastapi for vetiver) script/setup-rsconnect/, dump_api_keys.py + +Deleted from rsconnect-python: vetiver-testing/, tests/test_vetiver_pins.py, +the --vetiver marker, the nightly test-dev-connect job, root docker-compose.yml, +and the dev/dev-stop Justfile recipes. +``` + +The reverse-dependency canary ("does new rsconnect break vetiver?") lives in +vetiver's weekly CI matrix, which already installs `rsconnect` from `main` and +runs the integration tests. rsconnect PRs do not get that signal directly; a +failure notification on the weekly canary is recommended so rsconnect maintainers +learn of breakage without watching the schedule. + +### `with-connect` behavior the design relies on + +Verified against `posit-dev/with-connect@main`: + +- Exports exactly two env vars to the wrapped command: `CONNECT_SERVER` and + `CONNECT_API_KEY`. +- Bootstraps a single admin-privileged API key via JWT; it does **not** create + named PAM users. The bootstrapped username is assigned by Connect — derive it + at runtime, never hardcode it. +- CLI `--license` takes a file path (default `./rstudio-connect.lic`). The + Action's `license` input takes license *contents* and writes the file itself. +- Does nothing with `requirements.txt` / manifests. +- Default `version` is `release`; supports `--config`/`config-file`, `-e`/`env`, + `--port`. +- Tool runtime needs Python 3.13 / `uv`, satisfied via `uvx` or the Action. This + is independent of the test matrix: the wrapped `pytest` runs in the CI job's + own Python (3.9/3.10), so there is no Python-version regression. + +## vetiver-python changes + +### Test fixture — `vetiver/tests/test_rsconnect.py` + +- Read `CONNECT_SERVER` and `CONNECT_API_KEY` from `os.environ` instead of the + hardcoded `http://localhost:3939` and the `rsconnect_api_keys.json` file. +- Use the single `with-connect` admin key. Remove `get_key`, `rsc_from_key`, + `rsc_fs_from_key`, and the JSON helpers; the `rsc_short` fixture builds its + `BoardRsConnect` from the one env-provided key. The current test already uses + only one user (`susan`), so a single user covers everything it verifies. +- Do not hardcode a username. Derive it at runtime via + `rsc.get_user()["username"]` and build the pin name as `f"{username}/model"`; + update the corresponding `pin_read`, `deploy_connect(pin_name=...)`, and + `content_search` title lookup. +- The `Authorization: Key ...` header and prediction assertions + (`response.iloc[0, 0] == 44.47`, `len(response) == 100`) are unchanged. + +### Local dev — `Makefile` + +- Remove `dev-start`, `dev`, the `RSC_API_KEYS` variable, and the `dump_api_keys` + invocation. +- `make test-rsc` runs the `with-connect` CLI, e.g. + `uvx --from git+https://github.com/posit-dev/with-connect@ with-connect -- pytest vetiver -m 'rsc_test'`. + License resolves from `./rstudio-connect.lic` per the CLI default. +- `make test` (the `not rsc_test and not docker` run) is unchanged. + +### CI — `.github/workflows/tests.yml` and `weekly.yml` + +- In each Connect job, replace the `docker compose up --build -d` + `make dev` + block with the Action, pinned to a release tag or commit SHA: + + ```yaml + - uses: posit-dev/with-connect@ + with: + version: # release, or a pinned version in the weekly matrix + license: ${{ secrets.RSC_LICENSE }} # existing secret = license CONTENTS + command: | + pip freeze > requirements.txt + pytest vetiver -m 'rsc_test' + ``` + + The `pip freeze > requirements.txt` step stays because the test deploys with + `extra_files=["requirements.txt"]` so Connect can rebuild the model env; this is + unrelated to `with-connect`. +- The weekly matrix keeps its version combinations: vetiver `pypi` × rsconnect + `main`, and vetiver `main` × rsconnect `main`. Only the harness invocation + changes; the rsconnect-version install steps stay. These jobs are the canary. +- Add a failure notification (issue/Slack) on the weekly canary. + +### Deletions + +- `docker-compose.yml` +- `script/setup-rsconnect/` (`dump_api_keys.py`, `add-users.sh`, `users.txt`, + `rstudio-connect.gcfg`) +- `vetiver/tests/rsconnect_api_keys.json` + +## rsconnect-python changes + +### Keep and re-home `tests/test_main_system_caches.py` + +This is rsconnect's own integration test for the `system caches` CLI, unrelated +to vetiver, that currently freeloads on the vetiver harness (it reads +`vetiver-testing/rsconnect_api_keys.json` and runs inside the `test-dev-connect` +job). It must survive the harness removal: + +- It needs two privilege levels — an admin (can list/delete caches) and a + non-admin publisher (must be denied) — so it cannot use only the `with-connect` + admin key. +- Refactor it to read `CONNECT_SERVER`/`CONNECT_API_KEY` from env, and add a + fixture that uses the admin key to create a publisher user via the Connect API + and mint that user's key at runtime (replacing static `get_key("susan")`). +- Give rsconnect its own `with-connect`-based CI job (Action, pinned) that runs + `pytest tests/test_main_system_caches.py`. Confirm during implementation that + the cache add/remove host commands (`ADD_CACHE_COMMAND`/`RM_CACHE_COMMAND`) can + still target the `with-connect`-managed container. + +### Relabel the `actions.py` compatibility shim + +`actions.py` lines 281–442 (`validate_extra_files`, `validate_entry_point`, +`deploy_app`) are kept and reframed as a supported compatibility entry point used +by vetiver: + +- `deploy_python_fastapi` (line 446, called by vetiver's `deploy_connect`) does + `return deploy_app(...)`, and `deploy_app` calls the two `validate_*` functions + — all within this block. `deploy_app` has no other caller. These functions are + load-bearing for vetiver and have no `DeprecationWarning` or changelog notice, + so they are treated as a maintained shim, not dead code. +- Rewrite the block-marker comments (lines 281–285, 441–443) to describe this as + a supported compatibility entry point for `vetiver-python`, dropping the + "deprecated … will be removed in the future" language. +- Optional cleanup: have `deploy_app` call the `bundle.py` `validate_*` functions + instead of the local copies, so vetiver deploys stop emitting spurious "This + method has been moved and will be deprecated" warnings on every call. + +The identically-named `validate_extra_files` / `validate_entry_point` in +`bundle.py` (lines ~1774, ~1845) are the active versions used by the `main.py` +CLI and must not be altered. The `actions.py` copies are reachable only through +`deploy_app`. + +### Deletions + +- `vetiver-testing/` (entire directory). +- `tests/test_vetiver_pins.py`. +- `conftest.py`: the `--vetiver` option and `pytest.mark.vetiver` skip logic. +- `pyproject.toml`: the `vetiver` marker definition and the `vetiver-testing/` + ruff exclude. +- `.github/workflows/main.yml`: the `test-dev-connect` job (replaced by the + rsconnect-own `system caches` job above, which no longer installs vetiver). +- Root `docker-compose.yml` and the `dev` / `dev-stop` recipes in `Justfile` + (lines ~62–71). Their only consumers are the vetiver dev/test path and the + `test-dev-connect` job. The separate `integration-testing/docker-compose.yml` + is unrelated and stays. + +## Sequencing + +The two repos' changes are independent and may land in either order; vetiver +keeps working against rsconnect `main` throughout. Suggested intra-repo order for +rsconnect to keep CI green at every commit: + +1. Update `conftest.py` to drop the `--vetiver` option/marker logic. +2. Refactor and re-home `test_main_system_caches.py` onto the new `with-connect` + job (create publisher user via admin API; env-var creds). +3. Delete `tests/test_vetiver_pins.py`. +4. Update `pyproject.toml` (drop `vetiver` marker + ruff exclude). +5. Replace the `test-dev-connect` job in `main.yml`. +6. Delete the `dev`/`dev-stop` Justfile recipes and root `docker-compose.yml`. +7. Delete `vetiver-testing/`. +8. Relabel the `actions.py` shim comments. + +## Verification items + +- Confirm `with-connect`'s default Connect image/config enables Python content + execution (vetiver's test deploys a FastAPI app). If not, enable via the + Action's `config-file` or `-e`. +- Confirm the bootstrapped admin user can write pins to its own namespace and that + `rsc.get_user()["username"]` returns the expected value. +- Confirm creating a non-admin publisher user via the Connect API with the + bootstrap admin key works, for `test_main_system_caches.py`. +- Confirm the `system caches` host commands can still add/remove caches in the + `with-connect`-managed container. +- Pin a specific `with-connect` Action ref (tag or SHA) and CLI ref; record an + update strategy. + +## Risks / accepted tradeoffs + +- rsconnect PRs lose direct vetiver-breakage signal; it moves to vetiver's weekly + CI, with a failure notification to compensate. +- New external dependency on `posit-dev/with-connect`, mitigated by pinning to a + tag/SHA. Both repos rely on Docker either way. +- vetiver remains coupled to rsconnect's internal `deploy_python_fastapi` shim. + Removing that coupling is out of scope and would require a real deprecation + cycle plus migrating vetiver to the `rsconnect` CLI. From c6691829c16c1e5211f3f4130cc25349343f772f Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 14:44:35 -0400 Subject: [PATCH 02/11] docs: implementation plan for with-connect adoption --- .../2026-06-30-vetiver-with-connect-tests.md | 447 ++++++++++++++++++ 1 file changed, 447 insertions(+) create mode 100644 docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md diff --git a/docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md b/docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md new file mode 100644 index 00000000..460dc2f6 --- /dev/null +++ b/docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md @@ -0,0 +1,447 @@ +# vetiver-python: adopt with-connect for Connect tests — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace vetiver's bespoke Connect-in-Docker test harness with `posit-dev/with-connect`, so the Connect integration test reads credentials from the environment and the repo no longer carries its own docker-compose/user-bootstrap scripts. + +**Architecture:** `with-connect` boots a licensed Connect container, bootstraps a single admin API key, and exports `CONNECT_SERVER` + `CONNECT_API_KEY` to the wrapped command. The test fixture reads those env vars and derives the pin namespace from the authenticated user. CI uses the `with-connect` GitHub Action; local dev uses the `with-connect` CLI via `uvx`. + +**Tech Stack:** Python, pytest, pins, rsconnect-python, Docker, `uv`/`uvx`, GitHub Actions. + +## Global Constraints + +- vetiver supports Python 3.9+; the Connect test job runs on Python 3.9. (`with-connect`'s own 3.13 requirement is satisfied by `uvx`/the Action and does not affect the test interpreter.) +- `with-connect` exposes **only** `CONNECT_SERVER` and `CONNECT_API_KEY` to a wrapped command. Do not rely on any other injected env var, on named users (`susan`/`derek`/`admin`), or on a JSON keys file. +- Never hardcode a Connect username; derive it at runtime via the pins API. +- Pin `with-connect` to commit `0783dabdd24e360e985a4588ce1239c3dc31c542` (no release tags exist yet). Verify this is still the desired ref at execution time with `gh api repos/posit-dev/with-connect/commits/main -q .sha`. +- A valid `rstudio-connect.lic` must be present in the repo root for local runs (gitignored). CI passes the license via the `RSC_LICENSE` secret. +- The Connect test deploys with `extra_files=["requirements.txt"]`, so a `requirements.txt` snapshot of the test env must exist in the working directory before the test runs. + +--- + +### Task 1: Refactor `test_rsconnect.py` to env-var credentials + single user + +**Files:** +- Modify: `vetiver/tests/test_rsconnect.py` (full rewrite of fixtures/helpers) + +**Interfaces:** +- Consumes: `CONNECT_SERVER`, `CONNECT_API_KEY` from the environment (provided by `with-connect`). +- Produces: a single `rsc_admin` board fixture and a `username` fixture; no JSON-key helpers remain. + +- [ ] **Step 1: Replace the module so credentials are read lazily and the user is derived at runtime** + +Rationale for lazy reads: pytest imports every test module during collection even for `make test` (which deselects `rsc_test` by marker). Module-level `os.environ["CONNECT_SERVER"]` would raise `KeyError` during that collection. Use `os.environ.get(...)` at module scope and `pytest.skip(...)` inside the fixtures. + +Replace the entire contents of `vetiver/tests/test_rsconnect.py` with: + +```python +import os + +import numpy as np +import pandas as pd +import pytest +import sklearn + +import pins +from pins.boards import BoardRsConnect +from pins.rsconnect.api import RsConnectApi +from pins.rsconnect.fs import RsConnectFs +from rsconnect.api import RSConnectClient, RSConnectServer + +import vetiver + +RSC_SERVER_URL = os.environ.get("CONNECT_SERVER") +RSC_API_KEY = os.environ.get("CONNECT_API_KEY") + +pytestmark = pytest.mark.rsc_test # noqa + + +def _require_connect(): + if not RSC_SERVER_URL or not RSC_API_KEY: + pytest.skip("CONNECT_SERVER and CONNECT_API_KEY must be set (run via with-connect)") + + +def _api(): + return RsConnectApi(RSC_SERVER_URL, RSC_API_KEY) + + +def rsc_delete_user_content(rsc): + guid = rsc.get_user()["guid"] + content = rsc.get_content(owner_guid=guid) + for entry in content: + rsc.delete_content_item(entry["guid"]) + + +@pytest.fixture(scope="function") +def username(): + _require_connect() + return _api().get_user()["username"] + + +@pytest.fixture(scope="function") +def rsc_admin(): + # tears down content after each test + _require_connect() + fs = RsConnectFs(_api()) + rsc_delete_user_content(fs.api) + yield BoardRsConnect("", fs, allow_pickle_read=True) + rsc_delete_user_content(fs.api) + + +def test_deploy(rsc_admin, username): + np.random.seed(500) + + # Load data, model + X_df, y = vetiver.mock.get_mock_data() + model = vetiver.mock.get_mock_model().fit(X_df, y) + + pin_name = f"{username}/model" + v = vetiver.VetiverModel(model=model, prototype_data=X_df, model_name=pin_name) + + board = pins.board_connect( + server_url=RSC_SERVER_URL, api_key=RSC_API_KEY, allow_pickle_read=True + ) + + vetiver.vetiver_pin_write(board=board, model=v) + connect_server = RSConnectServer(url=RSC_SERVER_URL, api_key=RSC_API_KEY) + assert isinstance(board.pin_read(pin_name), sklearn.dummy.DummyRegressor) + + vetiver.deploy_connect( + connect_server=connect_server, + board=board, + pin_name=pin_name, + title="testapi", + extra_files=["requirements.txt"], + ) + + # get url of where content lives + client = RSConnectClient(connect_server) + dicts = client.content_search() + rsc_api = list(filter(lambda x: x["title"] == "testapi", dicts)) + content_url = rsc_api[0].get("content_url") + + h = {"Authorization": f"Key {RSC_API_KEY}"} + + endpoint = vetiver.vetiver_endpoint(content_url + "/predict") + response = vetiver.predict(endpoint, X_df, headers=h) + + assert isinstance(response, pd.DataFrame), response + assert response.iloc[0, 0] == 44.47 + assert len(response) == 100 +``` + +- [ ] **Step 2: Verify the normal (non-Connect) test suite still collects and runs** + +This confirms the lazy env reads didn't break collection of the module for the default run. + +Run: `pytest -m 'not rsc_test and not docker' -q` +Expected: PASS (no `KeyError` during collection; `test_rsconnect.py::test_deploy` is deselected by the marker). + +- [ ] **Step 3: Verify the Connect test passes under with-connect** + +Requires Docker + `rstudio-connect.lic` in the repo root. + +Run: +```bash +pip freeze > requirements.txt +uvx --from git+https://github.com/posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 \ + with-connect -- pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' -v +``` +Expected: PASS — `test_deploy` deploys the model and asserts `response.iloc[0, 0] == 44.47`. + +- [ ] **Step 4: Commit** + +```bash +git add vetiver/tests/test_rsconnect.py +git commit -m "test: read Connect creds from env, derive pin user at runtime" +``` + +--- + +### Task 2: Point `make test-rsc` at with-connect and remove the dev harness targets + +**Files:** +- Modify: `Makefile` + +**Interfaces:** +- Consumes: nothing new. +- Produces: a `test-rsc` target that no longer depends on `dev`/`dev-start`/`dev-stop` or `RSC_API_KEYS`. + +- [ ] **Step 1: Remove the `RSC_API_KEYS` variable** + +Delete this line near the top of `Makefile`: + +```make +RSC_API_KEYS=vetiver/tests/rsconnect_api_keys.json +``` + +- [ ] **Step 2: Replace the `test-rsc` target** + +Change: + +```make +test-rsc: clean-test + pytest +``` + +to: + +```make +test-rsc: clean-test + pip freeze > requirements.txt + uvx --from git+https://github.com/posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 \ + with-connect -- pytest vetiver -m 'rsc_test' +``` + +- [ ] **Step 3: Remove the `dev`, `dev-start`, `dev-stop`, and `$(RSC_API_KEYS)` targets** + +Delete these blocks from `Makefile`: + +```make +dev: vetiver/tests/rsconnect_api_keys.json + +dev-start: + docker compose up -d + docker compose exec -T rsconnect bash < script/setup-rsconnect/add-users.sh + # curl fails with error 52 without a short sleep.... + sleep 5 + curl -s --retry 10 --retry-connrefused http://localhost:3939 + +dev-stop: + docker compose down + rm -f $(RSC_API_KEYS) +``` + +and + +```make +$(RSC_API_KEYS): dev-start + python script/setup-rsconnect/dump_api_keys.py $@ +``` + +- [ ] **Step 4: Update the `help` target text** + +Delete these three lines from the `help` recipe: + +```make + @echo "dev - generate Connect API keys" + @echo "dev-start - start up development Connect in Docker" + @echo "dev-stop - stop Connect dev container" +``` + +- [ ] **Step 5: Verify the Makefile is well-formed and `test-rsc` runs** + +Run: `make -n test-rsc` +Expected: prints the `pip freeze` + `uvx ... with-connect -- pytest` commands with no `make` errors. + +Run (Docker + license present): `make test-rsc` +Expected: PASS (same green result as Task 1, Step 3). + +- [ ] **Step 6: Commit** + +```bash +git add Makefile +git commit -m "build: run rsc tests via with-connect, drop bespoke dev targets" +``` + +--- + +### Task 3: Convert the `test-connect` CI job to the with-connect Action + +**Files:** +- Modify: `.github/workflows/tests.yml` (the `test-connect` job) + +**Interfaces:** +- Consumes: `secrets.RSC_LICENSE` (license contents). +- Produces: a `test-connect` job that needs no `docker-compose.yml` or `make dev`. + +- [ ] **Step 1: Replace the `run Connect` + `Run tests` steps** + +In the `test-connect` job, replace these two steps: + +```yaml + - name: run Connect + run: | + docker compose up --build -d + pip freeze > requirements.txt + make dev + cat requirements.txt + env: + RSC_LICENSE: ${{ secrets.RSC_LICENSE }} + GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} + + # NOTE: edited to run checks for python package + - name: Run tests + run: | + pytest vetiver -m 'rsc_test' +``` + +with: + +```yaml + - name: Run Connect integration tests + uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 + with: + license: ${{ secrets.RSC_LICENSE }} + command: | + pip freeze > requirements.txt + pytest vetiver -m 'rsc_test' +``` + +Note: keep the existing `Install dependencies` step (it still installs `.[dev]` and the vetiver build) so `pip freeze` captures the deployable environment. + +- [ ] **Step 2: Lint the workflow YAML** + +Run: `python -c "import yaml,sys; yaml.safe_load(open('.github/workflows/tests.yml'))"` +Expected: no output (valid YAML). + +- [ ] **Step 3: Commit** + +```bash +git add .github/workflows/tests.yml +git commit -m "ci: run Connect tests via with-connect action" +``` + +--- + +### Task 4: Convert the weekly workflow jobs + +**Files:** +- Modify: `.github/workflows/weekly.yml` + +**Interfaces:** +- Consumes: `secrets.RSC_LICENSE`. +- Produces: the two rsconnect jobs run via the Action; the two pins jobs no longer start Connect. + +- [ ] **Step 1: Remove Connect setup from the two `*_pins_main` jobs** + +`vetiver_main_pins_main` and `vetiver_pypi_pins_main` run `make test` (which is `-m 'not rsc_test and not docker'`), so they do not need Connect. In **both** jobs, delete the `run Connect` step: + +```yaml + - name: run Connect + run: | + docker compose up --build -d + make dev + env: + RSC_LICENSE: ${{ secrets.RSC_LICENSE }} + GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} +``` + +Leave their final `make test` step intact. + +- [ ] **Step 2: Convert the two `*_rsconnect_latest` jobs to the Action** + +In `vetiver_pypi_rsconnect_latest` and `vetiver_latest_rsconnect_latest`, replace the `run Connect` + `Run Tests` steps: + +```yaml + - name: run Connect + run: | + docker compose up --build -d + make dev + env: + RSC_LICENSE: ${{ secrets.RSC_LICENSE }} + GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} + + - name: Run Tests + run: | + pytest -m 'rsc_test' +``` + +with: + +```yaml + - name: Run Connect integration tests + uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 + with: + license: ${{ secrets.RSC_LICENSE }} + command: | + pip freeze > requirements.txt + pytest vetiver -m 'rsc_test' +``` + +These jobs already run `pip freeze > requirements.txt` in their `Install dependencies` step; the line inside `command:` re-snapshots after all installs and is harmless. The rsconnect-from-`main` install steps stay unchanged — these jobs remain the reverse-dependency canary. + +- [ ] **Step 3: Add a failure notification to the canary jobs** + +Append this step to **both** `*_rsconnect_latest` jobs so rsconnect breakage is surfaced: + +```yaml + - name: Open issue on failure + if: failure() && github.event_name == 'schedule' + uses: actions/github-script@v7 + with: + script: | + github.rest.issues.create({ + owner: context.repo.owner, + repo: context.repo.repo, + title: `Weekly canary failed: ${context.job}`, + body: `The weekly Connect canary failed on run ${context.runId}. This often means a change in rsconnect-python \`main\` broke vetiver. See https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`, + labels: ["ci-failure"] + }) +``` + +- [ ] **Step 4: Lint the workflow YAML** + +Run: `python -c "import yaml,sys; yaml.safe_load(open('.github/workflows/weekly.yml'))"` +Expected: no output (valid YAML). + +- [ ] **Step 5: Commit** + +```bash +git add .github/workflows/weekly.yml +git commit -m "ci: weekly Connect jobs via with-connect; drop Connect from pins-only jobs" +``` + +--- + +### Task 5: Delete the bespoke harness files + +**Files:** +- Delete: `docker-compose.yml` +- Delete: `script/setup-rsconnect/dump_api_keys.py`, `add-users.sh`, `users.txt`, `rstudio-connect.gcfg` (the whole `script/setup-rsconnect/` directory) +- Delete: `vetiver/tests/rsconnect_api_keys.json` (if present / untracked) + +- [ ] **Step 1: Confirm nothing else references the harness paths** + +Run: +```bash +grep -rn "setup-rsconnect\|rsconnect_api_keys\|docker-compose\|docker compose" \ + --include='*.py' --include='*.yml' --include='*.yaml' --include='Makefile' . \ + | grep -v 'docs/superpowers' +``` +Expected: no hits outside `docs/superpowers/` (the design/plan docs). If `script/setup-docker/` appears, that is the *Docker* test path (`test-docker` job) and is unrelated — leave it. + +- [ ] **Step 2: Delete the files** + +Run: +```bash +git rm docker-compose.yml +git rm -r script/setup-rsconnect +rm -f vetiver/tests/rsconnect_api_keys.json +``` + +- [ ] **Step 3: Verify the default test suite still passes** + +Run: `pytest -m 'not rsc_test and not docker' -q` +Expected: PASS. + +- [ ] **Step 4: Verify the Connect test still passes end-to-end** + +Run: `make test-rsc` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add -A +git commit -m "chore: remove bespoke Connect test harness (replaced by with-connect)" +``` + +--- + +## Self-Review notes + +- Spec "vetiver-python changes" → Tasks 1–5 cover the fixture, Makefile, `tests.yml`, `weekly.yml`, and all deletions. +- The spec's `requirements.txt` requirement is honored in Task 1 Step 3, Task 2 Step 2, and the `command:` blocks in Tasks 3–4. +- `with-connect` ref is pinned to one SHA used identically in Tasks 1–4 (`0783dabdd24e360e985a4588ce1239c3dc31c542`). +- The lazy-env-read detail (Task 1) prevents a collection-time regression in the default `pytest` run. From f262ceb059960802a5e69634a7e6184565da9a41 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 14:53:57 -0400 Subject: [PATCH 03/11] chore: gitignore local rstudio-connect.lic --- .gitignore | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.gitignore b/.gitignore index c546a16a..884efd2c 100644 --- a/.gitignore +++ b/.gitignore @@ -149,3 +149,6 @@ NOTES.md /.quarto/ /.luarc.json + +# Local Connect license for with-connect integration tests +rstudio-connect.lic From f0be51c2c7c3a7f3e5023be241288573b06448d6 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 15:19:34 -0400 Subject: [PATCH 04/11] test: read Connect creds from env, derive pin user at runtime Replaces the bespoke docker-compose credential setup (hardcoded localhost:3939 + JSON key file with named users) with env-var credentials provided by with-connect. Credentials are read lazily at module scope via os.environ.get() to avoid KeyError during normal test collection when CONNECT_SERVER / CONNECT_API_KEY are not set. Additional fixes needed for compatibility with rsconnect-python 1.29.x and FastAPI 0.138.x / pydantic v2: - content_search() renamed to content_list() - new=True required to avoid stale AppStore GUID across fresh containers - content_url may include a trailing slash; strip before appending /predict - Content-Type: application/json required; modern FastAPI/pydantic v2 no longer accepts JSON body without explicit content-type header --- vetiver/tests/test_rsconnect.py | 76 +++++++++++++++------------------ 1 file changed, 35 insertions(+), 41 deletions(-) diff --git a/vetiver/tests/test_rsconnect.py b/vetiver/tests/test_rsconnect.py index adeafae3..81897193 100644 --- a/vetiver/tests/test_rsconnect.py +++ b/vetiver/tests/test_rsconnect.py @@ -1,40 +1,31 @@ +import os + +import numpy as np +import pandas as pd import pytest -import json import sklearn -import pins -import pandas as pd -import numpy as np +import pins from pins.boards import BoardRsConnect from pins.rsconnect.api import RsConnectApi from pins.rsconnect.fs import RsConnectFs -from rsconnect.api import RSConnectServer, RSConnectClient +from rsconnect.api import RSConnectClient, RSConnectServer import vetiver -RSC_SERVER_URL = "http://localhost:3939" -RSC_KEYS_FNAME = "vetiver/tests/rsconnect_api_keys.json" +RSC_SERVER_URL = os.environ.get("CONNECT_SERVER") +RSC_API_KEY = os.environ.get("CONNECT_API_KEY") pytestmark = pytest.mark.rsc_test # noqa -def get_key(name): - with open(RSC_KEYS_FNAME) as f: - api_key = json.load(f)[name] - return api_key - +def _require_connect(): + if not RSC_SERVER_URL or not RSC_API_KEY: + pytest.skip("CONNECT_SERVER and CONNECT_API_KEY must be set (run via with-connect)") -def rsc_from_key(name): - with open(RSC_KEYS_FNAME) as f: - api_key = json.load(f)[name] - return RsConnectApi(RSC_SERVER_URL, api_key) - -def rsc_fs_from_key(name): - - rsc = rsc_from_key(name) - - return RsConnectFs(rsc) +def _api(): + return RsConnectApi(RSC_SERVER_URL, RSC_API_KEY) def rsc_delete_user_content(rsc): @@ -45,52 +36,55 @@ def rsc_delete_user_content(rsc): @pytest.fixture(scope="function") -def rsc_short(): - # tears down content after each test - fs_susan = rsc_fs_from_key("susan") +def username(): + _require_connect() + return _api().get_user()["username"] - # delete any content that might already exist - rsc_delete_user_content(fs_susan.api) - yield BoardRsConnect( - "", fs_susan, allow_pickle_read=True - ) # fs_susan.ls to list content - - rsc_delete_user_content(fs_susan.api) +@pytest.fixture(scope="function") +def rsc_admin(): + # tears down content after each test + _require_connect() + fs = RsConnectFs(_api()) + rsc_delete_user_content(fs.api) + yield BoardRsConnect("", fs, allow_pickle_read=True) + rsc_delete_user_content(fs.api) -def test_deploy(rsc_short): +def test_deploy(rsc_admin, username): np.random.seed(500) # Load data, model X_df, y = vetiver.mock.get_mock_data() model = vetiver.mock.get_mock_model().fit(X_df, y) - v = vetiver.VetiverModel(model=model, prototype_data=X_df, model_name="susan/model") + pin_name = f"{username}/model" + v = vetiver.VetiverModel(model=model, prototype_data=X_df, model_name=pin_name) board = pins.board_connect( - server_url=RSC_SERVER_URL, api_key=get_key("susan"), allow_pickle_read=True + server_url=RSC_SERVER_URL, api_key=RSC_API_KEY, allow_pickle_read=True ) vetiver.vetiver_pin_write(board=board, model=v) - connect_server = RSConnectServer(url=RSC_SERVER_URL, api_key=get_key("susan")) - assert isinstance(board.pin_read("susan/model"), sklearn.dummy.DummyRegressor) + connect_server = RSConnectServer(url=RSC_SERVER_URL, api_key=RSC_API_KEY) + assert isinstance(board.pin_read(pin_name), sklearn.dummy.DummyRegressor) vetiver.deploy_connect( connect_server=connect_server, board=board, - pin_name="susan/model", + pin_name=pin_name, title="testapi", extra_files=["requirements.txt"], + new=True, ) # get url of where content lives client = RSConnectClient(connect_server) - dicts = client.content_search() + dicts = client.content_list() rsc_api = list(filter(lambda x: x["title"] == "testapi", dicts)) - content_url = rsc_api[0].get("content_url") + content_url = rsc_api[0].get("content_url").rstrip("/") - h = {"Authorization": f'Key {get_key("susan")}'} + h = {"Authorization": f"Key {RSC_API_KEY}", "Content-Type": "application/json"} endpoint = vetiver.vetiver_endpoint(content_url + "/predict") response = vetiver.predict(endpoint, X_df, headers=h) From 563b498b788df78b7c5e5eead0cc9c4955d7efae Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 15:30:15 -0400 Subject: [PATCH 05/11] fix: set Content-Type in predict() for DataFrame input; unmask integration test --- vetiver/server.py | 3 ++- vetiver/tests/test_rsconnect.py | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/vetiver/server.py b/vetiver/server.py index a5be4207..2c6d5679 100644 --- a/vetiver/server.py +++ b/vetiver/server.py @@ -394,8 +394,9 @@ def predict(endpoint, data: Union[dict, pd.DataFrame, pd.Series], **kw) -> pd.Da # TO DO: dispatch if isinstance(data, pd.DataFrame): + headers = {"Content-Type": "application/json", **kw.pop("headers", {})} response = requester.post( - endpoint, data=data.to_json(orient="records"), **kw + endpoint, data=data.to_json(orient="records"), headers=headers, **kw ) # TO DO: httpx deprecating data in favor of content for TestClient elif isinstance(data, pd.Series): response = requester.post(endpoint, json=[data.to_dict()], **kw) diff --git a/vetiver/tests/test_rsconnect.py b/vetiver/tests/test_rsconnect.py index 81897193..520b2b45 100644 --- a/vetiver/tests/test_rsconnect.py +++ b/vetiver/tests/test_rsconnect.py @@ -84,7 +84,7 @@ def test_deploy(rsc_admin, username): rsc_api = list(filter(lambda x: x["title"] == "testapi", dicts)) content_url = rsc_api[0].get("content_url").rstrip("/") - h = {"Authorization": f"Key {RSC_API_KEY}", "Content-Type": "application/json"} + h = {"Authorization": f"Key {RSC_API_KEY}"} endpoint = vetiver.vetiver_endpoint(content_url + "/predict") response = vetiver.predict(endpoint, X_df, headers=h) From bb2f894bc857f4a163d557d7d0f9e8c3e15cf5f1 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 15:36:28 -0400 Subject: [PATCH 06/11] build: run rsc tests via with-connect, drop bespoke dev targets - remove RSC_API_KEYS variable and dev/dev-start/dev-stop/$(RSC_API_KEYS) targets - test-rsc now boots Connect via pinned with-connect (0783dab) - generate requirements.txt via uv run pip freeze, filtering editable/local vetiver entries and appending plain 'vetiver' so Connect can pip-install it - invoke pytest via uv run --with pytest on the specific test file to avoid collection errors in unrelated test files --- Makefile | 23 +++-------------------- 1 file changed, 3 insertions(+), 20 deletions(-) diff --git a/Makefile b/Makefile index 2350c8c2..bc5a2b01 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,6 @@ .PHONY: clean-pyc clean-build clean docs UNAME := $(shell uname) -RSC_API_KEYS=vetiver/tests/rsconnect_api_keys.json ifeq ($(UNAME), Darwin) BROWSER := open @@ -19,9 +18,6 @@ help: @echo "coverage - check code coverage quickly with the default Python" @echo "docs - generate HTML documentation, including API docs" @echo "release - package and upload a release" - @echo "dev - generate Connect API keys" - @echo "dev-start - start up development Connect in Docker" - @echo "dev-stop - stop Connect dev container" clean: clean-build clean-pyc clean-test @@ -47,7 +43,9 @@ test-pdb: clean-test pytest -m 'not rsc_test and not docker' --pdb test-rsc: clean-test - pytest + uv run pip freeze | grep -v '^vetiver' | grep -v '^-e ' > requirements.txt && echo 'vetiver' >> requirements.txt + uvx --from git+https://github.com/posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 \ + with-connect -- uv run --with pytest pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' coverage: coverage report -m @@ -60,21 +58,6 @@ docs doc documentation: release: dist twine upload dist/* -dev: vetiver/tests/rsconnect_api_keys.json - -dev-start: - docker compose up -d - docker compose exec -T rsconnect bash < script/setup-rsconnect/add-users.sh - # curl fails with error 52 without a short sleep.... - sleep 5 - curl -s --retry 10 --retry-connrefused http://localhost:3939 - -dev-stop: - docker compose down - rm -f $(RSC_API_KEYS) - typecheck: pyright -$(RSC_API_KEYS): dev-start - python script/setup-rsconnect/dump_api_keys.py $@ From 66b8b87cd4528c8bd6e250be4b405f2374c9d8d8 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 15:39:33 -0400 Subject: [PATCH 07/11] ci: run Connect tests via with-connect action --- .github/workflows/tests.yml | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 087b5621..e4858e68 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -58,20 +58,13 @@ jobs: uv pip install ".[dev]" uv pip install --upgrade git+https://github.com/rstudio/vetiver-python@${{ github.sha }} echo {{ github.sha }} - - name: run Connect - run: | - docker compose up --build -d - pip freeze > requirements.txt - make dev - cat requirements.txt - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} - - # NOTE: edited to run checks for python package - - name: Run tests - run: | - pytest vetiver -m 'rsc_test' + - name: Run Connect integration tests + uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 + with: + license: ${{ secrets.RSC_LICENSE }} + command: | + pip freeze > requirements.txt + pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' test-docker: name: "Test Docker" From 03c90d0cc0778d39a5aa6a7ddeb977820034c5a5 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 15:42:34 -0400 Subject: [PATCH 08/11] ci: weekly Connect jobs via with-connect; drop Connect from pins-only jobs --- .github/workflows/weekly.yml | 82 ++++++++++++++++++------------------ 1 file changed, 42 insertions(+), 40 deletions(-) diff --git a/.github/workflows/weekly.yml b/.github/workflows/weekly.yml index a2c89ce6..01d72bf7 100644 --- a/.github/workflows/weekly.yml +++ b/.github/workflows/weekly.yml @@ -37,14 +37,6 @@ jobs: python -m pip install '.[all]' python -m pip install --upgrade git+https://github.com/rstudio/pins-python - - name: run Connect - run: | - docker compose up --build -d - make dev - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} - - name: Run tests run: | make test @@ -84,14 +76,6 @@ jobs: python -m pip install .[dev] python -m pip install --upgrade git+https://github.com/rstudio/pins-python - - name: run Connect - run: | - docker compose up --build -d - make dev - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} - - name: Run Tests run: | make test @@ -129,19 +113,28 @@ jobs: python -m pip install --upgrade pip python -m pip install .[dev] python -m pip install --upgrade git+https://github.com/rstudio/rsconnect-python - pip freeze > requirements.txt - - name: run Connect - run: | - docker compose up --build -d - make dev - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} - - - name: Run Tests - run: | - pytest -m 'rsc_test' + - name: Run Connect integration tests + uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 + with: + license: ${{ secrets.RSC_LICENSE }} + command: | + pip freeze | grep -v '^vetiver' | grep -v '^-e ' > requirements.txt + echo 'vetiver' >> requirements.txt + pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' + + - name: Open issue on failure + if: failure() && github.event_name == 'schedule' + uses: actions/github-script@v7 + with: + script: | + github.rest.issues.create({ + owner: context.repo.owner, + repo: context.repo.repo, + title: `Weekly canary failed: ${context.job}`, + body: `The weekly Connect canary failed on run ${context.runId}. This often means a change in rsconnect-python \`main\` broke vetiver. See https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`, + labels: ["ci-failure"] + }) vetiver_latest_rsconnect_latest: name: 'vetiver latest, rsconnect latest' @@ -164,16 +157,25 @@ jobs: python -m pip install --upgrade pip python -m pip install '.[all]' python -m pip install --upgrade git+https://github.com/rstudio/rsconnect-python - pip freeze > requirements.txt - - - name: run Connect - run: | - docker compose up --build -d - make dev - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} - - name: Run Tests - run: | - pytest -m 'rsc_test' + - name: Run Connect integration tests + uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 + with: + license: ${{ secrets.RSC_LICENSE }} + command: | + pip freeze | grep -v '^vetiver' | grep -v '^-e ' > requirements.txt + echo 'vetiver' >> requirements.txt + pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' + + - name: Open issue on failure + if: failure() && github.event_name == 'schedule' + uses: actions/github-script@v7 + with: + script: | + github.rest.issues.create({ + owner: context.repo.owner, + repo: context.repo.repo, + title: `Weekly canary failed: ${context.job}`, + body: `The weekly Connect canary failed on run ${context.runId}. This often means a change in rsconnect-python \`main\` broke vetiver. See https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`, + labels: ["ci-failure"] + }) From 5d343365e944c33f7478560cdb98524e889d59b3 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 15:46:34 -0400 Subject: [PATCH 09/11] chore: remove bespoke Connect test harness (replaced by with-connect) --- docker-compose.yml | 15 -------------- script/setup-rsconnect/add-users.sh | 1 - script/setup-rsconnect/dump_api_keys.py | 21 -------------------- script/setup-rsconnect/rstudio-connect.gcfg | 22 --------------------- script/setup-rsconnect/users.txt | 4 ---- vetiver/tests/rsconnect_api_keys.json | 1 - 6 files changed, 64 deletions(-) delete mode 100644 docker-compose.yml delete mode 100644 script/setup-rsconnect/add-users.sh delete mode 100644 script/setup-rsconnect/dump_api_keys.py delete mode 100644 script/setup-rsconnect/rstudio-connect.gcfg delete mode 100644 script/setup-rsconnect/users.txt delete mode 100644 vetiver/tests/rsconnect_api_keys.json diff --git a/docker-compose.yml b/docker-compose.yml deleted file mode 100644 index 4262845c..00000000 --- a/docker-compose.yml +++ /dev/null @@ -1,15 +0,0 @@ -services: - - rsconnect: - image: rstudio/rstudio-connect:latest - restart: always - ports: - - 3939:3939 - volumes: - - $PWD/script/setup-rsconnect/users.txt:/etc/users.txt - - $PWD/script/setup-rsconnect/rstudio-connect.gcfg:/etc/rstudio-connect/rstudio-connect.gcfg - # by default, mysql rounds to 4 decimals, but tests require more precision - privileged: true - environment: - RSTUDIO_CONNECT_HASTE: "enabled" - RSC_LICENSE: ${RSC_LICENSE} diff --git a/script/setup-rsconnect/add-users.sh b/script/setup-rsconnect/add-users.sh deleted file mode 100644 index 1df8c7f4..00000000 --- a/script/setup-rsconnect/add-users.sh +++ /dev/null @@ -1 +0,0 @@ -awk ' { system("useradd -m -s /bin/bash "$1); system("echo \""$1":"$2"\" | chpasswd"); system("id "$1) } ' /etc/users.txt diff --git a/script/setup-rsconnect/dump_api_keys.py b/script/setup-rsconnect/dump_api_keys.py deleted file mode 100644 index eebef59f..00000000 --- a/script/setup-rsconnect/dump_api_keys.py +++ /dev/null @@ -1,21 +0,0 @@ -import json -import sys - -from pins.rsconnect.api import _HackyConnect - -OUT_FILE = sys.argv[1] - - -def get_api_key(user, password, email): - rsc = _HackyConnect("http://localhost:3939") - - return rsc.create_first_admin(user, password, email).api_key - - -api_keys = { - "admin": get_api_key("admin", "admin0", "admin@example.com"), - "susan": get_api_key("susan", "susan", "susan@example.com"), - "derek": get_api_key("derek", "derek", "derek@example.com"), -} - -json.dump(api_keys, open(OUT_FILE, "w")) diff --git a/script/setup-rsconnect/rstudio-connect.gcfg b/script/setup-rsconnect/rstudio-connect.gcfg deleted file mode 100644 index 98d0add0..00000000 --- a/script/setup-rsconnect/rstudio-connect.gcfg +++ /dev/null @@ -1,22 +0,0 @@ -[Server] -DataDir = /data -Address = http://localhost:3939 - -[HTTP] -Listen = :3939 - -[Authentication] -Provider = pam - -[Authorization] -DefaultUserRole = publisher - -[Python] -Enabled = true -Executable = /opt/python/3.9.5/bin/python - -[RPackageRepository "CRAN"] -URL = https://packagemanager.rstudio.com/cran/__linux__/bionic/latest - -[RPackageRepository "RSPM"] -URL = https://packagemanager.rstudio.com/cran/__linux__/bionic/latest diff --git a/script/setup-rsconnect/users.txt b/script/setup-rsconnect/users.txt deleted file mode 100644 index dd4ec359..00000000 --- a/script/setup-rsconnect/users.txt +++ /dev/null @@ -1,4 +0,0 @@ -admin admin0 -test test -susan susan -derek derek diff --git a/vetiver/tests/rsconnect_api_keys.json b/vetiver/tests/rsconnect_api_keys.json deleted file mode 100644 index 594a58c3..00000000 --- a/vetiver/tests/rsconnect_api_keys.json +++ /dev/null @@ -1 +0,0 @@ -{"admin": "iv6yzMB4GwhQIXjI3lOWG2T21N02C9SU", "susan": "llQqii6bHqV5R9eMN7qpTvJqr3xtnRmy", "derek": "FkakZvdVk2ApxDVkbEn6u2g0eSul4tE6"} From 2962e3beb73da15a6f3423e0f4b3f85215db7b39 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 15:53:13 -0400 Subject: [PATCH 10/11] ci/build: document tests.yml freeze asymmetry; harden Makefile requirements.txt generation --- .github/workflows/tests.yml | 4 ++++ Makefile | 4 ++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index e4858e68..d91b54a1 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -62,6 +62,10 @@ jobs: uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 with: license: ${{ secrets.RSC_LICENSE }} + # Plain freeze (unlike the Makefile/weekly jobs, which filter): here + # vetiver is installed from git+...@, so the freeze yields an + # installable git URL and Connect deploys THIS PR's vetiver. Filtering + # would deploy released vetiver and miss PR-level serving regressions. command: | pip freeze > requirements.txt pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' diff --git a/Makefile b/Makefile index bc5a2b01..1def8eb3 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,6 @@ .PHONY: clean-pyc clean-build clean docs UNAME := $(shell uname) - ifeq ($(UNAME), Darwin) BROWSER := open else @@ -43,7 +42,8 @@ test-pdb: clean-test pytest -m 'not rsc_test and not docker' --pdb test-rsc: clean-test - uv run pip freeze | grep -v '^vetiver' | grep -v '^-e ' > requirements.txt && echo 'vetiver' >> requirements.txt + uv run pip freeze | grep -v '^vetiver' | grep -v '^-e ' > requirements.txt + echo 'vetiver' >> requirements.txt uvx --from git+https://github.com/posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 \ with-connect -- uv run --with pytest pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' From 8f6042d03e645c05cfd617c3bc019012f87321ee Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 16:30:37 -0400 Subject: [PATCH 11/11] rm superpowers --- .../2026-06-30-vetiver-with-connect-tests.md | 447 ------------------ ...iver-connect-test-simplification-design.md | 223 --------- 2 files changed, 670 deletions(-) delete mode 100644 docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md delete mode 100644 docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md diff --git a/docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md b/docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md deleted file mode 100644 index 460dc2f6..00000000 --- a/docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md +++ /dev/null @@ -1,447 +0,0 @@ -# vetiver-python: adopt with-connect for Connect tests — Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Replace vetiver's bespoke Connect-in-Docker test harness with `posit-dev/with-connect`, so the Connect integration test reads credentials from the environment and the repo no longer carries its own docker-compose/user-bootstrap scripts. - -**Architecture:** `with-connect` boots a licensed Connect container, bootstraps a single admin API key, and exports `CONNECT_SERVER` + `CONNECT_API_KEY` to the wrapped command. The test fixture reads those env vars and derives the pin namespace from the authenticated user. CI uses the `with-connect` GitHub Action; local dev uses the `with-connect` CLI via `uvx`. - -**Tech Stack:** Python, pytest, pins, rsconnect-python, Docker, `uv`/`uvx`, GitHub Actions. - -## Global Constraints - -- vetiver supports Python 3.9+; the Connect test job runs on Python 3.9. (`with-connect`'s own 3.13 requirement is satisfied by `uvx`/the Action and does not affect the test interpreter.) -- `with-connect` exposes **only** `CONNECT_SERVER` and `CONNECT_API_KEY` to a wrapped command. Do not rely on any other injected env var, on named users (`susan`/`derek`/`admin`), or on a JSON keys file. -- Never hardcode a Connect username; derive it at runtime via the pins API. -- Pin `with-connect` to commit `0783dabdd24e360e985a4588ce1239c3dc31c542` (no release tags exist yet). Verify this is still the desired ref at execution time with `gh api repos/posit-dev/with-connect/commits/main -q .sha`. -- A valid `rstudio-connect.lic` must be present in the repo root for local runs (gitignored). CI passes the license via the `RSC_LICENSE` secret. -- The Connect test deploys with `extra_files=["requirements.txt"]`, so a `requirements.txt` snapshot of the test env must exist in the working directory before the test runs. - ---- - -### Task 1: Refactor `test_rsconnect.py` to env-var credentials + single user - -**Files:** -- Modify: `vetiver/tests/test_rsconnect.py` (full rewrite of fixtures/helpers) - -**Interfaces:** -- Consumes: `CONNECT_SERVER`, `CONNECT_API_KEY` from the environment (provided by `with-connect`). -- Produces: a single `rsc_admin` board fixture and a `username` fixture; no JSON-key helpers remain. - -- [ ] **Step 1: Replace the module so credentials are read lazily and the user is derived at runtime** - -Rationale for lazy reads: pytest imports every test module during collection even for `make test` (which deselects `rsc_test` by marker). Module-level `os.environ["CONNECT_SERVER"]` would raise `KeyError` during that collection. Use `os.environ.get(...)` at module scope and `pytest.skip(...)` inside the fixtures. - -Replace the entire contents of `vetiver/tests/test_rsconnect.py` with: - -```python -import os - -import numpy as np -import pandas as pd -import pytest -import sklearn - -import pins -from pins.boards import BoardRsConnect -from pins.rsconnect.api import RsConnectApi -from pins.rsconnect.fs import RsConnectFs -from rsconnect.api import RSConnectClient, RSConnectServer - -import vetiver - -RSC_SERVER_URL = os.environ.get("CONNECT_SERVER") -RSC_API_KEY = os.environ.get("CONNECT_API_KEY") - -pytestmark = pytest.mark.rsc_test # noqa - - -def _require_connect(): - if not RSC_SERVER_URL or not RSC_API_KEY: - pytest.skip("CONNECT_SERVER and CONNECT_API_KEY must be set (run via with-connect)") - - -def _api(): - return RsConnectApi(RSC_SERVER_URL, RSC_API_KEY) - - -def rsc_delete_user_content(rsc): - guid = rsc.get_user()["guid"] - content = rsc.get_content(owner_guid=guid) - for entry in content: - rsc.delete_content_item(entry["guid"]) - - -@pytest.fixture(scope="function") -def username(): - _require_connect() - return _api().get_user()["username"] - - -@pytest.fixture(scope="function") -def rsc_admin(): - # tears down content after each test - _require_connect() - fs = RsConnectFs(_api()) - rsc_delete_user_content(fs.api) - yield BoardRsConnect("", fs, allow_pickle_read=True) - rsc_delete_user_content(fs.api) - - -def test_deploy(rsc_admin, username): - np.random.seed(500) - - # Load data, model - X_df, y = vetiver.mock.get_mock_data() - model = vetiver.mock.get_mock_model().fit(X_df, y) - - pin_name = f"{username}/model" - v = vetiver.VetiverModel(model=model, prototype_data=X_df, model_name=pin_name) - - board = pins.board_connect( - server_url=RSC_SERVER_URL, api_key=RSC_API_KEY, allow_pickle_read=True - ) - - vetiver.vetiver_pin_write(board=board, model=v) - connect_server = RSConnectServer(url=RSC_SERVER_URL, api_key=RSC_API_KEY) - assert isinstance(board.pin_read(pin_name), sklearn.dummy.DummyRegressor) - - vetiver.deploy_connect( - connect_server=connect_server, - board=board, - pin_name=pin_name, - title="testapi", - extra_files=["requirements.txt"], - ) - - # get url of where content lives - client = RSConnectClient(connect_server) - dicts = client.content_search() - rsc_api = list(filter(lambda x: x["title"] == "testapi", dicts)) - content_url = rsc_api[0].get("content_url") - - h = {"Authorization": f"Key {RSC_API_KEY}"} - - endpoint = vetiver.vetiver_endpoint(content_url + "/predict") - response = vetiver.predict(endpoint, X_df, headers=h) - - assert isinstance(response, pd.DataFrame), response - assert response.iloc[0, 0] == 44.47 - assert len(response) == 100 -``` - -- [ ] **Step 2: Verify the normal (non-Connect) test suite still collects and runs** - -This confirms the lazy env reads didn't break collection of the module for the default run. - -Run: `pytest -m 'not rsc_test and not docker' -q` -Expected: PASS (no `KeyError` during collection; `test_rsconnect.py::test_deploy` is deselected by the marker). - -- [ ] **Step 3: Verify the Connect test passes under with-connect** - -Requires Docker + `rstudio-connect.lic` in the repo root. - -Run: -```bash -pip freeze > requirements.txt -uvx --from git+https://github.com/posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 \ - with-connect -- pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' -v -``` -Expected: PASS — `test_deploy` deploys the model and asserts `response.iloc[0, 0] == 44.47`. - -- [ ] **Step 4: Commit** - -```bash -git add vetiver/tests/test_rsconnect.py -git commit -m "test: read Connect creds from env, derive pin user at runtime" -``` - ---- - -### Task 2: Point `make test-rsc` at with-connect and remove the dev harness targets - -**Files:** -- Modify: `Makefile` - -**Interfaces:** -- Consumes: nothing new. -- Produces: a `test-rsc` target that no longer depends on `dev`/`dev-start`/`dev-stop` or `RSC_API_KEYS`. - -- [ ] **Step 1: Remove the `RSC_API_KEYS` variable** - -Delete this line near the top of `Makefile`: - -```make -RSC_API_KEYS=vetiver/tests/rsconnect_api_keys.json -``` - -- [ ] **Step 2: Replace the `test-rsc` target** - -Change: - -```make -test-rsc: clean-test - pytest -``` - -to: - -```make -test-rsc: clean-test - pip freeze > requirements.txt - uvx --from git+https://github.com/posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 \ - with-connect -- pytest vetiver -m 'rsc_test' -``` - -- [ ] **Step 3: Remove the `dev`, `dev-start`, `dev-stop`, and `$(RSC_API_KEYS)` targets** - -Delete these blocks from `Makefile`: - -```make -dev: vetiver/tests/rsconnect_api_keys.json - -dev-start: - docker compose up -d - docker compose exec -T rsconnect bash < script/setup-rsconnect/add-users.sh - # curl fails with error 52 without a short sleep.... - sleep 5 - curl -s --retry 10 --retry-connrefused http://localhost:3939 - -dev-stop: - docker compose down - rm -f $(RSC_API_KEYS) -``` - -and - -```make -$(RSC_API_KEYS): dev-start - python script/setup-rsconnect/dump_api_keys.py $@ -``` - -- [ ] **Step 4: Update the `help` target text** - -Delete these three lines from the `help` recipe: - -```make - @echo "dev - generate Connect API keys" - @echo "dev-start - start up development Connect in Docker" - @echo "dev-stop - stop Connect dev container" -``` - -- [ ] **Step 5: Verify the Makefile is well-formed and `test-rsc` runs** - -Run: `make -n test-rsc` -Expected: prints the `pip freeze` + `uvx ... with-connect -- pytest` commands with no `make` errors. - -Run (Docker + license present): `make test-rsc` -Expected: PASS (same green result as Task 1, Step 3). - -- [ ] **Step 6: Commit** - -```bash -git add Makefile -git commit -m "build: run rsc tests via with-connect, drop bespoke dev targets" -``` - ---- - -### Task 3: Convert the `test-connect` CI job to the with-connect Action - -**Files:** -- Modify: `.github/workflows/tests.yml` (the `test-connect` job) - -**Interfaces:** -- Consumes: `secrets.RSC_LICENSE` (license contents). -- Produces: a `test-connect` job that needs no `docker-compose.yml` or `make dev`. - -- [ ] **Step 1: Replace the `run Connect` + `Run tests` steps** - -In the `test-connect` job, replace these two steps: - -```yaml - - name: run Connect - run: | - docker compose up --build -d - pip freeze > requirements.txt - make dev - cat requirements.txt - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} - - # NOTE: edited to run checks for python package - - name: Run tests - run: | - pytest vetiver -m 'rsc_test' -``` - -with: - -```yaml - - name: Run Connect integration tests - uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 - with: - license: ${{ secrets.RSC_LICENSE }} - command: | - pip freeze > requirements.txt - pytest vetiver -m 'rsc_test' -``` - -Note: keep the existing `Install dependencies` step (it still installs `.[dev]` and the vetiver build) so `pip freeze` captures the deployable environment. - -- [ ] **Step 2: Lint the workflow YAML** - -Run: `python -c "import yaml,sys; yaml.safe_load(open('.github/workflows/tests.yml'))"` -Expected: no output (valid YAML). - -- [ ] **Step 3: Commit** - -```bash -git add .github/workflows/tests.yml -git commit -m "ci: run Connect tests via with-connect action" -``` - ---- - -### Task 4: Convert the weekly workflow jobs - -**Files:** -- Modify: `.github/workflows/weekly.yml` - -**Interfaces:** -- Consumes: `secrets.RSC_LICENSE`. -- Produces: the two rsconnect jobs run via the Action; the two pins jobs no longer start Connect. - -- [ ] **Step 1: Remove Connect setup from the two `*_pins_main` jobs** - -`vetiver_main_pins_main` and `vetiver_pypi_pins_main` run `make test` (which is `-m 'not rsc_test and not docker'`), so they do not need Connect. In **both** jobs, delete the `run Connect` step: - -```yaml - - name: run Connect - run: | - docker compose up --build -d - make dev - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} -``` - -Leave their final `make test` step intact. - -- [ ] **Step 2: Convert the two `*_rsconnect_latest` jobs to the Action** - -In `vetiver_pypi_rsconnect_latest` and `vetiver_latest_rsconnect_latest`, replace the `run Connect` + `Run Tests` steps: - -```yaml - - name: run Connect - run: | - docker compose up --build -d - make dev - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} - - - name: Run Tests - run: | - pytest -m 'rsc_test' -``` - -with: - -```yaml - - name: Run Connect integration tests - uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 - with: - license: ${{ secrets.RSC_LICENSE }} - command: | - pip freeze > requirements.txt - pytest vetiver -m 'rsc_test' -``` - -These jobs already run `pip freeze > requirements.txt` in their `Install dependencies` step; the line inside `command:` re-snapshots after all installs and is harmless. The rsconnect-from-`main` install steps stay unchanged — these jobs remain the reverse-dependency canary. - -- [ ] **Step 3: Add a failure notification to the canary jobs** - -Append this step to **both** `*_rsconnect_latest` jobs so rsconnect breakage is surfaced: - -```yaml - - name: Open issue on failure - if: failure() && github.event_name == 'schedule' - uses: actions/github-script@v7 - with: - script: | - github.rest.issues.create({ - owner: context.repo.owner, - repo: context.repo.repo, - title: `Weekly canary failed: ${context.job}`, - body: `The weekly Connect canary failed on run ${context.runId}. This often means a change in rsconnect-python \`main\` broke vetiver. See https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`, - labels: ["ci-failure"] - }) -``` - -- [ ] **Step 4: Lint the workflow YAML** - -Run: `python -c "import yaml,sys; yaml.safe_load(open('.github/workflows/weekly.yml'))"` -Expected: no output (valid YAML). - -- [ ] **Step 5: Commit** - -```bash -git add .github/workflows/weekly.yml -git commit -m "ci: weekly Connect jobs via with-connect; drop Connect from pins-only jobs" -``` - ---- - -### Task 5: Delete the bespoke harness files - -**Files:** -- Delete: `docker-compose.yml` -- Delete: `script/setup-rsconnect/dump_api_keys.py`, `add-users.sh`, `users.txt`, `rstudio-connect.gcfg` (the whole `script/setup-rsconnect/` directory) -- Delete: `vetiver/tests/rsconnect_api_keys.json` (if present / untracked) - -- [ ] **Step 1: Confirm nothing else references the harness paths** - -Run: -```bash -grep -rn "setup-rsconnect\|rsconnect_api_keys\|docker-compose\|docker compose" \ - --include='*.py' --include='*.yml' --include='*.yaml' --include='Makefile' . \ - | grep -v 'docs/superpowers' -``` -Expected: no hits outside `docs/superpowers/` (the design/plan docs). If `script/setup-docker/` appears, that is the *Docker* test path (`test-docker` job) and is unrelated — leave it. - -- [ ] **Step 2: Delete the files** - -Run: -```bash -git rm docker-compose.yml -git rm -r script/setup-rsconnect -rm -f vetiver/tests/rsconnect_api_keys.json -``` - -- [ ] **Step 3: Verify the default test suite still passes** - -Run: `pytest -m 'not rsc_test and not docker' -q` -Expected: PASS. - -- [ ] **Step 4: Verify the Connect test still passes end-to-end** - -Run: `make test-rsc` -Expected: PASS. - -- [ ] **Step 5: Commit** - -```bash -git add -A -git commit -m "chore: remove bespoke Connect test harness (replaced by with-connect)" -``` - ---- - -## Self-Review notes - -- Spec "vetiver-python changes" → Tasks 1–5 cover the fixture, Makefile, `tests.yml`, `weekly.yml`, and all deletions. -- The spec's `requirements.txt` requirement is honored in Task 1 Step 3, Task 2 Step 2, and the `command:` blocks in Tasks 3–4. -- `with-connect` ref is pinned to one SHA used identically in Tasks 1–4 (`0783dabdd24e360e985a4588ce1239c3dc31c542`). -- The lazy-env-read detail (Task 1) prevents a collection-time regression in the default `pytest` run. diff --git a/docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md b/docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md deleted file mode 100644 index feda0813..00000000 --- a/docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md +++ /dev/null @@ -1,223 +0,0 @@ -# Simplifying the vetiver ↔ Connect integration-testing relationship - -**Date:** 2026-06-30 -**Status:** Approved design, pending implementation plan -**Repos affected:** `vetiver-python`, `rsconnect-python` - -## Problem - -The integration-testing relationship between `vetiver-python` and -`rsconnect-python` is both inverted and duplicated: - -- The real dependency is one-way: `vetiver-python` declares - `rsconnect-python>=1.11.0` as a production dependency and calls - `rsconnect.actions.deploy_python_fastapi` / `rsconnect.api.RSConnectServer` / - `RSConnectClient` directly. -- Despite not depending on vetiver, `rsconnect-python` carries vetiver-specific - test baggage: a `vetiver-testing/` harness, `tests/test_vetiver_pins.py`, a - `--vetiver` pytest marker, and a nightly `test-dev-connect` CI job that installs - vetiver and deploys a model. -- Both repos independently maintain a near-identical bespoke Connect-in-Docker - harness (`docker-compose.yml`, `setup-rsconnect/` with `dump_api_keys.py`, - `add-users.sh`, `users.txt`, `rstudio-connect.gcfg`). - -## Goal - -vetiver owns and runs all vetiver↔Connect integration tests in its own repo; -`rsconnect-python` carries no vetiver-specific test code. Both repos use -[`with-connect`](https://github.com/posit-dev/with-connect) — the preferred -shared tool for running Connect in Docker during tests — instead of a hand-rolled -harness. - -## Target architecture - -``` -rsconnect-python vetiver-python - ├ tests/test_main_system_caches.py ├ vetiver/tests/test_rsconnect.py - │ (own integration test; kept) │ (reads env creds, single admin user) - ├ CI: own with-connect job; ├ CI: posit-dev/with-connect Action (pinned) - │ creates publisher user via admin API ├ local: `make test-rsc` → uvx with-connect - └ actions.py compatibility shim └ deletes docker-compose.yml, - (deploy_python_fastapi for vetiver) script/setup-rsconnect/, dump_api_keys.py - -Deleted from rsconnect-python: vetiver-testing/, tests/test_vetiver_pins.py, -the --vetiver marker, the nightly test-dev-connect job, root docker-compose.yml, -and the dev/dev-stop Justfile recipes. -``` - -The reverse-dependency canary ("does new rsconnect break vetiver?") lives in -vetiver's weekly CI matrix, which already installs `rsconnect` from `main` and -runs the integration tests. rsconnect PRs do not get that signal directly; a -failure notification on the weekly canary is recommended so rsconnect maintainers -learn of breakage without watching the schedule. - -### `with-connect` behavior the design relies on - -Verified against `posit-dev/with-connect@main`: - -- Exports exactly two env vars to the wrapped command: `CONNECT_SERVER` and - `CONNECT_API_KEY`. -- Bootstraps a single admin-privileged API key via JWT; it does **not** create - named PAM users. The bootstrapped username is assigned by Connect — derive it - at runtime, never hardcode it. -- CLI `--license` takes a file path (default `./rstudio-connect.lic`). The - Action's `license` input takes license *contents* and writes the file itself. -- Does nothing with `requirements.txt` / manifests. -- Default `version` is `release`; supports `--config`/`config-file`, `-e`/`env`, - `--port`. -- Tool runtime needs Python 3.13 / `uv`, satisfied via `uvx` or the Action. This - is independent of the test matrix: the wrapped `pytest` runs in the CI job's - own Python (3.9/3.10), so there is no Python-version regression. - -## vetiver-python changes - -### Test fixture — `vetiver/tests/test_rsconnect.py` - -- Read `CONNECT_SERVER` and `CONNECT_API_KEY` from `os.environ` instead of the - hardcoded `http://localhost:3939` and the `rsconnect_api_keys.json` file. -- Use the single `with-connect` admin key. Remove `get_key`, `rsc_from_key`, - `rsc_fs_from_key`, and the JSON helpers; the `rsc_short` fixture builds its - `BoardRsConnect` from the one env-provided key. The current test already uses - only one user (`susan`), so a single user covers everything it verifies. -- Do not hardcode a username. Derive it at runtime via - `rsc.get_user()["username"]` and build the pin name as `f"{username}/model"`; - update the corresponding `pin_read`, `deploy_connect(pin_name=...)`, and - `content_search` title lookup. -- The `Authorization: Key ...` header and prediction assertions - (`response.iloc[0, 0] == 44.47`, `len(response) == 100`) are unchanged. - -### Local dev — `Makefile` - -- Remove `dev-start`, `dev`, the `RSC_API_KEYS` variable, and the `dump_api_keys` - invocation. -- `make test-rsc` runs the `with-connect` CLI, e.g. - `uvx --from git+https://github.com/posit-dev/with-connect@ with-connect -- pytest vetiver -m 'rsc_test'`. - License resolves from `./rstudio-connect.lic` per the CLI default. -- `make test` (the `not rsc_test and not docker` run) is unchanged. - -### CI — `.github/workflows/tests.yml` and `weekly.yml` - -- In each Connect job, replace the `docker compose up --build -d` + `make dev` - block with the Action, pinned to a release tag or commit SHA: - - ```yaml - - uses: posit-dev/with-connect@ - with: - version: # release, or a pinned version in the weekly matrix - license: ${{ secrets.RSC_LICENSE }} # existing secret = license CONTENTS - command: | - pip freeze > requirements.txt - pytest vetiver -m 'rsc_test' - ``` - - The `pip freeze > requirements.txt` step stays because the test deploys with - `extra_files=["requirements.txt"]` so Connect can rebuild the model env; this is - unrelated to `with-connect`. -- The weekly matrix keeps its version combinations: vetiver `pypi` × rsconnect - `main`, and vetiver `main` × rsconnect `main`. Only the harness invocation - changes; the rsconnect-version install steps stay. These jobs are the canary. -- Add a failure notification (issue/Slack) on the weekly canary. - -### Deletions - -- `docker-compose.yml` -- `script/setup-rsconnect/` (`dump_api_keys.py`, `add-users.sh`, `users.txt`, - `rstudio-connect.gcfg`) -- `vetiver/tests/rsconnect_api_keys.json` - -## rsconnect-python changes - -### Keep and re-home `tests/test_main_system_caches.py` - -This is rsconnect's own integration test for the `system caches` CLI, unrelated -to vetiver, that currently freeloads on the vetiver harness (it reads -`vetiver-testing/rsconnect_api_keys.json` and runs inside the `test-dev-connect` -job). It must survive the harness removal: - -- It needs two privilege levels — an admin (can list/delete caches) and a - non-admin publisher (must be denied) — so it cannot use only the `with-connect` - admin key. -- Refactor it to read `CONNECT_SERVER`/`CONNECT_API_KEY` from env, and add a - fixture that uses the admin key to create a publisher user via the Connect API - and mint that user's key at runtime (replacing static `get_key("susan")`). -- Give rsconnect its own `with-connect`-based CI job (Action, pinned) that runs - `pytest tests/test_main_system_caches.py`. Confirm during implementation that - the cache add/remove host commands (`ADD_CACHE_COMMAND`/`RM_CACHE_COMMAND`) can - still target the `with-connect`-managed container. - -### Relabel the `actions.py` compatibility shim - -`actions.py` lines 281–442 (`validate_extra_files`, `validate_entry_point`, -`deploy_app`) are kept and reframed as a supported compatibility entry point used -by vetiver: - -- `deploy_python_fastapi` (line 446, called by vetiver's `deploy_connect`) does - `return deploy_app(...)`, and `deploy_app` calls the two `validate_*` functions - — all within this block. `deploy_app` has no other caller. These functions are - load-bearing for vetiver and have no `DeprecationWarning` or changelog notice, - so they are treated as a maintained shim, not dead code. -- Rewrite the block-marker comments (lines 281–285, 441–443) to describe this as - a supported compatibility entry point for `vetiver-python`, dropping the - "deprecated … will be removed in the future" language. -- Optional cleanup: have `deploy_app` call the `bundle.py` `validate_*` functions - instead of the local copies, so vetiver deploys stop emitting spurious "This - method has been moved and will be deprecated" warnings on every call. - -The identically-named `validate_extra_files` / `validate_entry_point` in -`bundle.py` (lines ~1774, ~1845) are the active versions used by the `main.py` -CLI and must not be altered. The `actions.py` copies are reachable only through -`deploy_app`. - -### Deletions - -- `vetiver-testing/` (entire directory). -- `tests/test_vetiver_pins.py`. -- `conftest.py`: the `--vetiver` option and `pytest.mark.vetiver` skip logic. -- `pyproject.toml`: the `vetiver` marker definition and the `vetiver-testing/` - ruff exclude. -- `.github/workflows/main.yml`: the `test-dev-connect` job (replaced by the - rsconnect-own `system caches` job above, which no longer installs vetiver). -- Root `docker-compose.yml` and the `dev` / `dev-stop` recipes in `Justfile` - (lines ~62–71). Their only consumers are the vetiver dev/test path and the - `test-dev-connect` job. The separate `integration-testing/docker-compose.yml` - is unrelated and stays. - -## Sequencing - -The two repos' changes are independent and may land in either order; vetiver -keeps working against rsconnect `main` throughout. Suggested intra-repo order for -rsconnect to keep CI green at every commit: - -1. Update `conftest.py` to drop the `--vetiver` option/marker logic. -2. Refactor and re-home `test_main_system_caches.py` onto the new `with-connect` - job (create publisher user via admin API; env-var creds). -3. Delete `tests/test_vetiver_pins.py`. -4. Update `pyproject.toml` (drop `vetiver` marker + ruff exclude). -5. Replace the `test-dev-connect` job in `main.yml`. -6. Delete the `dev`/`dev-stop` Justfile recipes and root `docker-compose.yml`. -7. Delete `vetiver-testing/`. -8. Relabel the `actions.py` shim comments. - -## Verification items - -- Confirm `with-connect`'s default Connect image/config enables Python content - execution (vetiver's test deploys a FastAPI app). If not, enable via the - Action's `config-file` or `-e`. -- Confirm the bootstrapped admin user can write pins to its own namespace and that - `rsc.get_user()["username"]` returns the expected value. -- Confirm creating a non-admin publisher user via the Connect API with the - bootstrap admin key works, for `test_main_system_caches.py`. -- Confirm the `system caches` host commands can still add/remove caches in the - `with-connect`-managed container. -- Pin a specific `with-connect` Action ref (tag or SHA) and CLI ref; record an - update strategy. - -## Risks / accepted tradeoffs - -- rsconnect PRs lose direct vetiver-breakage signal; it moves to vetiver's weekly - CI, with a failure notification to compensate. -- New external dependency on `posit-dev/with-connect`, mitigated by pinning to a - tag/SHA. Both repos rely on Docker either way. -- vetiver remains coupled to rsconnect's internal `deploy_python_fastapi` shim. - Removing that coupling is out of scope and would require a real deprecation - cycle plus migrating vetiver to the `rsconnect` CLI.