From 276258f7c84cbc7c066021fca460bbb83845639a Mon Sep 17 00:00:00 2001 From: zeke <40004347+KAJdev@users.noreply.github.com> Date: Mon, 27 Jul 2026 11:49:00 -0700 Subject: [PATCH 1/2] docs: describe sdk runtime architecture --- README.md | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/README.md b/README.md index 0906a4b..cf77606 100644 --- a/README.md +++ b/README.md @@ -38,6 +38,49 @@ python -m runpod_sdk_runtime.task.runner `RUNPOD_RUNTIME_KIND` selects `queue` or `api` when running the shared bootstrap. +## Cold starts + +Deployment artifacts contain the application source, the matching Runpod SDK, and the +resolved Python environment. The bootstrap follows five phases: + +1. locate the host-provided app tree or extract the artifact +2. attach the artifact environment and source to `PYTHONPATH` +3. verify packages excluded from the artifact are available in the image +4. install resource-level system dependencies +5. start the queue or API worker + +Queue bootstrap failures are returned as structured job errors. API bootstrap failures +are served as HTTP 500 responses so failures remain visible without a crash loop. + +Live workers receive source with each request. Queue and API workers use the shared +execution engine for dependency installation, argument deserialization, execution, +streaming, and result serialization. + +## GPU package contract + +GPU images provide `torch`, `torchvision`, `torchaudio`, and `triton` for deployment +artifacts that exclude those packages by size. The GPU base uses CUDA 12.8 wheels to +support the driver versions available across the worker fleet. The SDK exclusion set +and the GPU base package set form one compatibility contract. + +## Custom images + +The SDK generates a POSIX `dockerArgs` launcher for custom images. The launcher finds a +Python interpreter, installs `runpod-sdk-runtime` when needed, and starts the selected +entrypoint. These environment variables support prerelease and pinned builds: + +- `RUNPOD_RUNTIME_PACKAGE_SPEC` selects the runtime package +- `RUNPOD_PACKAGE_SPEC` selects the Runpod SDK package + +Any PEP 508 package spec accepted by pip can be used, including a version pin or HTTPS +source archive. + +## Compatibility + +`runpod.apps.protocol` defines the function request and response contract shared by the +SDK and runtime. Runtime CI exercises the supported Python matrix against the Apps SDK +branch before runtime releases are published. + ## Development Install the Apps SDK checkout first, then install this project: From 670e4f8252c226575086c294d4197a35977698bd Mon Sep 17 00:00:00 2001 From: zeke <40004347+KAJdev@users.noreply.github.com> Date: Mon, 27 Jul 2026 13:18:07 -0700 Subject: [PATCH 2/2] docs: tighten runtime readme --- README.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/README.md b/README.md index cf77606..2e5da8d 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,6 @@ # Runpod SDK Runtime Worker runtime package and container images for apps built with the Runpod Python SDK. -The runtime has its own release cadence so worker fixes and image updates can ship -independently from SDK releases. ## Runtime images @@ -20,8 +18,7 @@ packages excluded from GPU deployment artifacts. | `runpod/task-gpu:py3.12-latest` | ephemeral GPU task pods | Python 3.10 through 3.14 are published. Release tags use -`py-`, and `latest` tracks the newest release. Manual image -publishes support channels such as `dev`. +`py-`, and `latest` tracks the newest release. ## Package