diff --git a/01_getting_started/05_local_modules/.gitignore b/01_getting_started/05_local_modules/.gitignore new file mode 100644 index 0000000..036e0e6 --- /dev/null +++ b/01_getting_started/05_local_modules/.gitignore @@ -0,0 +1,4 @@ +.flash/ +.venv/ +__pycache__/ +*.pyc diff --git a/01_getting_started/05_local_modules/README.md b/01_getting_started/05_local_modules/README.md new file mode 100644 index 0000000..52b83fe --- /dev/null +++ b/01_getting_started/05_local_modules/README.md @@ -0,0 +1,146 @@ +# Local (non-pip) modules + +Factor a Flash endpoint across local Python files that are **not** pip-installable — +a sibling module and a package that live next to the endpoint. This is the +capability added by **SLS-360**. + +``` +05_local_modules/ +├── cpu_worker.py # the @Endpoint — imports the local modules below +├── text_utils.py # local sibling module +└── greetings/ # local package + ├── __init__.py # re-exports render() from .messages (transitive import) + └── messages.py # greeting templates +``` + +`cpu_worker.py` imports `text_utils` and `greetings` **inside the function body**. +That is required on the live path (`flash dev` / `.run()`), where only the +function source plus its local-module closure are shipped to the worker — so +imports must resolve at call time. The same code works unchanged for +`flash deploy`, where the whole project tree is bundled. + +## Requirements + +Local-module bundling landed in the SDK in **`runpod-flash>=1.19.0`** (runpod/flash#352). +On 1.18.0 and earlier the modules are never collected, so every tier below fails. + +## Run it + +```bash +# Live dev server (see the live-path note below re: worker image) +flash dev +# then: curl -s localhost:8888/cpu_worker/runsync -d '{"input": {"name": "Ada", "lang": "es"}}' + +# Or deploy to Runpod +flash deploy +``` + +Expected result shape: + +```json +{"status": "success", "greeting": "HOLA, ADA!", "timestamp": "..."} +``` + +`greeting` is produced by the local modules: `greetings.render()` builds +`"Hola, Ada"`, then `text_utils.shout()` upper-cases and adds `!`. + +## Verifying SLS-360 + +Three tiers, in increasing cost. Tier 1 needs no infra; Tier 2 uses a real +Runpod endpoint; Tier 3 exercises the live inline-shipping path. + +### Tier 1 — build path (local, no infra) + +The build resolves each endpoint's local-import closure, force-includes those +files even when the ignore filter would drop them, and fails loudly if an +endpoint's local import can't be resolved. + +```bash +flash build + +# (a) sibling + package (init AND submodule) are all bundled: +tar tzf .flash/artifact.tar.gz | grep -E 'text_utils|greetings|cpu_worker' +# ./cpu_worker.py +# ./greetings/__init__.py +# ./greetings/messages.py +# ./text_utils.py +``` + +Force-include of an **ignore-dropped** module (this is the specific SLS-360 fix — +a module the ignore rules would silently drop is still bundled because an +endpoint imports it): + +```bash +# a helper whose name matches the built-in test_*.py ignore rule +printf 'SAMPLE_NAMES = ["Ada", "Alan", "Grace"]\n' > test_samples.py +# make the endpoint import it (add `import test_samples` inside greet()) +flash build +tar tzf .flash/artifact.tar.gz | grep test_samples.py # -> ./test_samples.py (rescued) +rm test_samples.py # revert the endpoint edit too +``` + +Loud failure on an unresolved endpoint import (clean error, exit 1, no traceback): + +```bash +# temporarily add `from . import missing_sibling` to cpu_worker.py, then: +flash build ; echo "exit=$?" +# ✗ .../cpu_worker.py: relative import (level=1, module='missing_sibling') +# could not be resolved to a local file under ... +# exit=1 +``` + +### Tier 2 — deploy path (real endpoint) — verified + +The deploy path works with **any** worker image: the local modules are physically +bundled into the artifact and unpacked onto the worker filesystem, so the +endpoint imports them at runtime. This validates SLS-360's build/deploy half +end-to-end on real infra today. + +```bash +flash deploy +curl -X POST https://api.runpod.ai/v2//runsync \ + -H "Authorization: Bearer $RUNPOD_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{"input": {"name": "Ada", "lang": "es"}}' +# -> {"output": {"greeting": "HOLA, ADA!", "status": "success", ...}, "status": "COMPLETED"} +flash undeploy --force # tear down when done +``` + +Verified on a real CPU endpoint: the worker imported the local sibling +(`text_utils`) and package (`greetings`) and returned `"HOLA, ADA!"`. + +> **Payload shape:** the generated queue handler calls the endpoint as +> `greet(**job_input)`, so the function takes typed params (`name`, `lang`) and +> the request nests them under `input`. A single `def greet(input_data: dict)` +> would instead require `{"input": {"input_data": {...}}}`. +> +> **Worker turnover:** a warm worker keeps serving the previous build for a short +> while after `flash deploy`. If a call returns stale behavior, retry until a +> fresh worker (new `workerId`) picks up the new build. + +### Tier 3 — live path (`flash dev` / `.run()`) + +This is the genuinely new capability: local modules are shipped **inline** with +the function and materialized on the worker's `sys.path` before the function +runs. It requires the worker runtime that materializes `FunctionRequest.modules` +(flash-worker PR runpod-workers/flash#100). + +- **Against stock Runpod workers**: the inline modules are ignored by the worker, + so `import text_utils` fails on the worker. This is expected until #100 is + released — as of this writing no published `runpod/flash-cpu` tag carries it + (newest stock tags date to 2026-04-22; #100 merged 2026-07-14). +- **To test now:** build the flash-worker image locally with #100 + (`make build` in the worker repo) and exercise it via the worker's + `make smoketest` with a request carrying `modules`, or via + `flash deploy --preview` (docker-compose) pointed at the local image. +- **After #100 ships:** `flash dev` + a POST to `/cpu_worker/runsync` works + directly, and the greeting is computed by the shipped local modules on the + worker. + +## Why in-function imports + +On the live path only the function's own source is extracted and shipped. Module +top-level imports/constants are not sent (that is the older AE-2308 behavior). +Keeping `import text_utils` / `from greetings import render` inside `greet()` +ensures the resolver discovers them and the worker can import them at call time — +and it stays correct for `flash deploy` too. diff --git a/01_getting_started/05_local_modules/cpu_worker.py b/01_getting_started/05_local_modules/cpu_worker.py new file mode 100644 index 0000000..bdd283f --- /dev/null +++ b/01_getting_started/05_local_modules/cpu_worker.py @@ -0,0 +1,34 @@ +# CPU serverless worker that factors its logic across local (non-pip) modules. +# run locally with: flash dev | deploy with: flash deploy +from runpod_flash import CpuInstanceType, Endpoint + + +@Endpoint( + name="01_05_local_modules", + cpu=CpuInstanceType.CPU3C_1_2, +) +async def greet(name: str = "world", lang: str = "en") -> dict: + """Greeting endpoint whose logic lives in local sibling + package modules. + + The imports are inside the function body on purpose. On the live path + (`flash dev` / `.run()`) only the function source plus its local-module + closure are shipped to the worker, so imports must resolve at call time. + The same code works unchanged for `flash deploy` (the whole tree is bundled). + """ + from datetime import datetime + + import text_utils + from greetings import render + + return { + "status": "success", + "greeting": text_utils.shout(render(name, lang)), + "timestamp": datetime.now().isoformat(), + } + + +if __name__ == "__main__": + import asyncio + + result = asyncio.run(greet(name="Flash", lang="es")) + print(result) diff --git a/01_getting_started/05_local_modules/greetings/__init__.py b/01_getting_started/05_local_modules/greetings/__init__.py new file mode 100644 index 0000000..b94e541 --- /dev/null +++ b/01_getting_started/05_local_modules/greetings/__init__.py @@ -0,0 +1,10 @@ +"""Local package whose __init__ re-exports from a submodule. + +Exercises transitive local-import resolution: the endpoint imports ``greetings``, +whose ``__init__`` imports ``greetings.messages`` — the resolver must pull in both +``greetings/__init__.py`` and ``greetings/messages.py``. +""" + +from .messages import render + +__all__ = ["render"] diff --git a/01_getting_started/05_local_modules/greetings/messages.py b/01_getting_started/05_local_modules/greetings/messages.py new file mode 100644 index 0000000..cf1ae58 --- /dev/null +++ b/01_getting_started/05_local_modules/greetings/messages.py @@ -0,0 +1,12 @@ +"""Package submodule with greeting templates.""" + +TEMPLATES = { + "en": "Hello, {name}", + "es": "Hola, {name}", + "fr": "Bonjour, {name}", +} + + +def render(name: str, lang: str = "en") -> str: + template = TEMPLATES.get(lang, TEMPLATES["en"]) + return template.format(name=name) diff --git a/01_getting_started/05_local_modules/pyproject.toml b/01_getting_started/05_local_modules/pyproject.toml new file mode 100644 index 0000000..f8db563 --- /dev/null +++ b/01_getting_started/05_local_modules/pyproject.toml @@ -0,0 +1,8 @@ +[project] +name = "05_local_modules" +version = "0.1.0" +description = "Factor a Flash endpoint across local (non-pip) modules (SLS-360)" +requires-python = ">=3.10" +dependencies = [ + "runpod-flash>=1.19.0", +] diff --git a/01_getting_started/05_local_modules/text_utils.py b/01_getting_started/05_local_modules/text_utils.py new file mode 100644 index 0000000..05c68da --- /dev/null +++ b/01_getting_started/05_local_modules/text_utils.py @@ -0,0 +1,10 @@ +"""Local sibling module (not pip-installable) that lives next to the endpoint. + +Demonstrates SLS-360: a Flash endpoint can factor logic into local files and +have them ship to the worker. +""" + + +def shout(text: str) -> str: + """Uppercase with emphasis.""" + return f"{text.upper()}!" diff --git a/01_getting_started/README.md b/01_getting_started/README.md index 4a18ac3..906e67f 100644 --- a/01_getting_started/README.md +++ b/01_getting_started/README.md @@ -56,12 +56,27 @@ Managing Python packages and system dependencies. - Version pinning for reproducibility - Dependency optimization strategies +### [05_local_modules](./05_local_modules/) +Factoring endpoint logic across local (non-pip) Python files. + +**What you'll learn:** +- Importing a local sibling module from an endpoint +- Importing a local package whose `__init__` re-exports from a submodule +- Why endpoint imports belong inside the function body +- How the build path resolves and bundles local modules + +**Concepts:** +- Local-module resolution: the endpoint's import closure ships alongside the function +- In-function imports as a requirement on the live path, and a no-op on the deploy path +- Endpoint imports override the build's ignore filter + ## Learning Path 1. Start with **01_hello_world** to understand the basics 2. Explore **03_mixed_workers** for cost optimization and validation patterns 3. Move to **02_cpu_worker** to learn CPU-only patterns 4. Master **04_dependencies** for production readiness +5. Finish with **05_local_modules** to split an endpoint across your own files ## Next Steps diff --git a/README.md b/README.md index 487ce16..40e9343 100644 --- a/README.md +++ b/README.md @@ -61,6 +61,7 @@ Open **http://localhost:8888/docs** to explore all endpoints. | | [02_cpu_worker](./01_getting_started/02_cpu_worker/) | CPU-only worker | | | [03_mixed_workers](./01_getting_started/03_mixed_workers/) | GPU + CPU pipeline | | | [04_dependencies](./01_getting_started/04_dependencies/) | Dependency management | +| | [05_local_modules](./01_getting_started/05_local_modules/) | Local (non-pip) modules and packages | | **ML Inference** | [01_text_to_speech](./02_ml_inference/01_text_to_speech/) | Qwen3-TTS model serving | | **Advanced** | [05_load_balancer](./03_advanced_workers/05_load_balancer/) | HTTP routing with load balancer | | **Scaling** | [01_autoscaling](./04_scaling_performance/01_autoscaling/) | Worker autoscaling configuration | diff --git a/pyproject.toml b/pyproject.toml index 3be2308..5dce648 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -20,7 +20,9 @@ dependencies = [ [dependency-groups] dev = [ - "ruff>=0.8.0", + # Cap below 0.16: ruff 0.16 formats Python code blocks in Markdown by + # default, which reflows all existing docs and breaks the format check. + "ruff>=0.8.0,<0.16", ] [build-system]