Skip to content

Pin dependencies to a verified stack; align container to Python 3.12#2

Open
Alvin-Nahabwe wants to merge 1 commit into
mainfrom
chore/pin-dependencies
Open

Pin dependencies to a verified stack; align container to Python 3.12#2
Alvin-Nahabwe wants to merge 1 commit into
mainfrom
chore/pin-dependencies

Conversation

@Alvin-Nahabwe

Copy link
Copy Markdown
Collaborator

Why

Every dependency in the DL service was unpinned, so a fresh pip install resolved whatever was latest on PyPI. That is the direct root cause of the runtime breakage fixed in #1: the code had drifted behind transformers 5.x, albumentations 2.x, and segmentation-models-pytorch 0.5. A clean install produced a service that crashed on two of its three segmentation model families and on object detection.

What

Pins requirements.txt to the exact versions verified end-to-end — full train → evaluate → save → inference — for all three tasks and both model families per task, on Python 3.12 / CPU torch:

Task Models verified
Image classification ViT
Object detection DETR (HF) + YOLO (Ultralytics)
Semantic segmentation SegFormer (HF) + U-Net (SMP)

Key pins: torch==2.13.0, transformers==5.13.1, datasets==5.0.0, albumentations==2.0.8, segmentation-models-pytorch==0.5.0, ultralytics==8.4.92, numpy==2.4.4, pandas==3.0.3.

  • torch is pinned by version only (no +cpu); on the CUDA base image PyPI serves the matching CUDA build (confirmed 2.13.0 is on the default index).
  • gradio and wandb removed — neither is imported anywhere in the codebase (training sets WANDB_DISABLED=true), and gradio pulls a large unused dependency tree. Re-add with a pin if a demo UI / W&B logging is built.
  • The pinned stack needs Python ≥ 3.11 (numpy 2.4 / pandas 3.0), so the Dockerfile base moves ubuntu22.04/py3.10 → ubuntu24.04/py3.12 and installs into a venv (24.04's system Python is externally-managed, PEP 668). The ubuntu24.04 CUDA base tag was confirmed to exist.

Verification

  • ✅ Pins run all five model paths on CPU / Python 3.12 (train + inference).
  • ⚠️ Not verified here: the CUDA container build itself — needs a GPU host. torch==2.13.0 and the ubuntu24.04 CUDA tag were confirmed to exist, but the image has not been built. Please docker build + one smoke run on the VM before deploying.

Merge order

Independent of #1 (touches only requirements.txt, Dockerfile, .gitignore), but they're two halves of one story: #1 makes the code run on the current libraries; this makes "the current libraries" a fixed, reproducible set. Recommend merging together.

🤖 Generated with Claude Code

Every dependency in the DL service was unpinned, so a fresh install
resolved whatever was latest on PyPI. That is the direct cause of the
runtime breakage fixed in the pipeline PR: the code had drifted behind
transformers 5.x, albumentations 2.x and segmentation-models-pytorch
0.5. A clean `pip install` produced a service that crashed on two of its
three segmentation model families and on object detection.

This pins requirements.txt to the exact versions verified end-to-end
(full train -> evaluate -> save -> inference) for all three tasks and
both model families per task, on Python 3.12 / CPU torch:
image classification (ViT), object detection (DETR + YOLO), semantic
segmentation (SegFormer + SMP U-Net).

- torch is pinned by version only; the CUDA base image pulls the matching
  CUDA build from PyPI.
- gradio and wandb are removed: neither is imported anywhere (training
  sets WANDB_DISABLED=true), and gradio adds a large unused tree.
- The pinned stack needs Python >= 3.11 (numpy 2.4 / pandas 3.0), so the
  Dockerfile base moves from ubuntu22.04/py3.10 to ubuntu24.04/py3.12 and
  installs into a venv (24.04's system Python is externally managed).

Verified: pins run all tasks on CPU/py3.12. NOT yet verified: the CUDA
container build itself (needs a GPU host) — torch==2.13.0 and the
ubuntu24.04 CUDA base tag were confirmed to exist, but the image has not
been built here.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant