Expose predicted per-Instance queue pressure and use it in least_load scoring

[zhy1658858023]

Motivation

• Least-load selection relied on instantaneous queue depths which can be zero under low RPS or quickly-drained queues, missing outstanding reservations tracked by the QueueManager; this change surfaces predicted per-Instance pressure so the strategy can avoid instances with pending reserved work.
• The change preserves privacy by exposing only aggregate numeric fields and preserves existing behavior when predicted hints are unavailable.

Description

• Added QueueManager.queue_pressure_snapshot() and kept queue_depth_snapshot() as a compatibility wrapper; the new snapshot returns instantaneous queue fields plus predicted reservation-only aggregates (pending_prefill_count, active_decode_count, next_slot_ready_in_ms, prefill_free_in_ms, predicted_total_backlog_ms) derived from internal reservation state while never exposing task payloads.
• Extended LeastLoadStrategy to include conservative predicted-pressure components (pending_prefill_count, active_decode_count, predicted_total_backlog_ms) with safe default weights (1.0, 0.5, 0.001) and integrated them into the existing weighted scoring while preserving inflight and instantaneous queue components and all fallback behaviors (missing predicted fields → instantaneous behavior; missing queue hints → inflight-only; all unknown → round-robin).
• Wired the same pressure snapshot into selection and debug/provider paths by using queue_pressure_snapshot() in selection, the control-plane queue snapshot provider, and pool-level snapshot aggregation so the select-time hint and /debug/instance_loads share identical data.
• Documentation: updated proxy/README.md to explain the three-layer least_load model and the debug curl example.

Changed files (why each was necessary): proxy/queue/manager.py (new snapshot + timeline-derived aggregates), proxy/strategy/least_load.py (add predicted fields and weights to scoring), proxy/proxy.py (use pressure snapshot for selection/provider/pool aggregation), proxy/resource/instance_pool.py (expose predicted fields in /debug/instance_loads), proxy/README.md (docs). No other modules or runtime behaviors (Scheduler routing, KDN/KVCache logic, IWS, queue release/forwarding, Resource Agent sampling, or pool_resource schema) were modified.

Testing

• Ran Python bytecode checks: PYTHONDONTWRITEBYTECODE=1 python3 -m py_compile proxy/queue/manager.py proxy/strategy/least_load.py proxy/strategy/factory.py proxy/proxy.py proxy/resource/instance_pool.py proxy/resource/p_control_plane.py proxy/queue/*.py test/demo_proxy.py which succeeded.
• Executed unit-style smoke tests against LeastLoadStrategy (lower inflight wins; higher pending_prefill_count loses when inflight equal; higher predicted_total_backlog_ms loses when instantaneous queue depths are zero; missing predicted fields fall back to previous queue-aware scoring; missing queue hints fall back to inflight-only; all-unknown → round-robin; ties use round-robin) and they passed via an isolated import test harness.
• Attempting a full import-based smoke that loads proxy failed here due to the environment missing uvicorn (ModuleNotFoundError), which is an environment limitation rather than a code error; unit smoke tests still validated behavior.

Branch/base info: base branch work (base SHA 17a7f124e4883710fe36016e711d6eb4a24d52d4), head branch codex/issue-123-predicted-instance-pressure (head SHA c54c2101fd63af562b1e4a5ad35bef68a9f74da0).

---

Codex Task