feat(aicore/fallback): add opt-in model fallback for Orchestration v2

[lenin-ribeiro]

Disclaimer: Do not include SAP-internal or customer-specific information in this PR (e.g. internal system URLs, customer names, tenant IDs, or confidential configurations). This is a public repository.

Stacked on feat/orchestration-filtering. This branch builds on the filtering PR. Open against that branch as the base to keep the diff focused; retarget to main once filtering merges.

Description

Adds opt-in model fallback for SAP AI Core Orchestration v2 to the sap_cloud_sdk.aicore module.

Orchestration v2 supports preference-ordered fallback module configurations: when the primary call fails (model unsupported in region, 429, 408, or any 5xx — and unsupported-model only for streaming), the server transparently retries with the next preference. The underlying litellm SAP provider already builds body["config"]["modules"] as a list when fallback_sap_modules is present in optional_params; what was missing was the SDK-side ergonomic surface and the response-side visibility into which preferences were skipped.

This PR introduces:

• FallbackModel, FallbackConfig — typed dataclasses for declaring per-preference model + params + version.
• set_fallbacks(config) — single entry point mirroring set_filtering(). Fallback is opt-in: set_aicore_config() does NOT activate it. Developers either call set_fallbacks(...) programmatically or set AICORE_FALLBACK_ENABLED=true (with AICORE_FALLBACK_MODELS or AICORE_FALLBACK_CONFIG) and call set_fallbacks() with no args.
• response.intermediate_failures — when the fallback path fires, the per-preference failure list from the orchestration response is surfaced as an attribute on the returned ModelResponse. None when the primary succeeded, useful as a quick check.
• Patch composition — the existing FilteringOrchestrationConfig is renamed to OrchestrationPatchConfig (alias kept for back-compat) and now owns both filtering and fallback. One install/uninstall path, no ordering issues.
• Filtering broadcast across fallbacks — when both filtering and fallback are active, the filtering config is applied to every module entry on the wire (previously modules[0] only). Consistent SDK-side default; if a fallback should run unfiltered, the developer can call disable_filtering() before the call.

Related Issue

N/A — additive feature, no issue tracked

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update
• Code refactoring

How to Test

Unit tests (no live credentials required)

uv run python -m pytest tests/aicore -v

Expect 142 passed, 8 skipped (the 8 skips are integration scenarios waiting for live env vars).

Integration tests (requires AI Core access)

1. Copy .env_integration_tests.example to .env_integration_tests and fill in the AI Core creds.
2. Set the new fallback secrets:

   AICORE_FALLBACK_TEST_PRIMARY_MODEL=sap/this-model-does-not-exist
   AICORE_FALLBACK_TEST_FALLBACK_MODEL=sap/mistralai--mistral-small-instruct

   The primary should be a model name the orchestration server reports as unsupported in your region — a genuinely nonexistent name works and avoids depending on transient 5xx errors.
3. Run:

   uv run python -m pytest tests/aicore/integration/test_fallback_bdd.py -v

4. Expected: 4 scenarios pass (primary succeeds, primary unsupported → fallback used, filtering + fallback composition, streaming + fallback).

Manual smoke

from sap_cloud_sdk.aicore import (
    FallbackConfig, FallbackModel, set_aicore_config, set_fallbacks,
)
from litellm import completion

set_aicore_config()
set_fallbacks(FallbackConfig([
    FallbackModel(model="sap/mistralai--mistral-small-instruct"),
]))

response = completion(
    model="sap/gpt-4o",  # or any potentially-unsupported model
    messages=[{"role": "user", "content": "Translate 'hello' to German."}],
)
print(response.choices[0].message.content)
print(getattr(response, "intermediate_failures", None))
# None if primary worked; list of failure dicts if fallback fired.

Checklist

• I have read the Contributing Guidelines
• I have verified that my changes solve the issue
• I have added/updated automated tests to cover my changes
• All tests pass locally (pytest tests/aicore, ruff check, ruff format --check, ty check)
• I have verified that my code follows the Code Guidelines
• I have updated documentation (if applicable) — aicore/user-guide.md gains a "Model Fallback (opt-in)" section
• I have added type hints for all public APIs
• My code does not contain sensitive information (credentials, tokens, etc.)
• I have followed Conventional Commits for commit messages

Breaking Changes

None for users of sap_cloud_sdk.aicore public APIs. All existing names (FilteringOrchestrationConfig, _install, etc.) remain importable via aliases.

There is one user-visible behavioural change that only affects users who use filtering AND fallback together (an impossible combination on main today since fallback didn't exist): when both are active, the filtering configuration now applies to every module entry (primary + every fallback), not just modules[0]. This is the safe-by-default semantic — to run a fallback unfiltered, explicitly disable_filtering() before that call. Documented in the user guide.

Additional Notes

Design choices (selected via user Q&A during planning)

• API style — patch-based, mirrors filtering. Single set_fallbacks() entry point, global state, no disable_fallbacks() (developers either opt in via set_fallbacks() or never call it; runtime clearing is set_fallbacks(None)).
• One subclass for both concerns. OrchestrationPatchConfig handles filtering injection AND fallback injection in one transform_request. Single install/uninstall lifecycle. Idempotent.
• Env vars opt-in. AICORE_FALLBACK_ENABLED defaults to false (unlike filtering, which is on by default after set_aicore_config()). Two-tier schema: AICORE_FALLBACK_MODELS (comma list, simple case) + AICORE_FALLBACK_CONFIG (JSON, full per-model config; takes precedence).
• intermediate_failures on the response object. Pydantic ModelResponse uses extra="allow", so we can attach the field directly. Accessed via getattr(response, "intermediate_failures", None).

v1 limitations (documented)

• intermediate_failures is surfaced for non-streaming responses only. Capturing the field from SAPStreamIterator chunks requires deeper changes to litellm internals and is deferred to a future iteration. The streaming integration test asserts that fallback still fires correctly server-side; it doesn't assert intermediate_failures.

Telemetry

Added Operation.AICORE_SET_FALLBACKS = "set_fallbacks". The set_fallbacks entry point is decorated with @record_metrics(Module.AICORE, Operation.AICORE_SET_FALLBACKS) per docs/GUIDELINES.md.

Files added / changed

NEW:
  src/sap_cloud_sdk/aicore/fallback/__init__.py
  src/sap_cloud_sdk/aicore/fallback/fallback.py
  tests/aicore/fallback/__init__.py
  tests/aicore/fallback/unit/__init__.py
  tests/aicore/fallback/unit/test_fallback_config.py
  tests/aicore/fallback/unit/test_patch.py
  tests/aicore/fallback/unit/test_set_fallbacks.py
  tests/aicore/integration/fallback.feature
  tests/aicore/integration/test_fallback_bdd.py

MODIFIED:
  src/sap_cloud_sdk/aicore/__init__.py            # export FallbackModel/FallbackConfig/set_fallbacks; docstring
  src/sap_cloud_sdk/aicore/filtering/filters.py   # OrchestrationPatchConfig rename + split _install + broadcast filtering + attach intermediate_failures
  src/sap_cloud_sdk/aicore/user-guide.md          # new "Model Fallback (opt-in)" section
  src/sap_cloud_sdk/core/telemetry/operation.py   # AICORE_SET_FALLBACKS
  tests/aicore/integration/conftest.py            # extend with fallback fixtures + clean-skip behaviour
  tests/core/unit/telemetry/test_operation.py     # expected enum count 152 → 153
  .env_integration_tests.example                  # AICORE_FALLBACK_TEST_PRIMARY_MODEL / _FALLBACK_MODEL