Skip to content

feat: add LiteLLM guardrail adapter (DataFogGuardrail)#154

Merged
sidmohan0 merged 4 commits into
devfrom
feat/litellm-guardrail
Jul 2, 2026
Merged

feat: add LiteLLM guardrail adapter (DataFogGuardrail)#154
sidmohan0 merged 4 commits into
devfrom
feat/litellm-guardrail

Conversation

@sidmohan0

Copy link
Copy Markdown
Contributor

What this is

DataFogGuardrail — an offline PII guardrail for the LiteLLM proxy (DFPY-117, second artifact of the growth-surfaces track after #152). Coded and tested against litellm 1.90.2.

  • pre_call: redacts PII from request messages in place (jane@…[EMAIL_1]) before the request leaves the gateway, or blocks the request (action: block) with HTTP 400
  • post_call: redacts PII from model responses before they reach the client
  • ~31µs per request, in-process — vs. LiteLLM's Beta Presidio integration, which requires a spaCy sidecar service and an HTTP hop per request
  • fail_policy: open (default — a guardrail bug never takes down the gateway) or closed (compliance deployments where unscanned egress is worse than downtime)

Config is three lines of YAML — see examples/litellm_guardrail/.

Review findings addressed (python-reviewer agent, traced against litellm internals)

  • CRITICAL ×2 (PII leakage via exception side channels): engine exceptions can embed scanned text, so the fail paths now raise from None (no __cause__ chain — litellm calls traceback.format_exc() for alerting, which prints chained causes) and log only the exception type, never str(exc). Regression tests assert no PII in the exception chain or log records.
  • HIGH (blocks surfaced as HTTP 500): bare ValueError has no status_code, so litellm mapped blocks to 500 and misclassified them as backend failures. Now raises HTTPException(400), which _is_guardrail_intervention recognizes as a policy intervention.
  • MEDIUM: example config registers mode: ["pre_call", "post_call"] (a bare pre_call never fires the response hook); decisions are recorded into litellm's standard guardrail logging with masked_entity_count; post_call in-place mutation documented as intentional.
  • LOW: non-text content parts (images etc.) are counted and debug-logged so the blind spot is auditable; fail-closed's 500 is now an asserted, intentional contrast to the block-path 400.

Notes

  • Module is opt-in (datafog.integrations.litellm_guardrail) — not imported by datafog core; no new core dependencies. Requires litellm only when used.
  • Same design language as the Claude Code hook (feat: add Claude Code hook adapter (datafog-hook) #152): high-precision default entities, matched PII never echoed anywhere.
  • Next step after merge: upstream a datafog guardrail provider to BerriAI/litellm following their guardrail-provider tutorial, using this as the reference implementation.

Test plan

  • 14 tests passing (tests/test_litellm_guardrail.py): redact/block paths, content-parts form, multi-message, entity config, fail-open/fail-closed, no-PII-in-exceptions/logs regression tests
  • black/isort/flake8 clean
  • Per-request overhead measured: 30.9µs
  • CI green

sidmohan0 added 4 commits July 2, 2026 12:02
Offline PII guardrail for the LiteLLM proxy: pre_call redacts or blocks
PII in request messages before egress, post_call redacts model
responses. In-process, ~31µs per request — no sidecar service.
Blocks surface as HTTP 400 guardrail interventions; engine errors
follow a configurable fail policy (open/closed) and never leak matched
text into logs or exception chains.
… coverage job

Installs litellm in the nlp-advanced CI profile so the DataFogGuardrail
tests execute where coverage thresholds are enforced, instead of being
skipped and dragging patch coverage to zero.
Base litellm does not include fastapi (it's in their proxy extra), so
test collection failed with ModuleNotFoundError in CI. Proxy deployments
always ship fastapi; CI now installs it alongside litellm and the test
module skips gracefully without it.

Adds edge-path tests (no-messages data, non-text content parts, opaque
responses, observability failure) bringing the adapter to 100% line
coverage.
@sidmohan0 sidmohan0 merged commit cf470a4 into dev Jul 2, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant