The next-generation API gateway — where Kong/Zuul meet AI. Enterprise-grade request routing, traffic shaping, and policy management powered by pluggable rate-limiting algorithms, Redis-backed distributed counters, Kafka event streaming, and LLM-driven natural-language configuration.
Stop writing YAML configs. Chat with your gateway instead. "Route all premium users to the new service cluster" — done. "Block requests over 10MB for non-VIP clients" — handled.
- 🧠 AI-Native Configuration: Configure routing policies and rate limits in plain English via MCP + LLM. No more YAML hell
- ⚡ Algorithm Flexibility: Token Bucket, Sliding Window, or custom algorithms — swap anytime, zero downtime
- 🔄 Distributed by Default: Redis-powered counters work seamlessly across 100+ instances with perfect consistency
- 📡 Event-Driven Architecture: Every API call streamed to Kafka for audit, analytics, compliance, and debugging
- 🎯 Zero-Trust Ready: JWT auth, mTLS support, encrypted service-to-service communication out of the box
- 📊 Production Observability: Real-time New Relic dashboards, health checks, and performance metrics
- 🚀 Cloud-Native Stack: Spring Boot 3.5 + Spring Cloud, containerized, Kubernetes-ready, horizontally scalable
See the full system architecture:
The gateway acts as the central entry point, orchestrating traffic through authentication, routing, rate-limiting, and auditing layers.
Central request router and rate-limiter implementation. Handles:
- Request routing to upstream services
- Rate limiting with pluggable algorithms
- Authentication and authorization enforcement
Architecture Details:
Three algorithms out-of-the-box:
- Token Bucket: Smooth traffic shaping with burst capacity
- Sliding Window: Precise request counting over time
- Custom: Extend with your own algorithm
All counters distributed across Redis for consistency across instances.
Centralized configuration management with service discovery integration for dynamic routing rules.
Exposes the Model Context Protocol interface for AI assistants to:
- Query current gateway configuration
- Modify routing rules in natural language
- Create rate limiting policies via conversation
Event Pipeline:
All API requests logged asynchronously to Kafka for:
- Compliance auditing
- Performance monitoring
- Root cause analysis
Real-time Observability:
Integrated monitoring with New Relic for dashboards and alerts.
| Service | Purpose |
|---|---|
| gateway-service | Core API gateway (routing, rate-limiting, auth) |
| config-service | Centralized rule and policy management |
| discovery-service | Service registry and load balancing |
| audit-service | Processes Kafka events for compliance |
| mcp-service | LLM integration for natural language config |
| ai-admin-service | Admin interface for AI-powered gateway management |
| user-service | Demo service (user auth backends) |
| order-service | Demo service (protected resource) |
- Java 21+
- Docker & Docker Compose
- Redis
- Kafka
- (Optional) OpenAI API key for MCP+LLM features
docker-compose up -dThis starts:
- Gateway Service (port 8080)
- Config Service (port 8081)
- Discovery Service (port 8761)
- User Service (port 8082)
- Order Service (port 8083)
- Redis, Kafka, PostgreSQL
curl -H "Authorization: Bearer <token>" \
http://localhost:8080/api/ordersThe gateway will:
- Authenticate the request
- Apply rate limiting (default: 100 req/min per user)
- Route to the order-service
- Log the event to Kafka
- Return the response
# config-service application.yml
gateway:
rateLimit:
algorithm: TOKEN_BUCKET # or SLIDING_WINDOW
capacity: 100 # tokens/time window
refillRate: 60 # tokens/minute
enabled: true# Chat with the gateway via MCP
"Set rate limit to 1000 requests per minute for premium users"
"Route /api/v2/* to the new service-v2 cluster"
"Enable rate limiting for anonymous users"Access dashboards at:
- Gateway Health: http://localhost:8080/actuator/health
- Observability: New Relic (configured via
newrelic.yml) - Kafka Audit Events: Check audit-service logs
- JWT Authentication on all routes
- Rate limiting prevents DDoS attacks
- Audit logging tracks all requests
- Service-to-service mTLS-ready (see
k8s/for Istio config)
- Kafka Audit Pipeline: See
docs/images/architecture/kafka_audit_pipeline.png - Infrastructure & AI Integration: See
docs/images/architecture/infrastructure_and_AI.png - API Gateway Design: See
docs/images/architecture/api_gateway_hld.png - Request/Response Sequences: See
docs/images/architecture/sequence_diagram.png
# Build all services
mvn clean package
# Run tests
mvn test
# Build Docker images
mvn dockerfile:build
# Deploy to Kubernetes
kubectl apply -f k8s/Questions? Check the architecture diagrams in docs/images/architecture/ or explore individual service READMEs.