[Feature]: request queueing / concurrency limit for multi-user deployments

[zeeid]

Problem / motivation

We run mcpproxy as a shared HTTP service (systemd, headless mode) accessed by multiple concurrent users/agents. mcp_config.json currently has no way to cap concurrent requests or queue excess ones - bursts of simultaneous tool calls go straight to upstream servers, including stateful ones (e.g. MySQL-backed MCP servers), risking overload on those upstream resources when usage scales beyond a single user.

Proposed solution

Add a config option (e.g. in mcp_config.json) to set:

• max_concurrent_requests (global and/or per-upstream-server)
• queue_size / queue_timeout for requests beyond the concurrency limit, so excess requests wait in a queue instead of failing or overwhelming the upstream

This would let a single mcpproxy instance safely serve multiple concurrent users without needing external infrastructure for basic overload protection.

Alternatives considered

Currently working around this with an nginx reverse proxy (limit_req/limit_conn) in front of mcpproxy, and separately capping max_user_connections at the database level for MySQL-backed MCP servers. This works for HTTP-facing setups, but native support would help stdio-based upstream servers too, where reverse-proxy-level limits don't apply.

Which edition?

Both / not sure

Pre-submission checklist

• I searched existing issues and discussions for a similar request

[Dumbris]

Thanks — good timing, we just finished a design pass on this. Plan is native limits in the proxy, pretty much what you asked for:

• max_concurrent_requests (global + per-server), queue_size, queue_timeout in mcp_config.json
• extra requests wait in a bounded FIFO queue; when it's full (or waits too long) they're rejected with a clear "server busy, retry" tool result — REST gets 429 + Retry-After
• the limit sits at the single choke point all tool calls go through, so stdio upstreams are covered too (including calls made from code_execution)
• 0 = unlimited, so nothing changes unless you opt in; hot-reloadable so you can tune limits without restarting

That should remove the need for the nginx workaround, especially for stdio servers. Will post the PR here when it's up.