io.github.jaimenbell/vllm-ops-mcp

vllm-ops-mcp

Read-only ops/health MCP server for a local, WSL2/systemd-managed vLLM server -- liveness vs. real-completion health tiers, GPU/VRAM status, systemd service status, and live serve-flag introspection. Built to the same standard as mcp-factory and desktop-mcp, and other local tooling in this operator's portfolio: own pyproject, own fastmcp server, honest README, real test suite, no product code before a spec was confirmed.

This is not a container-lifecycle manager. It assumes vLLM already runs as a bare-metal systemd unit inside WSL2 (this operator's actual setup -- see vllm-autostart), not inside Docker/Podman.

Quickstart

// ~/.claude.json (or any MCP-host stdio client config)
{
  "mcpServers": {
    "vllm-ops-mcp": {
      "type": "stdio",
      "command": "C:\\Users\\<you>\\projects\\vllm-ops-mcp\\.venv\\Scripts\\python.exe",
      "args": ["C:\\Users\\<you>\\projects\\vllm-ops-mcp\\run_server.py"]
    }
  }
}

Tools (Phase 1 -- all read-only, no gating required beyond a throughput cap)

Tool	What it does	What it can't do
check_health(deep: bool = False)	deep=False: GET /v1/models liveness check (same signal as llm_router.LocalLLM.is_healthy()). deep=True: additionally fires one minimal real /v1/chat/completions call (mirrors check-vllm.cmd's two-stage probe) to confirm the server actually generates, not just reports a model loaded.	Cannot fix a degraded server -- reports ground truth only. deep=True is rate-limited (default 20/min, see below).
list_models	/v1/models passthrough.	No detail beyond what vLLM's API itself exposes.
test_completion(prompt, max_tokens=16)	On-demand real completion with a caller-supplied prompt, for manual sanity checks.	Rate-limited (same bucket as check_health(deep=True)); refuses if the liveness check fails first (no point firing a completion at a server that isn't even listening). max_tokens is clamped server-side to 1024 (config.MAX_TEST_COMPLETION_TOKENS) and the prompt is capped at 8000 chars (config.MAX_TEST_COMPLETION_PROMPT_CHARS, oversized prompts are rejected, not truncated) -- the rate limiter only bounds call frequency, not the cost of a single call.
get_gpu_status	nvidia-smi wrapper: per-GPU VRAM used/total + utilization%, plus per-process VRAM via --query-compute-apps.	Per-process VRAM attribution does not work on this operator's actual WSL2 setup -- live-verified empty even while the vLLM process holds 13.7GB (see Limitations). Treat processes as best-effort/often-empty, not a reliable per-PID breakdown.
get_service_status	systemctl show <unit> -p ... (structured property output, not free-text parsing) via wsl -d <distro> from the Windows host, or native systemctl if this server itself runs inside WSL2/Linux. Returns load/active/sub state, restart count, main PID, last-active timestamp.	Read-only -- cannot start/stop/restart the unit. No computed uptime duration (deliberately -- see Limitations, clock-discipline note).
get_serve_config	Read-only launch-flag introspection. Prefers the LIVE process's actual argv (/proc/<pid>/cmdline of the systemd unit's MainPID -- ground truth of what's running right now), falling back to the static exec script (local mirror via VLLM_OPS_MCP_SERVE_CONFIG_PATH, else a live cat of the WSL-side exec script) only when the process isn't up.	vLLM doesn't expose its own launch flags over the API, so this never queries the server itself -- it's always shelling out. Exec-script fallback parsing is best-effort shell-text parsing (less precise than the live-argv path).

Future extension: restart_service (Phase 2, not built)

Deliberately not implemented in this release. Per the spec: vLLM is shared infra other bots/tools depend on (options-bot, qwen_cli.py callers, DeerFlow fallback), so a restart tool needs its own gating design (off by default, VLLM_OPS_MCP_ENABLE_RESTART=1, refuse to touch a healthy instance unless force=True, rate-capped) before it ships -- not retrofitted onto a read-only MVP. Tracked as a v2 idea, not a current capability. If built, it should wrap run-vllm.cmd's exact recipe (systemctl start vllm only, never blind stop/restart) rather than reinvent it.

The differentiator: probe HTTP, not the listener

Every health signal in this server comes from an actual HTTP request to the OpenAI-compatible API (urllib.request, not a socket/connect check). This is deliberate, not incidental: Get-NetTCPConnection and other listener-based checks miss WSL2-mirrored-networking-relayed ports entirely -- a genuinely healthy 127.0.0.1:8000 can be invisible to a listener probe run from the Windows host. check-vllm.cmd already embodies this lesson; this server generalizes it into two tiers (liveness vs. real generation) that nothing else in this operator's fleet exposes as MCP tools.

Security

Read this before you register vllm-ops-mcp. Honesty about blast radius is the point.

• This server shells out to nvidia-smi, systemctl (via wsl -d), and reads /proc/<pid>/cmdline. All six tools are read-only -- nothing here starts, stops, restarts, or kills a process, and no tool accepts an argument that gets passed to a shell unescaped (unit/distro names come from server-side config, not caller input; shlex.quote wraps the one caller-adjacent path that does reach a shell string, the unit name in systemctl show).
• check_health(deep=True) and test_completion fire a real inference call against a shared GPU server. Rate-limited (default 20 calls/min, VLLM_OPS_MCP_DEEP_RATE_LIMIT_PER_MIN) specifically because this vLLM instance is shared infra other bots/tools depend on -- a caller hammering this MCP could starve or slow other consumers even though every call individually looks "read-only."
• get_serve_config's live-process path reads /proc/<pid>/cmdline, which can include secrets if any were ever passed as CLI flags (none are, in this operator's actual setup -- the model path and flags are all non-secret; auth, if any, is environment-injected inside the exec script, not passed as an argv flag). Defense-in-depth: any argv entry whose preceding flag name matches a known-sensitive pattern (token, key, secret, password, passwd, credential -- config. SENSITIVE_ARGV_FLAG_PATTERNS) has its value redacted to [REDACTED] before being returned, in both the structured argv/flags fields and raw_text; the flag name itself is left visible so the shape is still diagnosable. This applies across all three sources (live_process, exec_script, local_config_mirror).
• No tool has any write/mutate capability in this release. There is no env var that turns one on -- restart_service doesn't exist in this codebase yet (see Future extension above), so there's nothing to accidentally enable.

Honest-capabilities table

Claim	Implementation	Verified by
Liveness health tier (/v1/models)	vllm_ops_mcp/probes.py::_probe_models, check_health	tests/test_probes.py::TestCheckHealthLiveness; live: tests/test_live_smoke.py::test_live_check_health_liveness
Deep/real-completion health tier	vllm_ops_mcp/probes.py::_probe_completion, check_health(deep=True)	tests/test_probes.py::TestCheckHealthDeep; live: test_live_check_health_deep
Model list passthrough	probes.py::list_models	tests/test_probes.py::TestListModels; live: test_live_list_models
On-demand completion sanity check	probes.py::test_completion	tests/test_probes.py::TestTestCompletion; live: test_live_test_completion
GPU/VRAM status via nvidia-smi	probes.py::get_gpu_status	tests/test_probes.py::TestGetGpuStatus; live: test_live_get_gpu_status
Per-process VRAM attribution	probes.py::get_gpu_status (--query-compute-apps)	Unit-tested with a mocked non-empty response; live-verified to return EMPTY on this operator's actual WSL2 setup -- see Limitations
systemd unit status via structured systemctl show	probes.py::get_service_status	tests/test_probes.py::TestGetServiceStatus; live: test_live_get_service_status
Live-process launch-flag introspection via /proc/<pid>/cmdline	probes.py::get_serve_config, parse_launch_flags	tests/test_probes.py::TestGetServeConfig, TestParseLaunchFlags; live: test_live_get_serve_config -- live-confirmed it recovers flags (--enable-auto-tool-choice --tool-call-parser qwen3_xml) that were not in this repo's own grounding spec, proving the live-argv approach beats trusting a static doc
Exec-script fallback when the unit is down	probes.py::get_serve_config, _parse_exec_script_text	tests/test_probes.py::TestGetServeConfig::test_falls_back_to_exec_script_when_service_down
Local config-mirror override path	probes.py::get_serve_config + config.serve_config_override_path	tests/test_probes.py::TestGetServeConfig::test_uses_local_override_file, test_override_file_missing_reports_error
Sensitive-flag-value redaction in get_serve_config	probes.py::redact_argv, redact_raw_text, config.SENSITIVE_ARGV_FLAG_PATTERNS	tests/test_probes.py::TestGetServeConfig::test_live_process_redacts_sensitive_flag_values, test_exec_script_raw_text_redacts_sensitive_flag_values, test_local_override_file_redacts_sensitive_flag_values
Deep-inference rate limiting (shared-infra protection)	config.py::TokenBucket, DeepRateLimiter, check_deep_rate_limit	tests/test_config.py::TestTokenBucket, TestDeepRateLimiter
WSL-vs-native shell-out abstraction	probes.py::_shell_args, config.runs_in_wsl	tests/test_config.py::TestRunsInWsl; tests/test_probes.py::TestGetServiceStatus::test_uses_wsl_prefix_on_windows, test_no_wsl_prefix_when_native
All 6 tools registered, no Phase-2 tool present	vllm_ops_mcp/server.py	tests/test_server.py

Limitations (read before relying on this)

• Per-process VRAM attribution is unreliable under this operator's actual WSL2 GPU-passthrough config. get_gpu_status's live smoke test (real nvidia-smi.exe against the real running vLLM process) returned an empty processes list even though the same call's gpus[0]. memory_used_mib correctly showed 13.7GB in use by that process. This is a documented nvidia-smi --query-compute-apps limitation under some WSL2/passthrough configurations, not a bug in this server's parsing -- the tool reports the empty result honestly (ok: true, processes: []) rather than fabricating attribution. Don't rely on processes for VRAM-overshoot root-causing on this setup; gpus[].memory_used_mib is the reliable signal.
• No computed service uptime. get_service_status returns the raw ActiveEnterTimestamp string from systemctl (already timezone-labeled, e.g. MDT) rather than computing an elapsed duration server-side -- deliberately, per this operator's own clock-discipline rule (never state an elapsed/ETA time from estimation; artifact timestamps are often UTC while local is UTC-6/-7). Compute the delta at the call site if you need it, with an explicit timezone conversion.
• get_serve_config's exec-script fallback is best-effort shell-text parsing, not an exact argv array -- it handles \ line continuations and basic quoting via shlex.split, but a sufficiently unusual exec script could parse incorrectly. The live-process path (/proc/<pid>/ cmdline) does not have this limitation (it's the kernel's exact argv, null-separated) and is always preferred when the unit is up.
• WSL2-specific by default, but the seam is explicit. get_service_status and get_serve_config's live-process path assume this server runs on the Windows host and shells into WSL2 via wsl -d <distro>. Set VLLM_OPS_MCP_RUNS_IN_WSL=1 if this server process itself runs inside WSL2/Linux natively (native systemctl/cat, no wsl -d hop) -- both paths are unit-tested (TestGetServiceStatus::test_uses_wsl_prefix_on_windows / test_no_wsl_prefix_when_native), but only the Windows-host path has been live-verified against this operator's actual setup.
• Single vLLM instance, single GPU, this operator's setup. No multi-instance/multi-GPU aggregation; get_gpu_status returns whatever nvidia-smi --query-gpu enumerates, which is every GPU visible to the process, not scoped to "the GPU vLLM is using" if more than one exists.
• No lifecycle control whatsoever in this release. See "Future extension" above -- this is intentional, not an oversight.

Env vars

Var	Effect	Default
VLLM_OPS_MCP_BASE_URL	OpenAI-compatible base URL	http://127.0.0.1:8000/v1 (explicit 127.0.0.1, not localhost -- see qwen_cli.py's IPv6-hang footgun note)
VLLM_OPS_MCP_MODEL	served-model-name, used in completion payloads	qwen3-14b
VLLM_OPS_MCP_WSL_DISTRO	WSL distro hosting vLLM	Ubuntu-22.04
VLLM_OPS_MCP_SERVICE_UNIT	systemd unit name	vllm
VLLM_OPS_MCP_EXEC_SCRIPT_PATH	WSL-side path to the serve exec script (fallback source)	~/vllm-systemd-exec.sh
VLLM_OPS_MCP_SERVE_CONFIG_PATH	local file mirror of the exec script; if set, get_serve_config reads it directly instead of shelling into WSL as a fallback source	unset
VLLM_OPS_MCP_NVIDIA_SMI_PATH	nvidia-smi binary override	nvidia-smi (Windows host) / nvidia-smi.exe (native WSL2)
VLLM_OPS_MCP_RUNS_IN_WSL	1 if this server process itself runs inside WSL2/Linux (skips the wsl -d hop)	auto-detected via sys.platform
VLLM_OPS_MCP_DEEP_RATE_LIMIT_PER_MIN	cap on check_health(deep=True) + test_completion calls/min	20
VLLM_OPS_MCP_LIVE	1 to run real-infra smoke tests (see Testing)	unset (skip)

Usage examples

// A tool call from the MCP host, illustrative -- not a shell command.
{"tool": "check_health", "arguments": {"deep": true}}
// -> {"ok": true, "status": "up", "tier": "deep", "model_id": "qwen3-14b",
//     "latency_ms": 32.0, "completion_s": 0.218, "error": ""}

{"tool": "get_serve_config", "arguments": {}}
// -> {"ok": true, "source": "live_process", "pid": 5645,
//     "flags": {"model": "Qwen/Qwen3-14B-AWQ", "served_model_name": "qwen3-14b",
//               "port": "8000", "max_model_len": "8192",
//               "gpu_memory_utilization": "0.72", "max_num_seqs": "8",
//               "enable_auto_tool_choice": true, "tool_call_parser": "qwen3_xml", ...}}

// rate limit exceeded (21st deep call within a minute):
{"tool": "test_completion", "arguments": {"prompt": "hi"}}
// -> {"ok": false, "error": {"type": "rate_limited", "group": "deep_inference",
//     "tool": "test_completion", "limit_per_min": 20, "retry_after_s": 4.87}}

Testing

CI (.github/workflows/ci.yml) runs this suite on every push/PR (ubuntu-latest -- the unit suite mocks HTTP/subprocess, no live server/GPU/WSL required) and fails the build if the Tests badge above drifts from what the suite actually reports -- see scripts/check_readme_counts.py.

# unit suite (mocked HTTP/subprocess, no live server/GPU/WSL required)
.venv\Scripts\python.exe -m pytest -q
# -> 95 passed, 7 skipped

# handshake check -- prints every registered tool name
.venv\Scripts\python.exe scripts\list_tools.py

# real-infra smokes (real vLLM completion, real nvidia-smi, real systemctl
# via WSL, real /proc/<pid>/cmdline read) -- read-only, safe to run anytime
# vLLM is up
VLLM_OPS_MCP_LIVE=1 .venv\Scripts\python.exe -m pytest -v -m live
# -> 7 passed (all live, against the actual running qwen3-14b instance)

Install

python -m venv .venv
.venv\Scripts\python.exe -m pip install -r requirements.txt

Registered in ~/.claude.json as vllm-ops-mcp (stdio, own .venv, no env overrides needed for this operator's default setup).

Commercial support

Maintained by Jaimen Bell. For production MCP integrations, custom servers, or agent-reliability work, see jaimenbell.dev.

Building your own MCP server? The MCP Starter Kit has templates, a build playbook, and packaging war-stories from shipping this one.

mcp-name: io.github.jaimenbell/vllm-ops-mcp