[Feature]: Integrate MCP Server into DiracX

[aldbr]

User Story

As a DiracX administrator / LHCb developer, I need a co-deployed MCP server integrated into DiracX so that AI agents and MCP clients (Claude Desktop, Cursor, etc.) can interact with DiracX with per-user authentication, access policy enforcement, and extensibility via entry points.

Feature Description

dirac-agentic currently provides a standalone MCP server (dirac-mcp) that wraps the DiracX REST API via the diracx Python client. This works for demos but has limitations for production:

• No per-user auth: runs as a single identity (credential file), not per-user
• No extensibility: LHCb and other extensions can't add MCP tools via entry points
• HTTP overhead: every tool call goes MCP → dirac-mcp → DiracX REST API → DB and back
• Access policies are opaque: enforced by the REST API, not controllable by MCP

The proposal is to create a diracx-mcp package in the DiracX monorepo, co-deployed as a peer of diracx-routers:

graph BT
    db["diracx-db<br/><i>database layer</i>"]
    logic["diracx-logic<br/><i>business logic + access policies<br/>+ token verification</i>"]
    routers["diracx-routers<br/><i>REST / FastAPI</i>"]
    mcp["diracx-mcp<br/><i>MCP / FastMCP</i>"]

    logic --> db
    routers --> logic
    mcp --> logic

    style mcp fill:#d4edda,stroke:#28a745
    style routers fill:#cce5ff,stroke:#004085

Loading

MCP is JSON-RPC 2.0 over streamable HTTP, a different protocol from REST. It cannot be a DiracxRouter. Instead, diracx-mcp is mounted as a separate ASGI sub-app on the same server (same approach as LbAPI).

What needs to change

Prerequisite — move access policies out of diracx-routers:

Access policies (BaseAccessPolicy, WMSAccessPolicy, SandboxAccessPolicy) currently live in diracx-routers and raise FastAPI's HTTPException. Since they are business rules, they belong in diracx-logic. The actual coupling is minimal (~70% already framework-agnostic).

New package — diracx-mcp:

• Depends on diracx-logic + diracx-db + fastmcp>=3.0 (does NOT depend on diracx-routers)
• Tools call diracx-logic directly and use access policies without FastAPI
• Extensible via a new diracx.mcp_tools entry point group — extensions (LHCb) register tools the same way they register routers today
• Mounted in factory.py as an ASGI sub-app with graceful degradation if not installed

Authentication:

• For human MCP clients (Claude Desktop, Cursor): FastMCP's OIDCProxy pointing to DiracX's own OIDC endpoints. DiracX issues the JWT, the MCP server validates it using the same read_token() logic.
• For programmatic agents (smolagents, langgraph) client_credentials flow.

Definition of Done

dirac-mcp should be:

• deployed and working in certification
• extensible
• accessible through agents without needing user interactions for the authentication
• extensively unit tested
• evaluated with a baseline model on simple scenarios would be nice

Alternatives Considered

Keep standalone MCP wrapping the DiracX REST API

Criterion	Co-deployed (proposed)	Standalone client (current)
Performance	Direct function calls, shared DB	HTTP round-trip per tool call
Auth model	Per-user (MCP OAuth → DiracX JWT)	Single identity (credential file)
Access policies	Enforced in MCP, extensible	Delegated to REST, opaque
Extensibility	Entry points for tools	Not extensible
DiracX changes	Yes (move policies to logic)	None
Deployment	Co-deployed with DiracX	Runs anywhere

The standalone version could be kept as a lightweight demo/proxy for prototyping without a full DiracX deployment.

MCP as a module inside diracx-routers

Simpler (no refactoring), but mixes REST and MCP concerns, forces fastmcp dependency on all deployments.

diracx-mcp depending on diracx-routers

No refactoring needed (access policies stay in routers), but diracx-mcp becomes a consumer of diracx-routers rather than a peer. Pragmatic but architecturally "impure": MCP is a protocol interface, not a REST client (many MCP servers are taking that direction though).

Related Issues

No response

Additional Context

• MCP Architecture Overview: MCP is JSON-RPC 2.0, not REST
• LbAPI MCP implementation: reference for mounting FastMCP as ASGI sub-app alongside FastAPI, with CERN SSO auth via OIDCProxy
• dirac-agentic: current standalone MCP server and agent examples