Supervisor Orchestrator

AI-powered multi-agent task orchestration system using BullMQ, Redis, and Model Context Protocol (MCP) servers.

Architecture Overview

┌─────────────────────────────────────────────────────────────┐
│                   SUPERVISOR LAYER                          │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐     │
│  │  Planning    │  │ Coordination │  │    Error     │     │
│  │   Engine     │  │    Engine    │  │   Handler    │     │
│  │(Sequential-  │  │  (BullMQ)    │  │              │     │
│  │  Thinking)   │  │              │  │              │     │
│  └──────────────┘  └──────────────┘  └──────────────┘     │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                     QUEUE TOPOLOGY                          │
│  ┌────────────┐ ┌────────────┐ ┌────────────┐             │
│  │ supervisor │ │ supervisor │ │ supervisor │             │
│  │  planning  │ │ coordination│ │   error    │             │
│  └────────────┘ └────────────┘ └────────────┘             │
│                                                             │
│  ┌────────────┐ ┌────────────┐ ┌────────────┐             │
│  │   worker   │ │   worker   │ │   worker   │             │
│  │    web     │ │    code    │ │  deployment│             │
│  └────────────┘ └────────────┘ └────────────┘             │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                    WORKER AGENTS                            │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐     │
│  │Hyperbrowser│ │ GitHub  │ │ Netlify  │ │   n8n    │     │
│  │    MCP   │ │   MCP    │ │   MCP    │ │   MCP    │     │
│  └──────────┘ └──────────┘ └──────────┘ └──────────┘     │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                  STATE MANAGEMENT                           │
│              Memory MCP (Knowledge Graph)                   │
│  - Agent tracking                                           │
│  - Task dependencies                                        │
│  - Execution history                                        │
└─────────────────────────────────────────────────────────────┘

Infrastructure

Core Components

Redis 8.2.3: Message broker and queue backend
BullMQ 5.x: Distributed job queue system
TypeScript 5.x: Type-safe orchestration logic
Node.js: Runtime environment

MCP Server Integration

BullMQ MCP (@adamhancock/bullmq-mcp): Queue management interface
Memory MCP: Knowledge graph for state tracking
Sequential-Thinking MCP: AI-powered task planning
GitHub MCP: Code repository operations
Netlify MCP: Deployment automation
Puppeteer/Chrome DevTools MCP: Web automation

Installation Status

✅ Redis installed and running (localhost:6379) ✅ BullMQ MCP server configured ✅ Project dependencies installed (48 packages) ✅ TypeScript configuration complete ✅ MCP servers configured in .mcp.json

Project Structure

~/supervisor-orchestrator/
├── .mcp.json                       # MCP server configuration
├── package.json                    # Dependencies and scripts
├── tsconfig.json                   # TypeScript configuration
├── src/                            # Source code (to be implemented)
│   ├── types.ts                    # TypeScript interfaces
│   ├── state-manager.ts            # Memory MCP interface
│   ├── planning-engine.ts          # Sequential-Thinking integration
│   ├── coordination-engine.ts      # BullMQ queue management
│   ├── supervisor-orchestrator.ts  # Main orchestrator class
│   └── index.ts                    # CLI entry point
├── dist/                           # Compiled JavaScript (after build)
└── README.md                       # This file

Queue Topology

The system uses 7 specialized BullMQ queues:

Supervisor Queues

supervisor-planning: High-level task decomposition and planning
supervisor-coordination: Multi-agent coordination and dependency management
supervisor-error-handling: Error recovery and retry logic

Worker Queues

worker-web-automation: Web scraping, browser automation (Hyperbrowser/Chrome DevTools)
worker-code-operations: GitHub operations, code analysis, repository management
worker-deployment: Netlify deployments, build triggers
worker-integration: n8n workflow automation, third-party integrations

Next Steps: Implementation Phase

1. Initialize Memory MCP Agents

Create agent entities in the Memory MCP knowledge graph:

// Via Claude Desktop or CLI
memory.create_entities([
  {
    name: "Agent_Hyperbrowser",
    entityType: "Worker",
    observations: ["Handles web automation and scraping tasks"]
  },
  {
    name: "Agent_GithubMCP",
    entityType: "Worker",
    observations: ["Manages code repository operations"]
  },
  {
    name: "Agent_NetlifyMCP",
    entityType: "Worker",
    observations: ["Handles deployment automation"]
  },
  {
    name: "Agent_N8NMCP",
    entityType: "Worker",
    observations: ["Manages workflow integrations"]
  },
  {
    name: "Agent_SequentialThinking",
    entityType: "Supervisor",
    observations: ["Plans and decomposes complex tasks"]
  }
]);

2. Initialize BullMQ Queues

Create the queue structure via BullMQ MCP:

# Connect to Redis
bullmq.connect({
  id: "supervisor-main",
  host: "localhost",
  port: 6379,
  db: 0
})

# Queues will be auto-created on first job submission

3. Implement Core Orchestration Logic

Create the following source files:

src/types.ts

export interface Task {
  id: string;
  type: 'web-automation' | 'code-ops' | 'deployment' | 'integration';
  description: string;
  dependencies: string[];
  status: 'pending' | 'in-progress' | 'completed' | 'failed';
  assignedAgent: string;
}

export interface WorkflowPlan {
  tasks: Task[];
  executionOrder: string[];
  estimatedDuration: number;
}

src/planning-engine.ts

Interface with Sequential-Thinking MCP
Decompose high-level requests into subtasks
Generate dependency graphs

src/coordination-engine.ts

Submit jobs to appropriate BullMQ queues
Monitor queue status and job completion
Handle parallelization and sequencing

src/state-manager.ts

Track task states in Memory MCP
Update agent availability
Maintain execution history

4. Build and Test

# Compile TypeScript
npm run build

# Run in development mode
npm run dev

# Start production server
npm start

5. Example Usage

Once implemented, you'll be able to execute complex workflows:

# Natural language request
"Deploy my website to Netlify after updating the GitHub repo with the latest changes from the staging branch"

# System execution:
# 1. Sequential-Thinking: Plan task breakdown
# 2. Memory: Create task entities and dependencies
# 3. BullMQ: Queue jobs in correct order
#    - worker-code-operations: Pull staging branch
#    - worker-code-operations: Merge to main
#    - worker-code-operations: Push to GitHub
#    - worker-deployment: Trigger Netlify build
# 4. Workers: Execute tasks via respective MCPs
# 5. Memory: Update task statuses
# 6. Supervisor: Verify completion

Monitoring and Debugging

Check Queue Status

# Via BullMQ MCP
bullmq.stats()  # View all queue statistics

Inspect Knowledge Graph

# Via Memory MCP
memory.read_graph()  # View all entities and relationships
memory.search_nodes({ query: "Agent_" })  # Find specific agents

View Redis Data

redis-cli
> KEYS *  # List all keys
> LRANGE bull:queue-name:wait 0 -1  # View waiting jobs

Environment Configuration

Redis Connection

Host: localhost
Port: 6379
Database: 0
URL: redis://localhost:6379

MCP Servers

All MCP servers are configured in .mcp.json and will auto-connect when Claude Code starts.

Security Considerations

Redis has no authentication by default (localhost-only binding)
API keys for GitHub, Netlify in user-level .mcp.json
Never commit .mcp.json with secrets to version control
Use environment variables for production deployments

Performance Tuning

Queue Concurrency

Adjust worker concurrency in BullMQ configuration:

const worker = new Worker('queue-name', async (job) => {
  // Process job
}, {
  connection: redisConnection,
  concurrency: 5  // Process 5 jobs simultaneously
});

Redis Memory

Monitor Redis memory usage:

redis-cli INFO memory

Troubleshooting

Redis Connection Issues

# Check if Redis is running
brew services list | rg redis

# Restart Redis
brew services restart redis

# Test connectivity
redis-cli ping  # Should return PONG

MCP Server Connection Failures

# Verify MCP server health
claude mcp list

# Reset project approvals
claude mcp reset-project-choices

Queue Job Failures

Check error logs via bullmq.get_job(jobId)
Inspect failed job details in Redis
Verify worker agent availability in Memory MCP

Development Roadmap

Phase 1: Core Infrastructure ✅ COMPLETE

Redis installation
BullMQ MCP integration
Project scaffolding
TypeScript configuration

Phase 2: Orchestration Logic (Current)

Implement planning engine
Build coordination engine
Create state manager
Develop worker adapters

Phase 3: Agent Integration

Connect GitHub MCP worker
Connect Netlify MCP worker
Connect web automation workers
Implement error handlers

Phase 4: Testing & Optimization

End-to-end workflow testing
Performance benchmarking
Error recovery scenarios
Production monitoring setup

License

MIT

Support

For issues or questions:

GitHub Issues: TBD
Documentation: This README
MCP Documentation: https://modelcontextprotocol.io

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
src		src
.env.example		.env.example
.gitignore		.gitignore
.mcp.json		.mcp.json
MCP-SDK-INTEGRATION-PLAN.md		MCP-SDK-INTEGRATION-PLAN.md
README.md		README.md
package.json		package.json
tsconfig.json		tsconfig.json

thefiredev-cloud/supervisor-orchestrator

Folders and files

Latest commit

History

Repository files navigation