feat: Discord mesh bridge with live message forwarding#372
feat: Discord mesh bridge with live message forwarding#372SimmerV wants to merge 9 commits intoMeshAddicts:developfrom
Conversation
… active-node priority
There was a problem hiding this comment.
Pull request overview
Adds a live mesh-to-Discord bridge (webhook-based sender identities, gateway aggregation, rich embeds) plus Discord slash commands and PostgreSQL persistence to support linking/banning/tracking nodes.
Changes:
- Introduces Discord bridge event flow (MQTT → in-memory queue → Discord webhook posts/edits) with rich text/position embeds.
- Adds Discord slash commands for node linking and bridge moderation (ban + tracker/balloon).
- Extends PostgreSQL schema and storage layer with new Discord bridge tables and helper queries.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
| storage/db/postgres.py | Adds query ordering and new DB helper methods for Discord bridge persistence (links/bans/tracking). |
| postgres/sql/schema.sql | Creates new Discord bridge tables (links, banned nodes, tracked nodes). |
| mqtt.py | Emits text/position events into an in-memory queue for the bridge. |
| memory_data_store.py | Adds discord_event_queue for MQTT→Discord event transport. |
| config.toml.sample | Documents new Discord bridge configuration options and channel mappings. |
| config.py | Adds default config + validation for Discord bridge settings. |
| bot/embeds.py | New embed builders for bridged text and position events (gateway grouping, deep links, map thumbnail). |
| bot/discord.py | Registers new cogs (admin commands + bridge). |
| bot/cogs/mesh_bridge.py | New MeshBridge cog that consumes queue events, aggregates, and posts/edits via webhooks. |
| bot/cogs/main_commands.py | Enhances /lookup and /mesh with DB-backed lookups and richer output. |
| bot/cogs/admin_commands.py | New slash commands for linking nodes and bridge moderation/tracking. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Try as hex ID | ||
| nid = _normalize_node_id(raw) | ||
| if nid: | ||
| # Verify it exists (optional — allow linking to unknown nodes) | ||
| node = self.data.nodes.get(nid) | ||
| if node: | ||
| name = node.get("longname", node.get("shortname", "")) | ||
| return nid, name if name not in ("Unknown", "UNK", "") else None | ||
| # Check DB | ||
| if self.data.pg_storage: | ||
| db_node = await self.data.pg_storage.query_node_by_id(nid) | ||
| if db_node: | ||
| name = db_node.get("longname", db_node.get("shortname", "")) | ||
| return nid, name if name not in ("Unknown", "UNK", "") else None | ||
| # Valid hex but not in DB — still allow it | ||
| return nid, None | ||
|
|
||
| # Try as integer ID | ||
| search = raw.replace("!", "").strip() | ||
| try: | ||
| id_int = int(search, 10) | ||
| nid = utils.convert_node_id_from_int_to_hex(id_int) | ||
| return nid, None | ||
| except (ValueError, TypeError): | ||
| pass |
There was a problem hiding this comment.
_resolve_node_id tries to interpret input as a hex node ID before attempting integer parsing. For digit-only inputs (e.g., "1234"), this will be treated as hex (00001234) instead of decimal (000004d2), which contradicts the command help text (“integer ID”) and can result in acting on the wrong node (ban/track/link). Prefer parsing as int first when the input is digits-only (and reserve hex parsing for values with !/0x prefixes or containing a-f).
| logger.info("Discord: /lookup: Looking up %s", flags.node) | ||
| search = flags.node.strip().lower().replace("!", "") | ||
| node = None | ||
| id_hex = None | ||
|
|
||
| # Try parsing as integer node ID | ||
| try: | ||
| id_int = int(flags.node, 10) | ||
| id_int = int(search, 10) | ||
| id_hex = utils.convert_node_id_from_int_to_hex(id_int) | ||
| except ValueError: | ||
| id_hex = flags.node | ||
| pass | ||
|
|
||
| # Try parsing as hex node ID | ||
| if id_hex is None and all(c in '0123456789abcdef' for c in search) and len(search) <= 8: | ||
| id_hex = search.zfill(8) |
There was a problem hiding this comment.
If the user passes an empty/placeholder node string like "!", search becomes empty and the hex-parse branch will treat it as valid (all(...) on an empty string is true) and resolve it to 00000000. Add an early guard for empty input (after stripping/removing !) so /lookup doesn’t accidentally look up node !00000000 on invalid input.
| # Emit event for Discord bridge | ||
| try: | ||
| self.data.discord_event_queue.put_nowait({ | ||
| 'type': 'position', | ||
| 'msg': msg, | ||
| 'node_id': id, | ||
| }) | ||
| except Exception: | ||
| pass |
There was a problem hiding this comment.
Events are enqueued to discord_event_queue unconditionally, but the MeshBridge consumer only starts when integrations.discord.bridge.enabled is true. If the Discord bot is running but the bridge is disabled (or not running at all), this unbounded queue will grow indefinitely and can exhaust memory. Gate the enqueue on the bridge-enabled config (or implement a bounded queue with drop/backpressure) so disabled bridge mode doesn’t accumulate events.
| # Emit event for Discord bridge | |
| try: | |
| self.data.discord_event_queue.put_nowait({ | |
| 'type': 'position', | |
| 'msg': msg, | |
| 'node_id': id, | |
| }) | |
| except Exception: | |
| pass | |
| # Emit event for Discord bridge, if the queue is available | |
| queue = getattr(self.data, "discord_event_queue", None) | |
| if queue is not None: | |
| try: | |
| queue.put_nowait({ | |
| 'type': 'position', | |
| 'msg': msg, | |
| 'node_id': id, | |
| }) | |
| except asyncio.QueueFull: | |
| # Drop event if the queue is bounded and full to avoid unbounded growth | |
| pass | |
| except Exception: | |
| logger.exception("Failed to enqueue Discord position event") |
| try: | ||
| self.data.discord_event_queue.put_nowait({ | ||
| 'type': 'text', | ||
| 'msg': msg, | ||
| 'chat': chat, | ||
| }) | ||
| except Exception: | ||
| pass |
There was a problem hiding this comment.
Text events are pushed into discord_event_queue even when the bridge consumer may not be running, and the queue is unbounded. This can lead to unbounded memory growth under normal operation if integrations.discord.bridge.enabled is false. Ensure enqueueing is conditional on the bridge being enabled/active (or enforce a max queue size and drop policy).
| try: | |
| self.data.discord_event_queue.put_nowait({ | |
| 'type': 'text', | |
| 'msg': msg, | |
| 'chat': chat, | |
| }) | |
| except Exception: | |
| pass | |
| bridge_enabled = ( | |
| self.config | |
| .get('integrations', {}) | |
| .get('discord', {}) | |
| .get('bridge', {}) | |
| .get('enabled', False) | |
| ) | |
| discord_queue = getattr(self.data, 'discord_event_queue', None) | |
| if bridge_enabled and discord_queue is not None: | |
| try: | |
| discord_queue.put_nowait({ | |
| 'type': 'text', | |
| 'msg': msg, | |
| 'chat': chat, | |
| }) | |
| except Exception: | |
| logger.warning("Failed to enqueue Discord text event", exc_info=True) |
| try: | ||
| # Look for an existing MeshInfo webhook | ||
| webhooks = await channel.webhooks() | ||
| for wh in webhooks: | ||
| if wh.name == WEBHOOK_NAME: | ||
| self._webhooks[channel.id] = wh | ||
| return wh | ||
|
|
||
| # Create one | ||
| wh = await channel.create_webhook(name=WEBHOOK_NAME) | ||
| self._webhooks[channel.id] = wh | ||
| return wh | ||
| except discord.Forbidden: | ||
| logger.warning("MeshBridge: No permission to manage webhooks in #%s — falling back to bot messages", channel.name) | ||
| return None | ||
| except Exception: |
There was a problem hiding this comment.
On discord.Forbidden (no Manage Webhooks permission), _get_webhook returns None but doesn’t cache that outcome. This will cause repeated channel.webhooks() calls and warning logs on every forwarded message, which is noisy and can become a throughput bottleneck. Cache a sentinel for “webhooks unavailable” per-channel (and possibly rate-limit the warning) so subsequent sends go straight to the bot-message fallback.
| async def get_node_owner(self, node_id: str) -> Optional[str]: | ||
| """Return the Discord user ID linked to a node, or None.""" | ||
| if not self._ready("get_node_owner"): | ||
| return None | ||
| try: | ||
| async with self.pool.acquire() as conn: | ||
| return await conn.fetchval( | ||
| "SELECT discord_user_id FROM discord_node_links WHERE node_id = $1 LIMIT 1", | ||
| node_id, | ||
| ) |
There was a problem hiding this comment.
discord_node_links allows multiple Discord users to be linked to the same node_id (unique is on (node_id, discord_user_id)), but get_node_owner returns an arbitrary single user via LIMIT 1. If the bridge intends a single owner per node (as implied by the embed’s singular “Owner” field), enforce uniqueness on node_id (and adjust link_node conflict handling), or change the API to return all linked owners deterministically.
| node_id VARCHAR(8) NOT NULL, | ||
| discord_user_id VARCHAR(20) NOT NULL, | ||
| created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), | ||
| CONSTRAINT discord_node_links_unique UNIQUE (node_id, discord_user_id) |
There was a problem hiding this comment.
Schema-wise, discord_node_links is unique on (node_id, discord_user_id), which permits multiple Discord users per node. Since the application currently models a single “owner” (get_node_owner + singular Owner field), consider enforcing uniqueness on node_id (or adding a separate owner column/table) to avoid ambiguous ownership and nondeterministic LIMIT 1 behavior.
| CONSTRAINT discord_node_links_unique UNIQUE (node_id, discord_user_id) | |
| CONSTRAINT discord_node_links_unique UNIQUE (node_id) |
| self.traceroutes_by_node: dict = {} | ||
|
|
||
| # Event queue for Discord bridge (MQTT -> Discord) | ||
| self.discord_event_queue: asyncio.Queue = asyncio.Queue() |
There was a problem hiding this comment.
discord_event_queue is created with the default maxsize=0 (unbounded). Since mesh traffic can be high-volume, this can become a memory risk if the Discord consumer is slow or temporarily stopped. Consider setting a reasonable maxsize and defining a drop/backpressure strategy (with logging/metrics) to prevent unbounded growth.
| self.discord_event_queue: asyncio.Queue = asyncio.Queue() | |
| # Use a bounded queue to avoid unbounded memory growth if the consumer is slow/stalled. | |
| discord_cfg = self.config.get("discord", {}) if isinstance(self.config, dict) else {} | |
| self.discord_event_queue_maxsize: int = int(discord_cfg.get("event_queue_maxsize", 1000)) | |
| self.discord_event_queue: asyncio.Queue = asyncio.Queue(maxsize=self.discord_event_queue_maxsize) |
| # Static map API for position embeds | ||
| MAP_API_URL = "https://api.smerty.org/staticmap" | ||
|
|
There was a problem hiding this comment.
MAP_API_URL is hard-coded to an external service, and build_position_embed always constructs a thumbnail URL containing precise lat/lon. This leaks location data to a third-party endpoint and makes the bridge dependent on that service’s uptime. Consider making the static-map base URL configurable (or optional/disabled by default) and documenting the privacy/operational implications for operators.
| async def link_node(self, node_id: str, discord_user_id: str) -> bool: | ||
| """Link a mesh node to a Discord user. Returns True on success.""" | ||
| if not self._ready("link_node"): | ||
| return False | ||
| try: | ||
| async with self.pool.acquire() as conn: | ||
| await conn.execute( | ||
| """INSERT INTO discord_node_links (node_id, discord_user_id) | ||
| VALUES ($1, $2) | ||
| ON CONFLICT (node_id, discord_user_id) DO NOTHING""", | ||
| node_id, discord_user_id, | ||
| ) |
There was a problem hiding this comment.
The Discord bridge DB helpers write node_id values as provided without using the existing _normalize_node_id helper. This can lead to mixed-case IDs, leading '!' prefixes, or short IDs being stored, which will break lookups and joins elsewhere. Normalize (and reject invalid) node IDs inside these helpers for consistency with the rest of PostgresStorage.
Summary