Make output schema validation non-fatal #73
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add patch-package to make MCP SDK validation warnings instead of errors - Add tests to verify patch continues working after SDK updates - Document approach in patches/README.md This improves resilience when APIs return data that doesn't perfectly match schemas, following the robustness principle.
Contributor
Author
|
closing in favour of #74 |
Contributor
Author
|
re-opening because #74 does not seem to address the issue |
mattpodwysocki
approved these changes
Jan 5, 2026
mattpodwysocki
approved these changes
Jan 5, 2026
Contributor
mattpodwysocki
left a comment
mattpodwysocki
left a comment
There was a problem hiding this comment.
Approved. This SDK patching approach makes sense given:
- Client compatibility issues - Different MCP clients (Gemini, Cursor) interpret Zod schemas differently, making permissive schema features like .passthrough() and .catch() unreliable
- Upstream API instability - Mapbox APIs don't follow strict semver, causing unexpected schema breaks
- Better UX - Users get data instead of errors when schemas drift
- Pragmatic trade-off - Patching maintenance burden is worth avoiding production failures
The patch converts validation errors to warnings, following the robustness principle: 'be liberal in what you accept.'
We'll complement this with live API monitoring (separate PR coming) to detect schema drift early.
mattpodwysocki
added a commit
that referenced
this pull request
Jan 5, 2026
Implements daily scheduled tests that make real API calls to detect schema drift and breaking changes from upstream Mapbox APIs. **Why this exists:** Mapbox APIs don't follow strict semantic versioning - responses can change without notice. This monitoring system provides early warning when schemas drift, preventing production failures. **How it works:** - GitHub Actions runs daily at midnight UTC - Tests call Mapbox APIs with representative queries - Responses are validated against our Zod output schemas - Failures are saved to test/failures/ for analysis - GitHub issues are created automatically with failure details - Test artifacts are uploaded for investigation **Monitored tools:** - SearchAndGeocodeTool (6 test queries) - CategorySearchTool (5 test categories) - ReverseGeocodeTool (4 test coordinates) **Running locally:** \`\`\`bash RUN_API_MONITORING=true npm test -- test/integration/live-api-monitoring.test.ts \`\`\` **Files:** - test/integration/live-api-monitoring.test.ts: Integration tests - .github/workflows/api-monitoring.yml: Daily scheduled workflow - test/integration/README.md: Complete documentation - .gitignore: Exclude test/failures/ directory This complements PR #73's non-fatal validation approach: - PR #73 makes validation failures non-fatal (resilience) - This PR detects schema drift early (observability) Together they provide resilient production behavior and early warning when schemas change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
mattpodwysocki
added a commit
that referenced
this pull request
Jan 5, 2026
This commit resolves critical installation failures that prevented users from running the package via `npx @mapbox/mcp-server`. ## Issues Fixed When users ran `npx @mapbox/mcp-server`, they encountered: ``` sh: mcp-server: command not found ``` This was caused by multiple installation script failures that prevented npm from properly setting up the binary. ## Changes ### 1. Fixed patch-package Dependency Issue **Problem:** The postinstall script runs patch-package, but it was only in devDependencies. End users don't get dev dependencies, causing postinstall to fail. **Solution:** - Moved patch-package from devDependencies to dependencies - Added patches/ directory to files array in package.json - Ensures patches are included in published package ### 2. Updated MCP SDK and Regenerated Patches **Problem:** Patch was for @modelcontextprotocol/[email protected], but the ^ version allowed 1.25.1 to be installed, causing patch failures. **Solution:** - Updated SDK dependency to ^1.25.1 - Regenerated patches for new SDK version - Removed old 1.21.1 patch file - Updated src/tools/BaseTool.ts with type annotations required by new SDK ### 3. Fixed prepare Script for End Users **Problem:** The prepare script checked for .git directory, but when users installed the package in THEIR git repo, it found their .git and tried to run husky (which wasn't installed), causing failures. **Solution:** Changed check from .git to .husky/setup-hooks.js: ```javascript // Before: Checks .git (exists in any git repo) "prepare": "... require('fs').accessSync('.git') ..." // After: Checks for actual setup file (only in dev environment) "prepare": "... require('fs').accessSync('.husky/setup-hooks.js') ..." ``` This ensures prepare only runs in the development environment of this package, not when installed as a dependency. ## Testing Verified the package works correctly: ```bash # Clean install test cd /tmp npx --yes "@mapbox/[email protected]" # ✅ Server starts successfully ``` Integration tests confirm: - ✅ All dependencies install correctly - ✅ MCP SDK patches apply successfully - ✅ mcp-server binary created and executable - ✅ Server runs without errors - ✅ All tools (SearchAndGeocode, CategorySearch, ReverseGeocode) working - ✅ Schema validation working with metadata fields (PR #86) - ✅ Non-fatal validation working (PR #73) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
mattpodwysocki
added a commit
that referenced
this pull request
Jan 5, 2026
This commit resolves critical installation failures that prevented users from running the package via `npx @mapbox/mcp-server`. When users ran `npx @mapbox/mcp-server`, they encountered: ``` sh: mcp-server: command not found ``` This was caused by multiple installation script failures that prevented npm from properly setting up the binary. **Problem:** The postinstall script runs patch-package, but it was only in devDependencies. End users don't get dev dependencies, causing postinstall to fail. **Solution:** - Moved patch-package from devDependencies to dependencies - Added patches/ directory to files array in package.json - Ensures patches are included in published package **Problem:** Patch was for @modelcontextprotocol/[email protected], but the ^ version allowed 1.25.1 to be installed, causing patch failures. **Solution:** - Updated SDK dependency to ^1.25.1 - Regenerated patches for new SDK version - Removed old 1.21.1 patch file - Updated src/tools/BaseTool.ts with type annotations required by new SDK **Problem:** The prepare script checked for .git directory, but when users installed the package in THEIR git repo, it found their .git and tried to run husky (which wasn't installed), causing failures. **Solution:** Changed check from .git to .husky/setup-hooks.js: ```javascript // Before: Checks .git (exists in any git repo) "prepare": "... require('fs').accessSync('.git') ..." // After: Checks for actual setup file (only in dev environment) "prepare": "... require('fs').accessSync('.husky/setup-hooks.js') ..." ``` This ensures prepare only runs in the development environment of this package, not when installed as a dependency. Verified the package works correctly: ```bash cd /tmp npx --yes "@mapbox/[email protected]" ``` Integration tests confirm: - ✅ All dependencies install correctly - ✅ MCP SDK patches apply successfully - ✅ mcp-server binary created and executable - ✅ Server runs without errors - ✅ All tools (SearchAndGeocode, CategorySearch, ReverseGeocode) working - ✅ Schema validation working with metadata fields (PR #86) - ✅ Non-fatal validation working (PR #73) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
mattpodwysocki
added a commit
that referenced
this pull request
Jan 5, 2026
This commit resolves critical installation failures that prevented users from running the package via `npx @mapbox/mcp-server`. When users ran `npx @mapbox/mcp-server`, they encountered: ``` sh: mcp-server: command not found ``` This was caused by multiple installation script failures that prevented npm from properly setting up the binary. **Problem:** The postinstall script runs patch-package, but it was only in devDependencies. End users don't get dev dependencies, causing postinstall to fail. **Solution:** - Moved patch-package from devDependencies to dependencies - Removed duplicate patch-package entry in devDependencies (caused by rebase) - Added patches/ directory to files array in package.json - Ensures patches are included in published package **Problem:** Patch was for @modelcontextprotocol/[email protected], but the ^ version allowed 1.25.1 to be installed, causing patch failures. **Solution:** - Updated SDK dependency to ^1.25.1 - Regenerated patches for new SDK version - Removed old 1.21.1 patch file - Updated src/tools/BaseTool.ts with type annotations required by new SDK **Problem:** The prepare script checked for .git directory, but when users installed the package in THEIR git repo, it found their .git and tried to run husky (which wasn't installed), causing failures. **Solution:** Changed check from .git to .husky/setup-hooks.js: ```javascript // Before: Checks .git (exists in any git repo) "prepare": "... require('fs').accessSync('.git') ..." // After: Checks for actual setup file (only in dev environment) "prepare": "... require('fs').accessSync('.husky/setup-hooks.js') ..." ``` This ensures prepare only runs in the development environment of this package, not when installed as a dependency. **Problem:** TypeScript "Type instantiation is excessively deep" errors after rebasing onto PR #73 with SDK 1.25.1. **Solution:** Added type assertions (`as any`) to break complex type inference chains in: - src/tools/BaseTool.ts (registerTool call) - src/index.ts (setRequestHandler call) These assertions are necessary workarounds for TypeScript's limitations with deeply nested generic types in the MCP SDK. Verified the package works correctly: ```bash cd /tmp npx --yes "@mapbox/[email protected]" ``` Integration tests confirm: - ✅ All dependencies install correctly - ✅ MCP SDK patches apply successfully - ✅ mcp-server binary created and executable - ✅ Server runs without errors - ✅ All tools (SearchAndGeocode, CategorySearch, ReverseGeocode) working - ✅ Schema validation working with metadata fields (PR #86) - ✅ Non-fatal validation working (PR #73) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
mattpodwysocki
added a commit
that referenced
this pull request
Jan 5, 2026
This commit resolves critical installation failures that prevented users from running the package via `npx @mapbox/mcp-server`. When users ran `npx @mapbox/mcp-server`, they encountered: ``` sh: mcp-server: command not found ``` This was caused by multiple installation script failures that prevented npm from properly setting up the binary. **Problem:** The postinstall script runs patch-package, but it was only in devDependencies. End users don't get dev dependencies, causing postinstall to fail. **Solution:** - Moved patch-package from devDependencies to dependencies - Removed duplicate patch-package entry in devDependencies (caused by rebase) - Added patches/ directory to files array in package.json - Ensures patches are included in published package **Problem:** Patch was for @modelcontextprotocol/[email protected], but the ^ version allowed 1.25.1 to be installed, causing patch failures. **Solution:** - Updated SDK dependency to ^1.25.1 - Regenerated patches for new SDK version - Removed old 1.21.1 patch file - Updated src/tools/BaseTool.ts with type annotations required by new SDK **Problem:** The prepare script checked for .git directory, but when users installed the package in THEIR git repo, it found their .git and tried to run husky (which wasn't installed), causing failures. **Solution:** Changed check from .git to .husky/setup-hooks.js: ```javascript // Before: Checks .git (exists in any git repo) "prepare": "... require('fs').accessSync('.git') ..." // After: Checks for actual setup file (only in dev environment) "prepare": "... require('fs').accessSync('.husky/setup-hooks.js') ..." ``` This ensures prepare only runs in the development environment of this package, not when installed as a dependency. **Problem:** TypeScript "Type instantiation is excessively deep" errors after rebasing onto PR #73 with SDK 1.25.1. **Solution:** Added type assertions (`as any`) to break complex type inference chains in: - src/tools/BaseTool.ts (registerTool call) - src/index.ts (setRequestHandler call) These assertions are necessary workarounds for TypeScript's limitations with deeply nested generic types in the MCP SDK. Verified the package works correctly: ```bash cd /tmp npx --yes "@mapbox/[email protected]" ``` Integration tests confirm: - ✅ All dependencies install correctly - ✅ MCP SDK patches apply successfully - ✅ mcp-server binary created and executable - ✅ Server runs without errors - ✅ All tools (SearchAndGeocode, CategorySearch, ReverseGeocode) working - ✅ Schema validation working with metadata fields (PR #86) - ✅ Non-fatal validation working (PR #73) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR improves the MCP server's resilience by making output schema validation non-fatal. When APIs return data that doesn't perfectly match the expected schema, the server now logs warnings instead of throwing errors.
The change follows the robustness principle: be conservative in what you send, liberal in what you accept. Output schemas remain valuable for documentation while improving real-world reliability.
Testing
test/patches/mcp-sdk-patch.test.tsExample with MCP inspector
Making sure the tests catch missing patching
In this case I just removed node modules and reinstalled with patching removed from postinstall script. The idea is to get test to fail since mismatch in schema will cause the test to throw.
Looks all good.
Checklist
Additional Notes
Implementation uses
patch-packageto modify the MCP SDK validation behavior. The patch:npm installvia postinstall scriptpatches/README.mdThe test suite will fail loudly if the SDK is updated and the patch needs recreation, providing clear instructions for fixes.