docs: add S3 connector documentation

[nexxeln]

Add comprehensive documentation for the S3 connector including:

• Quick setup with TypeScript, Python, and cURL examples
• S3-compatible services support (MinIO, DigitalOcean Spaces, R2)
• Prefix filtering and dynamic container tag extraction
• Connection management and sync behavior
• IAM permissions and security best practices

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Updated (UTC)
✅ Deployment successful! View logs	supermemory-app	4850856	Jan 09 2026, 03:26 AM

[claude]

Pull Request Review: S3 Connector Documentation

Overall Assessment

This is a well-structured documentation PR that follows the existing connector documentation patterns. The content is clear, comprehensive, and provides good examples across multiple languages. However, I have identified several concerns that should be addressed.

---

Critical Issues

1. S3 Provider Not in Codebase Schema 🚨

The most critical issue: S3 is not currently defined in the ConnectionProviderEnum in packages/validation/schemas.ts:

export const ConnectionProviderEnum = z.enum([
  "notion",
  "google-drive",
  "onedrive",
])

Impact: This documentation documents an S3 connector that doesn't appear to exist in the codebase yet.

Recommendation:

• Either this PR should be merged only after the S3 connector implementation is completed
• Or clearly mark this documentation as "Coming Soon" or "In Development"
• The ConnectionProviderEnum needs to include "s3" before this connector can work

---

Code Quality & Best Practices

2. Documentation Structure ✅

• Follows the established pattern from other connectors (Notion, Google Drive)
• Proper use of MDX components (<Tabs>, <Note>, <Warning>)
• Consistent code examples across TypeScript, Python, and cURL

3. Content Organization ✅

• Good logical flow from Quick Setup → Configuration → Advanced Features
• Clear parameter documentation in table format
• Helpful examples for different S3-compatible services

---

Potential Issues

4. API Consistency

The documentation shows:

await client.connections.create('s3', { ... })

However, this needs verification:

• Does the connections API support the s3 provider string?
• Are all documented parameters (accessKeyId, secretAccessKey, bucket, region, endpoint, prefix, containerTagRegex) supported by the backend validation schemas?

5. Security Concerns

Good practices shown:

• IAM permissions section with minimal required permissions ✅
• Warning about not including the endpoint parameter in table but showing it in examples

Recommendations:

• Consider adding a warning about not committing AWS credentials to version control
• Recommend using environment variables (which is shown in examples, but could be emphasized)
• Add note about rotating credentials periodically

6. Error Handling

The error codes table is minimal (401, 403, 404). Consider adding:

• 400: Invalid bucket name or region format
• 500: S3 service unavailable
• Specific errors for containerTagRegex validation failures

7. Missing Information

Sync Behavior Table:

• States "Every 4 hours + manual triggers" - is this consistent with the cron schedule mentioned in CLAUDE.md?
• No information about:
  • What happens if a file is deleted from S3?
  • How file modifications are detected (ETag, Last-Modified)?
  • Rate limits from S3 API side

Document Limit:

• Default is 10,000 - is this the same as other connectors?
• What happens when the limit is reached? (First 10k files? Error? Warning?)

8. Container Tag Regex Validation

The regex example shows:

containerTagRegex: 'users/(?<userId>[^/]+)/'

Questions:

• The warning says "must contain a named capture group (?<userId>...)" - is userId the only allowed name, or is this just an example?
• What happens if multiple files match and extract different container tags?
• Should clarify if this extracts one tag or multiple tags per file

---

Performance Considerations

9. S3 Bucket Size

• No mention of performance implications for large buckets (millions of files)
• Consider adding guidance on:
  • Using prefix filtering for better performance
  • Impact of initial sync time for large buckets
  • Recommendation for incremental syncs

10. S3-Compatible Services

Good coverage of MinIO, DigitalOcean Spaces, and R2. However:

• Should mention if there are any feature differences between AWS S3 and S3-compatible services
• Any known compatibility issues?

---

Documentation Style Issues

11. Inconsistencies with Other Connectors

Different from Notion/Google Drive docs:

• No "Supported Document Types" section showing which file types are synced
• No "Advanced Configuration" section for content filtering (like LLM filtering in other connectors)
• Missing connection management section with listDocuments examples

Missing compared to other docs:

• No mention of document processing status (extracting, chunking, done, failed)
• No examples of searching synced S3 documents
• No troubleshooting section (other connectors have this as a separate page, but basic troubleshooting could be included)

12. Minor Issues

• Line 11: "Scale Plan or higher" - is this the correct plan name? Verify against pricing tiers
• Line 143: deleteByID vs line 210: deleteByProvider - both methods shown but inconsistently (first example only shows deleteByID)
• The manual sync section (lines 159-173) uses slightly different API than other connectors - verify this is correct

---

Test Coverage

Not shown in this PR:

• No tests for S3 connector documentation examples
• Should verify all code examples actually work

---

Recommendations Summary

Before merging:

1. ✅ Add s3 to ConnectionProviderEnum in schemas
2. ✅ Verify all API parameters are supported by backend validation
3. ✅ Add supported file types section
4. ✅ Clarify containerTagRegex behavior (allowed capture group names, multiple files)
5. ✅ Add more comprehensive error codes
6. ✅ Add sync behavior details (deletions, modifications)
7. ✅ Consider adding security best practices section
8. ✅ Add troubleshooting guidance for common issues

Nice to have:

• Performance guidance for large buckets
• Examples of searching synced S3 documents
• More context on S3-compatible service compatibility

---

Verdict

The documentation is well-written and follows good patterns, but it documents a feature that doesn't appear to be implemented yet. This PR should either:

1. Wait for the S3 connector implementation to be merged first
2. Be marked as draft/preview documentation
3. Include the backend implementation in the same PR

Recommended Action: Request changes - hold until backend implementation is ready or clearly mark as "Coming Soon".

[graphite-app]

Merge activity

• Jan 9, 3:13 AM UTC: Dhravya added this pull request to the Graphite merge queue.
• Jan 9, 3:14 AM UTC: Merged by the Graphite merge queue.

[claude]

PR Review: S3 Connector Documentation

Summary

This PR adds comprehensive documentation for the S3 connector. The documentation is well-structured and follows the existing patterns in the codebase. Overall, this is a solid addition with clear examples and good organization.

✅ Strengths

1. Consistent Structure: Follows the same pattern as existing connector docs (Notion, Google Drive, Web Crawler)
2. Multi-Language Examples: Provides TypeScript, Python, and cURL examples consistently
3. Comprehensive Coverage: Includes setup, configuration, S3-compatible services, prefix filtering, dynamic tags, IAM permissions, and error codes
4. Good Use of Warnings/Notes: Appropriately highlights important information like plan requirements and regex constraints
5. Proper Navigation Integration: Correctly added to docs.json in alphabetical order within the connectors section

📝 Suggestions & Issues

1. Security Concern: Exposed Credentials in Examples

Severity: Medium

Lines 56-57 in the cURL example show what appear to be example AWS credentials:

"accessKeyId": "AKIAIOSFODNN7EXAMPLE",
"secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

While these are clearly marked as examples (with "EXAMPLE" in the strings), it would be safer to use more obviously fake credentials to avoid any confusion:

"accessKeyId": "AKIA...",
"secretAccessKey": "your-secret-key-here"

Reference: apps/docs/connectors/s3.mdx:56-57

2. Inconsistency: Container Tags in Quick Setup

Severity: Low

The Quick Setup examples show different container tag patterns across languages:

• TypeScript: ['org-123'] (line 29)
• Python: ['org-123', 's3-sync'] (line 46)
• cURL: ['org-123'] (line 60)

For consistency and to avoid confusion, all three examples should use the same tags.

Reference: apps/docs/connectors/s3.mdx:29, 46, 60

3. Missing Context: No OAuth Flow Explanation

Severity: Low

Unlike OAuth-based connectors (Google Drive, Notion), the S3 connector doesn't require OAuth. However, the docs don't explicitly state this. Adding a note similar to the web-crawler docs would be helpful:

<Note>
The S3 connector uses direct API authentication with access keys and doesn't require OAuth. The connection is established immediately upon creation.
</Note>

This clarifies expectations for users familiar with other connectors.

Reference: apps/docs/connectors/s3.mdx:13

4. Incomplete Code Example in S3-Compatible Services

Severity: Low

Lines 95-99 show endpoint examples but break from the TypeScript code block style:

// DigitalOcean Spaces
endpoint: 'https://nyc3.digitaloceanspaces.com'

// Cloudflare R2
endpoint: 'https://ACCOUNT_ID.r2.cloudflarestorage.com'

These should either be:

• Wrapped in complete code examples (preferred)
• Formatted as a simple list with descriptions

Reference: apps/docs/connectors/s3.mdx:95-99

5. Documentation: Container Tag Regex Capture Group Name

Severity: Low

The warning states the regex must contain a named capture group (?<userId>...) (line 134), but this appears to be a placeholder name. The documentation should clarify that:

• The capture group name is flexible (not limited to "userId")
• The captured value becomes the container tag

Reference: apps/docs/connectors/s3.mdx:134

6. Potential Enhancement: Supported File Types

Severity: Low

Consider adding a section about supported file types, similar to how other connector docs mention what content types are processed. This would help users understand what files will be synced and indexed.

7. Potential Enhancement: Region List or Examples

Severity: Low

While "us-east-1" is used as an example, it might be helpful to:

• Mention this applies to AWS regions
• Note that for S3-compatible services, the region may be arbitrary or required by the service
• Provide 2-3 common region examples

🔍 Code Quality

• ✅ Markdown formatting is correct
• ✅ Syntax highlighting for code blocks is appropriate
• ✅ Tables are properly formatted
• ✅ Links are valid
• ✅ Navigation structure (docs.json) follows conventions

🧪 Testing Recommendations

1. Verify all API endpoints mentioned in the documentation are implemented
2. Test that the containerTagRegex validation (200 character limit, named capture group) is enforced
3. Confirm the sync schedule (every 4 hours) matches the implementation
4. Validate that the IAM permissions listed are actually the minimum required

📊 Overall Assessment

Recommendation: Approve with minor revisions

This is high-quality documentation that will help users effectively integrate S3 with Supermemory. The issues identified are minor and mostly relate to consistency and clarity improvements. The structure, completeness, and adherence to existing patterns are excellent.

Priority fixes before merge:

1. Address the security concern with example credentials (issue Gemini api #1)
2. Standardize container tags across examples (issue [ImgBot] Optimize images #2)

Nice-to-have improvements:

• Add OAuth clarification note
• Complete the S3-compatible services examples
• Clarify container tag regex capture group naming
• Add supported file types section

---

Great work on this documentation! 🚀

Generated with Claude Code