feat!: s3-ddb cache handler replaces efs, run next build on host, use nextjs official adapter

[bestickley]

Fixes #165, #142, #127, #121, #115

This PR will be released as 0.5.0-beta.1. It will remain beta until next.config.js' experimental.adapterPath is no longer experimental. This PR requires at least Next.js v16.1.1-canary.19 as it relies on Image Custom Cache Handler. While it's not ideal to depend upon experimental or canary features, this aligns with cdk-nextjs' design philosophy of not tampering with Next.js internals and relying on public APIs wherever possible.

• Use S3 + DynamoDB as Next.js Cache replacing EFS
  • Create NextjsCache
  • Remove NextjsAssetDeployment
  • Fix APP_PAGE cahe kind with ⨯ Error: failed to pipe response error by storing as utf8 instead of base64
  • .next/cdk-nextjs-init-cache - fill with fetch and prerender json values
• Run next build on host machine, not in container - simplifying build process and reducing build time.
  • buildContext is replaced with buildDirectory
  • Replace sharp with linux musl libvips since host machine is likely diff arch and platform then cloud/runtime
• Use Next.js Official Adapters to simplify next.config.ts management.
• NextjsRegionalFunctions now supports streaming (since API GW now supports it)
• Improve docs by writing caching, pruning, and nextjs build output guide
• Upgrade examples to Next.js 16
• Upgrade Node.js to 24 (LTS)
• Ensure e2e tests passing
• Explore usage of S3 Directory Buckets (One Zone Express) for lower cost and faster response times for cache bucket. Decided not to use it. See S3 Directory Bucket (One Zone Express) vs Standard Buckets #191

Follow up Work

• Had to unfortunately include public/ in image, Create NextjsImageOptimizationFunction #193 will fix this and allow us to dynamically pull public images from s3.
• Fix flaky/failing isr and revalidation e2e tests in Fix flaky/failing isr and revalidation e2e tests #194

[mergify]

❌ The current Mergify configuration is invalid

Details

• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → strict
• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → commit_message → title+body

[mergify]

❌ The current Mergify configuration is invalid

Details

• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → strict
• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → commit_message → title+body

[mergify]

❌ The current Mergify configuration is invalid

Details

• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → strict
• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → commit_message → title+body

[mergify]

❌ The current Mergify configuration is invalid

Details

• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → strict
• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → commit_message → title+body

[mergify]

❌ The current Mergify configuration is invalid

Details

• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → strict
• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → commit_message → title+body

[mergify]

❌ The current Mergify configuration is invalid

Details

• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → strict
• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → commit_message → title+body

[mergify]

❌ The current Mergify configuration is invalid

Details

• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → strict
• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → commit_message → title+body

[mergify]

❌ The current Mergify configuration is invalid

Details

• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → strict
• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → commit_message → title+body

[mergify]

❌ The current Mergify configuration is invalid

Details

• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → strict
• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → commit_message → title+body

[mergify]

❌ The current Mergify configuration is invalid

Details

• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → strict
• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → commit_message → title+body

[mergify]

❌ The current Mergify configuration is invalid

Details

• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → strict
• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → commit_message → title+body

[mergify]

❌ The current Mergify configuration is invalid

Details

• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → strict
• Extra inputs are not permitted @ root → pull_request_rules → item 0 → actions → merge → commit_message → title+body

[LaurenceGA]

Hey. Just wondering what's the next step on this PR?

[bestickley]

Hey Laurence, I've kinda been waiting for Next.js to release 16.2 so canary isn't required for image custom cache handler but it's taken a lot longer for them than I expected :/