Skip to content

Add workflows + second public R2 bucket#234

Open
Judyzc wants to merge 4 commits intocloudflare:mainfrom
Judyzc:issue-209-workflows
Open

Add workflows + second public R2 bucket#234
Judyzc wants to merge 4 commits intocloudflare:mainfrom
Judyzc:issue-209-workflows

Conversation

@Judyzc
Copy link
Copy Markdown
Contributor

@Judyzc Judyzc commented Mar 27, 2026

Addresses (fixes) #209. Tested on staging.

High-level explanation of changes:

  • 1st Problem: AI content rating could time out after 30 seconds because Cloudflare Workers can only run a request for 30 seconds. This can leave a test in "IN_PROGRESS" state, never able to be accessed or re-uploaded.
    • Solution: added Cloudflare Workflow for AI content rating. Workflows can run up to 5 minutes, auto retry step(s) if configured.
  • 2nd Problem: filmstrip and other assets from R2 respond intermittently with http 500 errors, because the D1/prisma lookup for the asset's content rating (we only want to serve safe assets so we need to ensure the content_rating for each asset is "SAFE") overwhelms the database pool connection.
    • Solution: make a 2nd and always public R2 bucket for "SAFE" content that will have a custom domain endpoint. This public endpoint will serve assets instead of D1/the Worker. I've already made and configured the custom domains for the public R2 buckets (staging and prod) using the production UI dashboard.
    • Solution should work because essentially "if someone has a testId, they could already read all the files. The public bucket doesn't open any new attack surface — it just serves the same already-public data without a Worker in the middle" (argument from AI)

New upload flows for different environments:

Dev, AI off (default)

  • Upload → files written to local private R2, no workflow, test accessible immediately (AI disabled so no rating gate)
  • Serving → All assets served through /api/tests/* → local Worker → local private R2

Dev, AI on (.dev.vars override)

  • Upload → files written to local private R2, workflow runs locally, AI rates it, if SAFE copies to local public R2
  • Serving → PUBLIC_ASSETS_URL absent → hasPublicBucket() false → resolveAssetBase returns '' → all assets still served through /api/tests/* → local Worker → local private R2 (public R2 is ignored for serving)

Staging

  • Upload → files written to remote private R2, workflow triggers on Cloudflare, if SAFE copies to remote public R2
  • Serving → For SAFE tests: resolveAssetBase finds config.json in public R2 → screenshot/filmstrip/video URLs built as https://assets-staging.telescopetest.io/... in page HTML → browser loads them directly from the edge, no Worker
  • For pre-workflow tests (not yet in public R2): falls back to /api/tests/* → Worker → private R2

Production

Follow-up

We'll need some kind of one-time script or process to move all the current R2 files that are safe into the public bucket. Right now, the code has a fallback to use D1/the Worker if the asset isn't in the public bucket. This is so the PR doesn't break any existing results/functionality when merged.

But I think ideally we'd do this process: (1) merge this PR (with fallback code) -> (2) run one-time script to move files -> (3) clean up fallback code

@Judyzc Judyzc changed the title Issue 209 workflows Add workflows + second public R2 bucket Mar 27, 2026
@Judyzc Judyzc force-pushed the issue-209-workflows branch 4 times, most recently from b06ec4c to 1917897 Compare March 30, 2026 22:17
…1 and R2 calls, renamed files, checked if Cloudflare Workflow works with astro dev (hot reload) and staging
@Judyzc Judyzc force-pushed the issue-209-workflows branch from 1917897 to 60a95a6 Compare March 30, 2026 22:32
"$schema": "node_modules/wrangler/config-schema.json",
"name": "telescopetest-io",
"account_id": "4b1c95badf17540db82cd8b431ae3eb0",
"main": "./src/worker.ts",
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

./src/worker.ts is required to export AiContentRatingWorkflow as a named export alongside Astro's request handler — the standard @astrojs/cloudflare/entrypoints/server entrypoint doesn't support additional exports (https://docs.astro.build/en/guides/integrations-guide/cloudflare/#changed-custom-entrypoint-api).

@Judyzc Judyzc marked this pull request as ready for review March 30, 2026 23:04
@Judyzc Judyzc requested a review from a team March 30, 2026 23:04
@Judyzc Judyzc marked this pull request as draft March 30, 2026 23:04
if (!listed.objects || listed.objects.length === 0) {
// Paginate R2 list() — returns at most 1000 objects per call
// https://developers.cloudflare.com/r2/api/workers/workers-api-reference/#r2listoptions
let listed = await bucket.list({ prefix });
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now get R2 assets from public bucket if safe (bypass Worker completely).

// Try public bucket first — SAFE files are copied here by the workflow, no rating check needed.
// Serve directly (no redirect) since fetch() calls can't follow cross-origin redirects (CORS).
const publicFile = hasPublicBucket()
? await env.PUBLIC_RESULTS_BUCKET!.get(key).catch(() => null)
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now get R2 assets from public bucket if safe (bypass Worker completely).

@Judyzc Judyzc marked this pull request as ready for review March 31, 2026 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant