OTLP Optimizer

A Cloudflare Worker that optimizes OTLP (OpenTelemetry Protocol) JSON payloads by deduplicating redundant Resource and Scope declarations before forwarding to your observability backend.

This worker is completely stateless, and should be fairly easy to operate.

Problem

Cloudflare OTLP exports are currently limited in their batching logic. Due to this limitation, the output contains repeated identical Resource and Scope sections for each log record or span. The structure of the payload looks ~like the following:

Resource { service.name: "my-app" }
  Scope { name: "my-logger" }
    LogRecord: A
Resource { service.name: "my-app" }
  Scope { name: "my-logger" }
    LogRecord: B
Resource { service.name: "my-app" }
  Scope { name: "my-logger" }
    LogRecord: C

This is valid OTLP, but can balloon the payload size beyond what some vendors will accept.

Solution

This proxy takes a batch and consolidates records with identical Resource/Scope into single entries:

Resource { service.name: "my-app" }
  Scope { name: "my-logger" }
    LogRecords: A, B, C

Features

• Optimizes both logs (/v1/logs) and traces (/v1/traces) endpoints
• Bearer token authentication
• Configurable destination URLs and headers per endpoint
• Passes through errors from destination so that the original OTLP export can handle retries

Quick Start

Prerequisites

• Node.js (v18+)
• Wrangler CLI (npm install -g wrangler)
• A Cloudflare account

1. Clone and Install

git clone <repository-url>
cd otlp-optimizer
npm install

2. Configure

Edit wrangler.jsonc to set your destination URLs:

{
  "name": "otlp-optimizer",
  "main": "src/index.ts",
  "compatibility_date": "2026-01-28",
  "compatibility_flags": ["nodejs_compat"],
  "observability": { "enabled": true },
  "vars": {
    "LOGS_DESTINATION_URL": "https://your-otel-collector.example.com/v1/logs",
    "TRACES_DESTINATION_URL": "https://your-otel-collector.example.com/v1/traces",
  }
}

3. Set Secrets

Set your authentication token and any sensitive headers:

# Optional: Token for authenticating incoming requests
wrangler secret put AUTH_TOKEN
# Enter your secret token when prompted

# Optional: If destination headers contain API keys
wrangler secret put LOGS_DESTINATION_HEADERS
# Enter JSON like: {"Authorization": "Bearer your-api-key"}

wrangler secret put TRACES_DESTINATION_HEADERS
# Enter JSON like: {"X-Api-Key": "your-api-key"}

4. Deploy

wrangler deploy

5. Configure Your OTLP Exporter

Point your OpenTelemetry exporter to the worker URL:

https://otlp-optimizer.<your-subdomain>.workers.dev/v1/logs https://otlp-optimizer.<your-subdomain>.workers.dev/v1/traces

If you have set an AUTH_TOKEN, include it in the headers:

Authorization: Bearer your-auth-token

Configuration Reference

Environment Variables

Variable	Description
AUTH_TOKEN	Bearer token for authenticating incoming requests. Set as a secret.
LOGS_DESTINATION_URL	URL to forward optimized logs (e.g., https://api.honeycomb.io/v1/logs)
TRACES_DESTINATION_URL	URL to forward optimized traces (e.g., https://api.honeycomb.io/v1/traces)
LOGS_DESTINATION_HEADERS	JSON object of headers to add to logs requests (default: {})
TRACES_DESTINATION_HEADERS	JSON object of headers to add to traces requests (default: {})

Header Configuration

Headers are specified as JSON objects. For sensitive values (API keys), use secrets:

# Headers should be set as secrets:
wrangler secret put LOGS_DESTINATION_HEADERS
# Then enter: {"Authorization": "Api-Key your-key", "X-Custom": "value"}

API Endpoints

POST /v1/logs

Receives OTLP logs, optimizes, and forwards to LOGS_DESTINATION_URL.

POST /v1/traces

Receives OTLP traces, optimizes, and forwards to TRACES_DESTINATION_URL.

Request Requirements

• Method: POST
• Content-Type: application/json
• Authorization: Bearer <AUTH_TOKEN>
• Content-Encoding (optional): gzip or deflate

Response Codes

Code	Description
200	Success - payload optimized and forwarded
400	Invalid JSON or malformed OTLP payload
401	Missing or invalid Bearer token
404	Unknown endpoint
405	Method not allowed (must be POST)
415	Unsupported Content-Type or Content-Encoding
502	Destination returned an error

Local Development

Run Tests

npm test

Run Tests in Watch Mode

npm test -- --watch

Local Dev Server

wrangler dev

Note: You'll need to set environment variables locally. Create a .dev.vars file:

AUTH_TOKEN=dev-token
LOGS_DESTINATION_URL=https://your-dev-endpoint.example.com/v1/logs
TRACES_DESTINATION_URL=https://your-dev-endpoint.example.com/v1/traces
LOGS_DESTINATION_HEADERS={}
TRACES_DESTINATION_HEADERS={}

Example: Honeycomb Configuration

// wrangler.jsonc
{
  "vars": {
    "LOGS_DESTINATION_URL": "https://api.honeycomb.io/v1/logs",
    "TRACES_DESTINATION_URL": "https://api.honeycomb.io/v1/traces"
  }
}

# Set secrets
wrangler secret put AUTH_TOKEN
# Enter: your-chosen-auth-token

wrangler secret put LOGS_DESTINATION_HEADERS
# Enter: {"X-Honeycomb-Team": "your-api-key", "X-Honeycomb-Dataset": "your-dataset"}

wrangler secret put TRACES_DESTINATION_HEADERS
# Enter: {"X-Honeycomb-Team": "your-api-key", "X-Honeycomb-Dataset": "your-dataset"}

Example: Grafana Cloud Configuration

// wrangler.jsonc
{
  "vars": {
    "LOGS_DESTINATION_URL": "https://otlp-gateway-prod-us-central-0.grafana.net/otlp/v1/logs",
    "TRACES_DESTINATION_URL": "https://otlp-gateway-prod-us-central-0.grafana.net/otlp/v1/traces"
  }
}

wrangler secret put AUTH_TOKEN
# Enter: your-chosen-auth-token

wrangler secret put LOGS_DESTINATION_HEADERS
# Enter: {"Authorization": "Basic <base64-encoded-credentials>"}

wrangler secret put TRACES_DESTINATION_HEADERS
# Enter: {"Authorization": "Basic <base64-encoded-credentials>"}