fix(provider): correct datum endpoint, delegation pool id, and UTxO mapping in Koios, Kupmios, Blockfrost

.changeset/fix-provider-conformance.md

@@ -0,0 +1,18 @@
+---
+"@evolution-sdk/evolution": patch
+---
+
+Fix several provider mapping bugs that caused incorrect or missing data in `getDelegation`, `getDatum`, and `getUtxos` responses.
+
+**Koios**
+
+- `getDelegation`: was decoding the pool ID with `PoolKeyHash.FromHex` but Koios returns a bech32 `pool1…` string — switched to `PoolKeyHash.FromBech32`
+- `getUtxos`: `datumOption` and `scriptRef` fields were never populated — all UTxOs returned `datumOption: null, scriptRef: null` regardless of on-chain state. Now correctly maps inline datums, datum hashes, and native/Plutus script references.
+
+**Kupmios (Ogmios)**
+
+- `getDelegation`: the Ogmios v6 response is an array, but the code was using `Object.values(result)[0]` which silently produced wrong data on some responses. Switched to `result[0]`. Also corrected the field path from `delegate.id` to `stakePool.id` to match the v6 schema, and decoded the bech32 pool ID through `Schema.decode(PoolKeyHash.FromBech32)` so the return type satisfies `Provider.Delegation`.
+
+**Blockfrost**
+
+- `getDatum`: was calling `/scripts/datum/{hash}` which returns only the data hash — should be `/scripts/datum/{hash}/cbor` to get the actual CBOR-encoded datum value. Switched endpoint and response schema to `BlockfrostDatumCbor`.

.env.test.local.example

@@ -0,0 +1,41 @@
+# Provider integration test configuration
+# Copy this file to .env.test.local and fill in your keys.
+# .env.test.local is gitignored — never commit it.
+
+# ── Koios ─────────────────────────────────────────────────────────────────────
+# Koios uses a public endpoint — no API key needed.
+# Set KOIOS_ENABLED=true to opt in (off by default to avoid rate limiting CI).
+# KOIOS_ENABLED=true
+# Optional: override the default public preprod endpoint
+# KOIOS_PREPROD_URL=https://preprod.koios.rest/api/v1
+
+# ── Blockfrost ────────────────────────────────────────────────────────────────
+# Required to run Blockfrost tests. Get a key at https://blockfrost.io
+BLOCKFROST_PREPROD_KEY=preprodXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
+# Optional: override the default preprod endpoint
+# BLOCKFROST_PREPROD_URL=https://cardano-preprod.blockfrost.io/api/v0
+
+# ── Maestro ───────────────────────────────────────────────────────────────────
+# Required to run Maestro tests. Get a key at https://gomaestro.org
+MAESTRO_PREPROD_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
+# Optional: override the default preprod endpoint
+# MAESTRO_PREPROD_URL=https://preprod.gomaestro-api.org/v1
+
+# ── Kupmios (self-hosted or Demeter.run) ──────────────────────────────────────
+# Both URLs are required to run Kupmios tests.
+# If you use Demeter, provide one Kupo key and one Ogmios key.
+# Leave this section unset if you want Kupmios tests to stay skipped.
+KUPMIOS_KUPO_URL=https://your-kupo-endpoint
+KUPMIOS_OGMIOS_URL=https://your-ogmios-endpoint
+KUPMIOS_KUPO_KEY=your-kupo-key
+KUPMIOS_OGMIOS_KEY=your-ogmios-key
+
+# Optional overrides if Kupo / Ogmios need different headers:
+# KUPMIOS_KUPO_HEADER_JSON={"dmtr-api-key":"your-kupo-key"}
+# KUPMIOS_OGMIOS_HEADER_JSON={"dmtr-api-key":"your-ogmios-key"}
+
+# Self-hosted local example:
+# KUPMIOS_KUPO_URL=http://localhost:1442
+# KUPMIOS_OGMIOS_URL=http://localhost:1337
+# KUPMIOS_KUPO_KEY=
+# KUPMIOS_OGMIOS_KEY=

.gitignore

@@ -18,3 +18,7 @@ docs/.twoslash
 # Ignore debug
 debug/
 CLAUDE.md
+
+# Local env overrides — never commit API keys
+.env.local
+.env.test.local

docs/content/docs/transactions/meta.json

@@ -4,6 +4,7 @@
     "index",
     "first-transaction",
     "simple-payment",
-    "multi-output"
+    "multi-output",
+    "retry-safe"
   ]
 }

docs/content/docs/transactions/retry-safe.mdx

@@ -0,0 +1,184 @@
+---
+title: "Retry-Safe Transactions"
+description: "Build transactions that automatically retry when the node rejects inputs due to indexer lag"
+---
+
+# Retry-Safe Transactions
+
+Structure your build-sign-submit pipeline so that retrying re-reads all chain state from scratch every time.
+
+## The Problem
+
+This is one of the most common challenges Cardano developers face when building applications that submit sequential transactions.
+
+When you submit a transaction it doesn't immediately become part of the chain. It first enters the **mempool** of the node you submitted to. A block producer then picks it up and includes it in a new **block**. That block propagates across the network, gets validated, and is attached to the chain. Only once your provider node has received and processed that block does its UTxO set reflect the spent inputs — a process that typically takes 10–30 seconds, and can be longer under network congestion or in the event of a chain fork.
+
+Until that happens, the UTxOs consumed by your transaction still appear as unspent when you query your provider. If you immediately build the next transaction using those stale UTxOs, the node will reject it with `BadInputsUTxO` — because from the ledger's perspective, those inputs no longer exist.
+
+This is not a bug. It is an inherent property of how Cardano's consensus and block propagation work.
+
+The fix is straightforward: **all chain state reads must happen inside the action**, not before it. UTxOs, script UTxOs, datums, oracle values — anything queried from your provider must be re-read on every attempt so each retry works with the freshest available view of the chain.
+
+## How It Works
+
+An "action" is the complete unit of work — read chain state, build, sign, and submit — wrapped in a single retryable function or Effect. When the node rejects the transaction, the retry re-runs from the top, re-reading everything before building again.
+
+```
+retry attempt N
+  └─ read chain state     ← fresh every attempt (UTxOs, datums, script state, ...)
+  └─ build tx
+  └─ sign
+  └─ submit to node
+       ├─ accepted → done
+       └─ BadInputsUTxO → retry attempt N+1
+```
+
+Querying chain state **outside** the action and passing it in as a static value defeats this — the same snapshot is reused on every retry.
+
+## Usage
+
+### Plain async with manual retry
+
+The simplest approach: wrap the full pipeline in an async function and call it from a retry loop.
+
+```ts twoslash
+import { Address, Assets, createClient } from "@evolution-sdk/evolution";
+
+const client = createClient({
+  network: "preprod",
+  provider: {
+    type: "blockfrost",
+    baseUrl: "https://cardano-preprod.blockfrost.io/api/v0",
+    projectId: process.env.BLOCKFROST_API_KEY!
+  },
+  wallet: { type: "seed", mnemonic: process.env.WALLET_MNEMONIC!, accountIndex: 0 }
+});
+
+const recipient = Address.fromBech32("addr_test1vrm9x2dgvdau8vckj4duc89m638t8djmluqw5pdrFollw8qd9k63");
+
+// The action fetches UTxOs at call time — safe to retry
+async function sendPayment() {
+  const tx = await client
+    .newTx()
+    .payToAddress({ address: recipient, assets: Assets.fromLovelace(2_000_000n) })
+    .build();
+
+  const signed = await tx.sign();
+  return signed.submit();
+}
+
+// Simple retry with delay
+async function withRetry<T>(action: () => Promise<T>, retries = 3, delayMs = 3000): Promise<T> {
+  for (let attempt = 1; attempt <= retries; attempt++) {
+    try {
+      return await action();
+    } catch (err) {
+      if (attempt === retries) throw err;
+      await new Promise(resolve => setTimeout(resolve, delayMs));
+    }
+  }
+  throw new Error("unreachable");
+}
+
+const txHash = await withRetry(sendPayment);
+console.log("Submitted:", txHash);
+```
+
+### With script UTxOs
+
+When collecting from a script address, query the script UTxOs inside the action so each retry gets a fresh view of what is available at that address.
+
+```ts twoslash
+import { Address, Assets, createClient } from "@evolution-sdk/evolution";
+
+const client = createClient({
+  network: "preprod",
+  provider: {
+    type: "blockfrost",
+    baseUrl: "https://cardano-preprod.blockfrost.io/api/v0",
+    projectId: process.env.BLOCKFROST_API_KEY!
+  },
+  wallet: { type: "seed", mnemonic: process.env.WALLET_MNEMONIC!, accountIndex: 0 }
+});
+
+// Illustrative snippet (not runnable as-is) — redeemer and scriptAddress are placeholders
+async function unlockFromScript() {
+  const scriptAddress = Address.fromBech32("addr_test1...");
+  const recipient = Address.fromBech32("addr_test1vrm9x2dgvdau8vckj4duc89m638t8djmluqw5pdrFollw8qd9k63");
+
+  // Script UTxOs fetched inside the action — re-run on every retry
+  const scriptUtxos = await client.getUtxos(scriptAddress);
+
+  const tx = await client
+    .newTx()
+    .collectFrom({ inputs: scriptUtxos })
+    .payToAddress({ address: recipient, assets: Assets.fromLovelace(5_000_000n) })
+    .build();
+
+  const signed = await tx.sign();
+  return signed.submit();
+}
+```
+
+### Using Effect for structured retry
+
+When using Effect, compose the full pipeline as a single `Effect.gen` and apply `Effect.retry` directly. `Schedule` controls the timing and number of attempts.
+
+```ts twoslash
+import { Address, createClient } from "@evolution-sdk/evolution";
+import { Effect, Schedule } from "effect";
+
+const client = createClient({
+  network: "preprod",
+  provider: {
+    type: "blockfrost",
+    baseUrl: "https://cardano-preprod.blockfrost.io/api/v0",
+    projectId: process.env.BLOCKFROST_API_KEY!
+  },
+  wallet: { type: "seed", mnemonic: process.env.WALLET_MNEMONIC!, accountIndex: 0 }
+});
+
+// Illustrative snippet (not runnable as-is) — scriptAddress and redeemer are placeholders
+const unlockAction = Effect.gen(function* () {
+  // Script UTxOs fetched fresh on every attempt
+  const scriptUtxos = yield* client.Effect.getUtxos(Address.fromBech32("addr_test1vrm9x2dgvdau8vckj4duc89m638t8djmluqw5pdrFollw8qd9k63"));
+
+  const signBuilder = yield* client.newTx()
+    .collectFrom({ inputs: scriptUtxos })
+    .buildEffect();
+
+  return yield* signBuilder.Effect.signAndSubmit();
+});
+
+// Retry up to 3 times with a 3-second delay between attempts
+const txHash = await unlockAction.pipe(
+  Effect.retry(Schedule.recurs(3).pipe(Schedule.addDelay(() => "3 seconds"))),
+  Effect.runPromise
+);
+
+console.log("Submitted:", txHash);
+```
+
+`Effect.retry` re-runs the entire `Effect.gen` block on failure — every chain state read inside it is re-executed on each attempt.
+
+## Gotchas
+
+- **Read all chain state inside the action, not outside.** Any indexer call made before the action — UTxOs, datums, script state, oracle values — captures a snapshot that is reused on every retry. Move those reads inside the action so each attempt queries the indexer fresh.
+- **Retry does not fix insufficient funds.** If the wallet genuinely does not have enough ADA, the node will reject for a different reason and retrying will always fail. Check balances before entering a retry loop.
+- **`Effect.retry` retries on any failure by default.** If you use Kupmios (which submits directly via Ogmios to the node), you can narrow retries to stale-input rejections specifically by matching `"BadInputsUTxO"` in the error message — this is the node's ledger validation error surfaced through the submission response:
+
+  ```ts
+  Effect.retry(
+    Schedule.recurs(3).pipe(Schedule.addDelay(() => "3 seconds")),
+    { while: (err) => err.message.includes("BadInputsUTxO") }
+  )
+  ```
+
+  Other indexers relay the same node error in different formats — check the raw cause for the specific message.
+- **Indexer lag is not instant.** A 0ms retry delay may still read the same stale data. Add at least a 2–3 second delay between attempts.
+
+## Next Steps
+
+- [Simple Payment](/docs/transactions/simple-payment) — Basic transaction building
+- [First Transaction](/docs/transactions/first-transaction) — Complete walkthrough
+- [Error Handling](/docs/advanced/error-handling) — Typed errors with Effect

docs/package.json

@@ -38,6 +38,7 @@
     "@types/react": "^19.1.11",
     "@types/react-dom": "^19.1.7",
     "baseline-browser-mapping": "^2.9.8",
+    "effect": "^3.19.3",
     "postcss": "^8.5.6",
     "shiki": "^3.15.0",
     "tailwindcss": "^4.1.12",

package.json

@@ -56,6 +56,7 @@
     "tsx": "^4.20.4",
     "turbo": "^2.5.6",
     "typescript": "^5.9.2",
+    "vite": "^6.0.5",
     "vitest": "^3.2.4"
   },
   "engines": {

packages/evolution/src/sdk/provider/internal/BlockfrostEffect.ts

@@ -557,8 +557,8 @@ export const getDatum = (baseUrl: string, projectId?: string) => (datumHash: Dat
   const datumHashHex = Bytes.toHex(datumHash.hash)
   return withRateLimit(
     HttpUtils.get(
-      `${baseUrl}/scripts/datum/${datumHashHex}`,
-      Blockfrost.BlockfrostDatum,
+      `${baseUrl}/scripts/datum/${datumHashHex}/cbor`,
+      BlockfrostDatumCbor,
       createHeaders(projectId)
     ).pipe(
       Effect.flatMap((datum) => {