Add rejection reason storage to finalize_store

[Copilot]

Rejected transactions now store their rejection reasons in a new rejection_reason_map within the finalize storage, enabling API queries for why transactions were rejected.

Changes

• Storage layer: Added RejectionReasonMap DataMap to FinalizeStorage trait with implementations for both RocksDB and Memory backends
• FinalizeStore API: Added methods to insert/retrieve rejection reasons by transaction ID
  • insert_rejection_reason(transaction_id, reason)
  • get_rejection_reason(transaction_id)
  • contains_rejection_reason(transaction_id)
• VM integration: Modified VM::atomic_speculate_inner to store rejection reasons when creating rejected deployment or execution transactions
• Atomic operations: Integrated rejection_reason_map into all atomic batch operations (start, checkpoint, rewind, abort, finish)

Example Usage

// After speculation, query why a transaction was rejected
let tx_id = rejected_transaction.id();
if let Some(reason) = vm.finalize_store().get_rejection_reason(&tx_id)? {
    println!("Transaction rejected: {}", reason);
    // Output: "Failed to finalize execution: division by zero"
}

Rejection reasons include descriptive error context such as "Failed to finalize deployment: {error}" or "Program {id} has already been deployed in this block".

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[Feature] Add build feature to store rejection error strings to finalize_store</issue_title>
<issue_description>## 🚀 Feature

In this PR we ensure that rejection reasons are logged: https://github.com/ProvableHQ/snarkVM/pull/2970/files

We want to be able to surface these rejection reasons via an API, so we should make sure they are stored to a new rocksdb DataMap

Partially fixes ProvableHQ/snarkOS#3934</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes [Feature] Add build feature to store rejection error strings to finalize_store #3100

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[ljedrz]

I would recommend against a String rejection reason; the storage might be subject to compression, but I'm not confident a string like Program {id} has already been deployed in this block would compress reliably, despite both the ID already present in a key and a constant error context string.

If we want the tx rejection reasons to be both auditable and lightweight, we should isolate them into a dedicated enum.

[ljedrz]

@vicsn do we want to be specific about the nature of the finalization errors?

[vicsn]

@vicsn do we want to be specific about the nature of the finalization errors?

Yes, you can get inspiration from these errors: https://github.com/ProvableHQ/snarkVM/pull/3081/files#diff-7d838a571bdcbb8174c371914f0794cc9881157d10a2115b374cbc6b0021618eR117

Which could in turn enable pretty-prenting something like: Finalize instruction (add r0 r1 into r2) at index 7 with values r0={r0}, r1={r1}

[ljedrz]

Blocked by #3122.

[vicsn]

Can we have a more symmetric FailedToDeploy(reason)

[vicsn]

Perhaps this is still on your roadmap, but the primary usecase I had in mind was to fill this enum with the logged error present in this scope, so we learn why an Execution fails.

Now that you did all the work on StackInit errors already, I guess we can keep them, but where possible feel free to leave stuff as future work.

[vicsn]

I wonder how maintainable all of these enums are.

Perhaps this was what you were previously warning me about.

If we want to remove or change any of these errors, the serialization will change.

And while we may be able to prevent external indexers from relying on the format, a node's own database can end up misinterpreting it's own data.

If we want to stick with this approach, I guess we need versioning in some places? Perhaps a unit test can warn us if we change the wrong things?

And if that's not feasible, perhaps we just stick to untyped strings...

[vicsn]

Update: for deployments, we should just have a single high level enum DeploymentError, we don't have to go more granular. For executions, we should try and include the instruction line which failed. For other types of finalization errors, we should discuss whether it's worth to make enums.

I'd still be keen to know how to incorporate versioning in the enum serialization or types.

[kaimast]

We should annotate these with #[non_exhaustive], so adding a new error does not break the API.