add: track ephemeral dust #103
Conversation
tankyleo
force-pushed
the
ephemeral-dust
branch
4 times, most recently
from
395a212 to
58b01e2
Compare
|
Cool! Code wise this looks good. Did you do any analysis yet into how often it detected a ephemeral dust spent? |
|
Here are some quick numbers: SELECT
block_stats.height,
tx_spending_ephemeral_dust, pool_id
FROM tx_stats
JOIN block_stats ON tx_stats.height = block_stats.height
WHERE tx_spending_ephemeral_dust > 0
ORDER BY block_stats.height;
Note that I started |
|
Then looking at when pools mined their first ephemeral dust: SELECT
t.pool_id,
MIN(CASE WHEN t.tx_spending_ephemeral_dust > 0 THEN t.height END) AS first_ephemeral_dust_height,
MIN(CASE WHEN t.tx_spending_ephemeral_dust > 0 THEN t.date END) AS first_ephemeral_dust_date
FROM (
SELECT
bs.date,
bs.height,
ts.tx_spending_ephemeral_dust,
bs.pool_id
FROM tx_stats ts
JOIN block_stats bs ON ts.height = bs.height
WHERE ts.tx_spending_ephemeral_dust > 0
) t
GROUP BY t.pool_id
ORDER BY first_ephemeral_dust_date;
|
|
Nice, I've started a full sync with this branch too and will have a look as well. I want to build a chart of ephemeral dust spends, then this is good to go. |
Feel free to post these stats to https://bnoc.xyz/t/bitcoin-core-versions-run-by-mining-pools/57 if you want. It seems to me like MaraPool was fairly quick to update. Did AntPool upgrade directly to v29, likely skipping v28? SpiderPool updated to v29 sometime around ~2025-09-07 after having upgraded to v28 on ~2025-07-28. F2Pool upgraded to v28 ~2025-05-23 and then v29 around ~2025-06-14? |
Yes, although Antpool still mines some blocks dropping ephemeral dust spends today. I'm thinking they are running v29 on only some of their nodes, serving different templates to subsets of their miners. Would that correspond to your past observations of Antpool ? |
|
That is an overall caveat for this PR: a mining pool mining an ephemeral dust spend on one block does not guarantee that they will mine ephemeral dust spend on all their blocks. |
Will do once you are able to reproduce the results here on your machine :) |
In your observation tooling do you have anything to audit blocks produced my miners, as compared to the template served by your local nodes ? If not would that be an interesting addition ? mempool.space has an audit feature, but I am not very confident in it. For example it shows this transaction as "expected in block" when their own data shows that this transaction should have been picked up by the previous block according to v29 policy, since it was confirmed "after 28 minutes". https://mempool.space/tx/9a48a07affedb6b4bae1888c0876d813a5d132f8020ca5d265bb27738673d01f#vout=0 |
Nonetheless, when hovering over this transaction in the audit of the previous block, it is clearly marked as "Removed". |
Yes. My assumption is that they run multiple nodes in different locations. And maybe even there have diversity of nodes they query for templates. This makes sense from a high availability standpoint. E.g.:
Just recently they mined two blocks at the same height with different block contents but the same block header time. This is a clear indication that they serve different work to different miners: https://bnoc.xyz/t/antpool-mines-two-blocks-at-height-925051/60 |
This also applies to F2Pool; in this block: |
I do and I think this actually inspired the block audit feature on mempool-space! See https://b10c.me/projects/016-miningpool-observer/ and https://miningpool.observer - but note that I only recently upgrade the node behind it and forgot to turn the backend back on, so the current data isn't reliable for ephemeral dust. |
Awesome thank you no rush. I wonder if we could use data from miningpool.observer in addition to the data collected by this PR to answer the question: "out of all blocks where mining pool X was expected to mine ephemeral dust, what fraction of their blocks did include ephemeral dust". |
Running this on a database with the full chain indexed results in the following for me:
This seems to indicate a few false positives to me? Not sure if it's possible to filter these out. Also, Ocean seems to have mined its first ephemeral dust spent in 925458? |
|
With the above fixups (will squash before merge), I now get the following results on a full chain scan:
I added all of the false positives to the tests so we can make tweaks to the code and quickly figure out whether the changes would catch these false positives. You should see that asserting only that the parent transaction is V3 filters those false positives. I now also assert that the child transaction has a non-zero fee, and is V3 with the goal of filtering any future false positives. Finally I now filter out any coinbase transactions as we are only interested in transactions submitted via the p2p network.
Yes they mined the ephemeral dust I've been sprinkling on the blockchain recently |
|
Let me know if you'd like to tighten the conditions further; for example a chain of V3 transactions alternating between zero fee and non-zero fee would still be counted as ephemeral dust even though they violate the single-child TRUC policy. |
Do you think there's a reason someone is going to make transactions with this patter? Or possibly: is this policy going to be loosed anytime soon? I haven't been following these developments closely. |
backend/src/stats.rs
Outdated
| // We do not include any cases of ephemeral dust on coinbase transactions (these transactions have their | ||
| // `tx.fee` set to `None`); we are only interested in cases of ephemeral dust that could have been submitted | ||
| // via the p2p network. | ||
| if let (Some(Amount::ZERO), 3) = (tx.fee, tx.version) { |
There was a problem hiding this comment.
nit: Is this equivalent to the following?
| if let (Some(Amount::ZERO), 3) = (tx.fee, tx.version) { | |
| if tx.fee == Some(Amount::ZERO) && tx.version == 3 { |
If so, then I think that might be a little easier to read.
I am not aware of anything changing in the near future. It seems like the current restrictions are ok for the vast majority of use cases. |
|
Addressed feedback, also removed all the newlines from the |
AFAIK they don't do any dust checks at all, so they by default would support eph dust |
|
Looks good to me, feel free to squash. I'll do a final review pass tomorrow. Good idea with the newline removal from test files. Maybe extract the new line removal from existing test files into it's own commit. |
Eclair and LDK have recently reached cross-compatibility on zero-fee commitment channels, and LDK has just released v0.2 which allows people to open such channels. Nonetheless, the commitment transactions of zero-fee commitment channels will propagate to miners only if their nodes accept ephemeral dust. Timely confirmation of commitment transactions is crucial to the security model of Lightning, hence the need to observe which mining pools mine ephemeral dust. Here, we make a best-effort attempt at only counting the transactions in a block spending ephemeral dust that could have been submitted via the P2P network. We are not interested in coinbase transactions, or transactions submitted out-of-band. Therefore, we assume that any transaction spending ephemeral dust is version 3, non-zero-fee, smaller than 1000vB, and spends from a version 3, zero-fee transaction itself smaller than 10_000vB. See TRUC policy rules defined in BIP-431 for further details. For now, we do not assert that the parent transaction does not have any ancestors in the same block, or that the child transaction does not have any descendants in the same block (this would violate current policy on TRUC packages). This check could be added in the future should false-positives appear in the data.
tankyleo
force-pushed
the
ephemeral-dust
branch
from
d855e14 to
85fc0ac
Compare
|
Squashed the commits with the following diff diff --git a/backend/src/stats.rs b/backend/src/stats.rs
index 8fb0bae..488857b 100644
--- a/backend/src/stats.rs
+++ b/backend/src/stats.rs
@@ -385,9 +385,9 @@ impl TxStats {
.take(&(txid, *vout))
.is_some()
&& tx.version == 3
- // unwrap safety: this is a child transaction of a parent in the same block, hence it must not be a
- // coinbase transaction
- && tx.fee.unwrap() != Amount::ZERO;
+ // unwrap safety: we filter for non-coinbase inputs above, hence this transaction is non-coinbase
+ && tx.fee.unwrap() != Amount::ZERO
+ && tx.vsize <= 1_000;
if (tx_spending_ephemeral_dust || ephemeral_dust_outpoints_in_this_block.is_empty())
&& tx_spending_newly_created_utxos
@@ -405,7 +405,7 @@ impl TxStats {
// We do not include any cases of ephemeral dust on coinbase transactions (these transactions have their
// `tx.fee` set to `None`); we are only interested in cases of ephemeral dust that could have been submitted
// via the p2p network.
- if tx.fee == Some(Amount::ZERO) && tx.version == 3 {
+ if tx.version == 3 && tx.fee == Some(Amount::ZERO) && tx.vsize <= 10_000 {
let staged_ephemeral_dust_outpoints: Vec<_> = tx
.output
.iter()Note I now also assert that the parent is <= 10_000vB and the child is <= 1000vB, see BIP431. |
|
LGTM! Thank you! I'll set up a chart for this. |
Thanks for the review and thanks for the chart ! |
Adding chart in #119 |
3 participants
As part of #101