Skip to content

Conversation

@kaimast
Copy link
Contributor

@kaimast kaimast commented Dec 3, 2025

Fixes #4013.

This PR adds a new node-data directory that contains all node-specific data (essentially everything but the ledger). This is located at ~/.aleo/storage/node-data-{network_id} for production or ./node-data-{network_id}-{dev} for development nodes. Alternatively, users can also pass a value to --node-data and store the data at a custom path.

At startup, a node will check for any files that indicate that the old storage format is used. In that case, a message is printed in how to migrate, and the node will not start. Additionally, the FAQ in the README contains information on how to migrate.

Alternatively, nodes can also be started with --auto-migrate-node-data to do this automatically. This is not enabled by default out of extra precaution.

@kaimast kaimast changed the base branch from staging to feat/signal-handling December 3, 2025 00:00
@kaimast kaimast force-pushed the feat/filesystem-reorg branch from 6e62037 to 66e9415 Compare December 3, 2025 00:25
@kaimast kaimast changed the base branch from feat/signal-handling to staging December 3, 2025 00:26
@kaimast kaimast marked this pull request as ready for review December 3, 2025 00:59
@kaimast kaimast requested a review from vicsn December 3, 2025 00:59
@kaimast kaimast marked this pull request as draft December 3, 2025 01:00
@kaimast kaimast changed the base branch from staging to feat/signal-handling December 3, 2025 01:01
Base automatically changed from feat/signal-handling to staging December 5, 2025 09:44
@vicsn
Copy link
Collaborator

vicsn commented Dec 5, 2025

We need to either provide a migration tool (e.g., snarkos migrate) or a document on how to move to the new layout.

I think documentation in the snarkOS README, release and release blog should be sufficient. .current-proposal-cache*, cached_*_peers and jwt_secret_*.txt are all written at every startup/shutdown.

I picked the name node-config as a placeholder, and we can consider changing it.

I think this may not reflect how important it is for validators to keep .current-proposal-cache*. Please bring some alternatives to the next protocol weekly sync.

Should nodes check for the new directory, so that even if --node-config-dir is not passed, the new layout is used?

Given that backups are essentially required are for validators, what about we make it a mandatory flag for --validator nodes so they're forced to think about it. I will be very clear about it in the next release.

@vicsn vicsn mentioned this pull request Dec 5, 2025
@kaimast kaimast force-pushed the feat/filesystem-reorg branch from 66e9415 to ddda90a Compare December 5, 2025 19:10
@kaimast kaimast force-pushed the feat/filesystem-reorg branch 2 times, most recently from eef8c59 to 601266f Compare December 16, 2025 23:57
@kaimast kaimast changed the base branch from staging to fix/scripts December 16, 2025 23:58
Base automatically changed from fix/scripts to staging December 18, 2025 02:32
@kaimast kaimast force-pushed the feat/filesystem-reorg branch 3 times, most recently from 4a27527 to 7ce27dc Compare December 23, 2025 17:36
@kaimast kaimast marked this pull request as ready for review December 23, 2025 17:36
@kaimast kaimast force-pushed the feat/filesystem-reorg branch from 7ce27dc to 743f6df Compare December 24, 2025 12:51
@kaimast kaimast requested a review from vicsn December 24, 2025 12:53
Copy link
Collaborator

@vicsn vicsn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there!

@vicsn
Copy link
Collaborator

vicsn commented Dec 26, 2025

Note this may need a minor merge to include #4054 once it's ready

@kaimast
Copy link
Contributor Author

kaimast commented Jan 2, 2026

Note this may need a minor merge to include #4054 once it's ready

c8f5891 merged staging include #4054's changes.

@vicsn vicsn requested a review from ljedrz January 3, 2026 11:57
Copy link
Collaborator

@ljedrz ljedrz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the general direction, there are just a few errors left to iron out, especially this one.

Copy link
Collaborator

@ljedrz ljedrz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left one last comment, but LGTM otherwise 👍.

Copy link
Collaborator

@vicsn vicsn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@vicsn vicsn merged commit d23be76 into staging Jan 9, 2026
4 checks passed
@vicsn vicsn deleted the feat/filesystem-reorg branch January 9, 2026 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Proposal] Make storage to filesystem less error-prone

4 participants