CHIP-0049: 3.0 Fork Info #161
Conversation
danieljperry
changed the title
|
This CHIP from Arvid Norberg provides the necessary timelines for the hard fork to support CHIP-48. It is now a |
|
From my point of view, there are two 'phases' of the timeline discussed in this CHIP:
I do not have an opinion regarding the timing of the first stage. CNI has an amazing blockchain team, and I'm confident the proposed changes will go through extensive testing before Chia 3.0 is rolled out. Security of the transition and the new format must come on the first place. That being said, I have several arguments for a significantly shorter hard fork activation height & phase-out period:
With these arguments laid out, I'd like to ask that the hard fork activates in the second week of January, and that the phase-out period takes 32 days. I consider these to be reasonable timelines assuming Chia 3.0 is out before January 1st, 2026. As is the spirit of CHIPs, I'm interested in hearing arguments for and against short timelines post-Chia 3.0. Thank you very much for this CHIP, and for all the work put towards the new format so far. Chia's new consensus and its approach to decentralization were the main things that first attracted me to the community. I'm glad to see the vision is still alive, and I hope we can adopt a timeline that puts the 'grinding phase' behind us as soon as possible now that there's a clear path forward. |
|
I fully support Yak's post. I don't have much more to say other than the security and robustness of the chain outweighs trying to avoid ruffling some feathers. |
|
Congrats on the CHIP release! I haven't had a chance to deep dive into the proof of space format itself but I had open questions/thoughts about this hard fork proposal.
|
|
Disclaimer: I'm just giving my personal opinions below, not speaking on behalf of CNI or anyone else. I'm very excited about the prospect of Chia's NC climbing back up to where it was at before, and I agree that it would be ideal if the timelines can be expedited. However I also think it's important to balance this with the fact that this is perhaps the largest and most controversial (in terms of netspace) change made to Chia since mainnet launch. It's important that enough time is given for:
If the implementation is rushed or not properly tested it could lead to disaster or needing to make further changes earlier than intended. In general a hard fork must be done with extreme caution, as an insufficient amount of the netspace being upgraded by the time it activates could lead to large reorgs, a chain split, and general network instability. When you bring a complete overhaul of consensus into the mix, there's even more risk. However, the activation and transition times are debatable. I suppose the safe default answer is we should wait a long period of time to make sure everyone is up to date in time (currently the CHIP proposes roughly 6 months for activation and 8.5 months for transitioning the old plots out). And on the flip side, many are anxious for this to be finished as soon as possible to bring the NC, netspace, and node count back up, and preserve the network's security against grinding and rental attacks. My gut instinct would be to pick somewhere in the middle of the two proposed timelines. Something like:
This is based on some rough data I've seen on how long it takes for previous versions to reach majority netspace, as well as how long I think it would realistically take for most farmers to upgrade and replot. In other words, I think we should aim to have the new format activate and transition as fast as possible without putting the network's security at risk or making it a pain to replot in such a timeframe. Curious to see what other people think about this. Take my timeline with a grain of salt, as it's not based on much. |
|
I am also in favor for a more quick activation and phase-out. Not sure what are the best numbers but my assomptions would be something like: Hard-fork activation: ~2 months after the We can see in the graph from the Version History Tab on dashboard.chia.net, that it always takes roughly 1 month for a version to be adopted by a majority of nodes. This is a big enough event on the Chia blockchain to expect an active response from the community. If the hard-fork can occur in January 2026, the 90 days would still allow the northern hemisphere to replot during the cold season. I also support point 8 from SlowestTimelord for an offset timeline between the filter reduction and the plot difficulty adjustment. |
|
It is my belief that there should be no downside to replotting aside from losing compression. New plots should be 100% effective from the start if it is possible to do so. Otherwise there is a deterrent toward replotting. As a small farmer, there shouldn't be a question between do I plot for an hour a day replacing plots, wait 'till the middle and spend 2 days replotting everything, ect. The incentive should be to get done as quickly as possible. |
This could have a perverse effect of making Netspace fall very hard in the first hours and days of the switch. That's why a hard cut over is dangerous from an overall security perspective. |
While I agree that there is a potential for this - I do not see that being the likely outcome. You pose a risk that farmers will shut off entire farms while I do not believe that would be the case as there is still an incentive to farm old plots so most farmers will remove and replace at the smallest level possible for their setup. Be that plot by plot (I personally wrote a batch file that replaced old plots one at a time when I replotted for pooling and compression) or disk by disk. I just don't foresee a situation where anyone takes down their entire farm for any reason other than they choose not to replot at all. |
|
I don't think we want to underestimate the lure of |
If the user was going to do this, they will do this anyway. All the increasing effectiveness of new plots does is make the calculation of when more difficult and disincentivizes the only actions that would defend against it (having more new plots on the network at 100% already). In your situation - where the new plots are less effective - if we hit that 30 to 40 percent reduced effectiveness mark and new plots are only 30 to 40 percent effective instead of 100% - what happens if there is a widespread fall off of solo/official pool plots at the same time? The people who have replotted don't provide efficient security so we could end up in a situation of disproportionate compressed control. We both know that far too many of those are using compression are also unofficial means to farm that is detrimental to the network. I will admit that I may not be seeing the forest thru the trees here, but I believe there is more risk with the additional disincentive toward replotting than if it was not there. |
|
I suggest thinking about the 50%-50% point as the important time. The game will be that to maximize earnings the largest farms with the most GPUs will want to wait until just before that point to start their replot. Smaller farms will probably want to replot before them as a play to have maximal earnings against any larger farm that waited a bit too long. Also starting at 100% effectiveness for new means a nasty game of chicken that I expect a lot of farmers would resent. |
100% effectiveness of new plots is already their compression percent less effective than them in the first epoch. In fact most current farmers would not even reach 1:1 parity with new plots in the hypothetical version I proposed until the 2nd thru 4th reduction epoch. Anyone that would resent that has unreasonable expectations unless the reduction epochs are set wildly too close together. Epochs don't even need to really be the same amount of time. It could be for example that the first 30% takes 6 weeks while the remainder is cut down to 1 week per 10% leaving it roughly still half way thru that the rough 50/50 point is reached. |
|
I disagree with the commenters asking to shrink the transition window. Linear decay of existing plots over 256 days seems sensible. It allows large farmers to balance plotting time needed vs existing compression, with a clear incentive to switch. Small farms will naturally be more agile. Remember that the new format is intentionally slow to plot (~40 TiB per day on a 3060). None of the commenters have given a compelling reason to shorten the transition window. If the aim is the bring NC up as quickly as possible, or bring old farmers back into the fold, then it's the launch that should come forwards. I'd argue that 12 months is too long for the new format to start. The most compressed plots are in the 200%-300% range, so the incentive to replot these plots is around +4 months from the new format going live, or 16 months from now. It's these plots specifically which have driven NC down. Can't we bring the effective date forwards by six months? Plot difficulty schedule doesn't make sense A farmer plotting minimum difficulty would need a full replot every 2 years. Alternatively, by spending ~60% more time plotting +1 difficulty the replot is reduce to every 4 years. From a farming perspective, it doesn't ever make sense to plot the network minimum. Unless we need this 2 year cadence and granularity for griding resistance the minimum difficult should be +2 every 4 years to avoid unnecessary and inefficient replots. |
Curious to challenge all of you directly on this - you're all respected members of the community with a deep interest and commitment to the Chia ecosystem. Maybe I'm not seeing what you are. What are you actually expecting to achieve with a shorter transition? |
|
in my opinion, new plot format should be activated much sooner after 3.0. Say 30 days to activate the hardfork. At that time, both plots contribute equally. after 3 months the old plot phase out begins. 3-6 months later old format is completely deprecated. |
I've re-plotted a 16PB farm before, using overkill hardware (10TB RAM + 20x A4000). It took almost 2 months, but the planning and setup before took quite a while as well. Farmers will need to dust off their plotters, connect them all up again, debug a lot of issues again, etc etc. Anything below 6 months will have you lose netspace from large farms. |
As I have no experience at this level, I was wondering what your opinion on new plot effectiveness would be. Would you prefer the new plots have reduced effectiveness or be fully effective? With a farm of your size what would make you consider deleting all plots to start the process rather than replacing in a more granular fashion such as plot by plot, disk by disk or controller by controller? And would there be a difference in this plan if new plots were not being limited in effectiveness? |
|
Comment on the "a 3060 can do 40TB a day" statement. That output is a theoretical number and will be much lower due to drive speeds, usb, network, controller load, downtime, etc. - farms are very different. A pro has better tools with flash buffer space and/or fast networking. A kind of regular farm below 1-2PB will probably do the rm -f for 2,3,4 drives, fill them again and add back to the farm. I think the transitioning perid should be as painless as possible. 90 days is very short, 6 months sounds more practical to me. |
|
I think we also need to state the overall goal of the transition. is it to prevent NOSSD from gaining 51% of the network? A drop in netspace with a high NC doesn't seem like it would be an issue without a possible 51% attack. If thats the case, we could temporarily lower the plot filter to disrupt those with high compute |
The goal is simply to give everyone ample time to replot their farms without causing any major disruptions. It's not about singling any individuals out or intentionally disrupting those with high compute. |
What major disruptions could be caused by a drop in netspace? |
By "major disruptions" I was referring to individual farmers, not the global netspace. If a farmer has to replot tens of PBs and doesn't have sufficient time to do so before the old plots are fully phased out, then that will cause a major disruption for the farmer. If the netspace drops by a large amount in a short amount of time, the network will slow down, and then heal to its intended speed over the subsequent two days. The network itself is quite resilient to such changes. |
|
I"m willing to bet most farmers can replot their entire farm in 1-2 months. As farms grow, so do plotters. Maybe Gene or someone with a large chia following could run a poll on reddit and X. Then it would just be what others have said. how long does it take for people to transition to the upgrade and based off the poll, how long to replot. |
would also be interesting to put up a "I don't plan to replot" option too |
I would prefer they have 1:1 effectiveness from the start if I was a larger farmer. Meaning effective and physical space being the same. Since you lose the old compression on the replot. But from older discussions I remember giving the new plots a boost, not a nerf? To avoid this individual netspace loss at the beginning of the re-plotting phase.
You never do that, always delete in batches, like per JBOD or something.
Maybe most by count, but not by netspace. |
Disagree. Something I think many people miss is that whilst large farms may have a bit more GPU power, it's already in use for farming. HDDs and GPU are not even necessarily colocated, and many slow-plotted these large farms by expanding gradually. Plotting at scale has a lot of challenges that might not be encountered by small farms. At anything like the phase-out times some people are calling for I'd fail to replot everything in time and a chunk of space would drop out of the network. I expect that would happen to a majority of large farms. |
"The phase-out period would then take an additional 256 days, where the 'weight' of old plots lowers while that of plots using the new format increases." makes it sound like the way that I had understood it where new plots would start at very low effectiveness and increase in effectiveness inversely to the decrease of the old plots. Basically - creating a bell curve where somewhere around 40% or 30% effectiveness on old plots we're likely around the bottom of a bell curve where I believe we would see network capacity around 50% of current.
This is exactly what I thought would be the case but Gene seems to fear a mass rm -rf *.plot party or something - and I can somewhat see this when folks finally start ditching nossd and any other entirely incompatible plot formats - although I still think anyone that absolutely cannot bring different farmers online would likely remove drive by drive to replot and only 'mass delete' once they're about halfway thru effective capacity and can bring back the harvester by swapping to just the drives using new plots. |
|
One farmer type I am not seeing discussed here are set and forget, low technical knowledge farmers. Generally these farmers are small to mid size and have mostly self-contained farming rigs but these are definitely the types of farmers that would rm -rf plots once it makes sense for them to do so (i.e. they want to remove the plots, start the plotting of new, and not touch it again until the next update). Another aspect that has not been discussed much is plot tools that the community can help to build which would provide a more seamless upgrade path for these types of farmers and perform the plot replacements on their behalf. Personally on my farm I am likely to either use a tool like the above or just nuke plots a few drives at a time. |
|
As for activation timeline, I believe (based on the chia dashboard version history) that 30 days is far too short and the shortest time period that should be considered is 60 days but 90 at a minimum would be better. It is critical to have a majority of nodes/space transitioned before that activation and as mentioned there are a large number of set and forget farmers. |
|
My thoughts on activation timeline is why set in stone how quickly the network can switch over to the new and better format? Why not make it a simple formula that calculates from day one what the ratio of new formats vs old formats is. This would be used to calculate the effectiveness always two days out based on the ratio of the new format adoption. You can still keep in the failsafe default timeline to ensure there is a switch over. This way if it is also based on farmers switching faster it will happen sooner. So for instance: Scenario 1
Scenario 2
Using a ratio of OPOS:NPOS to calculate plot effectiveness on a delayed two day look back will ensure large amounts of netspace are not taken offline. As stated earlier most large farms will not start switching over until we reach 50:50 anyway. So this suggestion can move this window forward faster and safer based on farmers willing to switch sooner. |
|
It's been radio-silent for quite some time now, given that all of the farmers are eagerly waiting on this decision, is there a rough timescale as to when to expect being able to begin replotting to the new format so that we can make decisions about required hardware? |
We're working hard to get this across the finish line. I can't give an exact timeline at this point, but I can provide some guidance as to what the process will look like going forward. Once we have all of the components (plotter, prover, solver) ready, we'll need to do extensive testing to make sure everything works as intended. We'll pull in community members who are willing to help us out. After we have verified that everything needed for the fork is working, we'll release Chia 3.0. We'll still need to refine some of our tooling over the subsequent months, but the activation block height will be set in stone at that point. After 3.0 is released, it will be another 3-6 months (to be decided ahead of time) before it activates. During this time, you will be able to replot, but there won't be any financial incentive to do so because the new plots won't be valid yet. This will likely be a good time for most farmers to plan for the future, including any hardware requirements. Once the fork activates, the new plots will become valid. But that doesn't mean that everyone should begin replotting right away. We're planning to include an additional 256 days where the old plots will remain valid, and they will lose value on a linear scale during that period. For many farmers, it might make sense to wait a few months before replotting. We'll have some tools available to determine exactly when the best time to start replotting will be. In addition, we're working hard to make the transition as smooth as possible. Our goal is to provide an easy way to replace old plots with new ones, one plot at a time, during the 256 day transition period. |
|
The term |
danieljperry
changed the title
Provide all necessary timelines for the hard fork to support the new proof format.