Pectra Retrospective

The Nethermind team have discussed our opinions on increasing the rate at which we ship forks and thought of some ways to improve the process.

Fork Scoping

We should aim to have a provisional plan for the next 5 hard forks, mapping Vitalik’s roadmap into concrete updates. There would still be flexibility to change fork scoping, especially more distant ones, but having this provisional timeline would greatly help in deciding where to allocate resources.

For imminent forks (within a year) we should avoid adding new EIPs to the scope unless they are already implemented and tested; and the current scope is live on a devnet.

Targeting a cadence of around six months per hard fork could help to prevent forks becoming too large. Large feature forks could be alternated with smaller forks focused on UX and quality of life improvements, giving more time to test complex new features; it is also crucial to consider cross-fork dependencies and plan accordingly.

Parallelising Implementation

Having a clear plan can help to parallelise work on future forks. Once a fork is shipped, we could already have the next in devnet phase rather than starting to discuss the scope at that point.

Specs and Testing

EIP champions should have greater responsibility in ensuring their specifications are rigorous in order to avoid differences in interpretation of the spec leading to delays. They should also coordinate with the testing team to create spec tests. Formalising and testing earlier in the process would save time overall.

Adopting EIP versioning (EIP-7577) would help to coordinate implementing spec changes.

Furthermore, more resources could be allocated to testing, and client teams could be more involved.

ACD Efficiency

Client teams should do more asynchronously before ACD to make the calls more productive.

Specifically, there could be a place where client teams can review EIPs, leave opinions, and feedback. We already have Ethmagicians, but we could create a dashboard which is more structured than a forum thread. This would display which EIPs are provisionally scheduled for each fork and could be annotated by client teams with their opinions and implementation status.

Additionally, there should be greater expectations for ACD agenda items to be reviewed before the call where possible (teams could assign members to review different items), to avoid situations where no one has had a chance to review.

4 Likes

This is a rewording of my comments during ACD last week. The problem isn’t the fork size, it’s that forks are bloated with EIPS that seem “small enough” that including them feels like it will not cause any delay in the fork. These claims are often taken at face value.
Core developers get very busy implementing EIPs that are completely unrelated to the big picture of the roadmap, but it feels like something is being done because yet another EIPS is included.

One can add as many committees and smaller decision groups as one wants, the problem will still be the same if there is no commitment to the overall roadmap, and if frivolous EIPs keep getting scheduled.

I agree with the Nethermind team, that there should be a much higher expectation on the quality of the testing and the specification of each EIP. This is exactly what we have tried to achieve with the verkle trees/stateless effort, and yet this has played against it: people realized how complex it was and preferred (putatively) smaller EIPs that felt like we something would ship fast.

There needs to be a way to commit to long term items on the roadmap as a way to signal that work doesn’t get wasted. In other words, we need commitment from core developers and especially the research team, that it is safe to work on extremely challenging and difficult forks. Without these guarantees:

  1. it’s impossible to plan and work in parallel - only the next fork is thought of while everything else is a “maybe”
  2. the amount of people willing to take the risk of working on extremely ambitious forks will decrease, as the costs to do so are high and the rewards non-existent.

This is not a question of decentralization: large, centralized companies run into the same kind of issues as they grow successful.

There’s no easy way to solve these issues, but the few steps below would be a good start:

  • EIP champions should justify on ACD how an EIP advances the roadmap (or Increase the security of the network, obviously). That means EIP that are currently scheduled for Prague or Osaka.
  • Don’t change the road map because twitter panics. Any change to the roadmap should be a month-long process involving many stakeholders, probably during an interop or a major conference.
  • Improve communication between research and core devs. The latest example is inclusion lists (and quantum resistance a close second). While these topics do need to be researched, they have been pushed onto the core development agenda way too early, caused a lot of confusion and had a negative impact on the timelines as teams worked on building them before realizing the idea wasn’t ready for its prime.
  • Commit to forks long-term. This doesn’t mean that any change should be slated for a specific fork ahead of time. This means there needs to be tracks, and if the output of any of these tracks is ready (i.e. devnets running and execution specs written) then they should be shipped in the next fork. We can have 4 forks a year with this (or 0, which means teams are working on adding value to the protocol, and that’s fine).
3 Likes

Two arguments being mixed together here. If an EIP isn’t ready, we should be comfortable moving it to the release after, in order to hit a regular delivery cadence. Confidence in the timing of the next release takes pressure off this decision.

100%. One of the best ways to do this is to plan n+2 ahead. We can move a lot faster on fork n+1 with this in mind also to prevent a lot of rebasing/merge conflict with more fork scope certainty.

However dropping clients (and whole sets of validators who run them) is a far more serious consideration and definitely not something that should become normal.

Definitely wouldn’t be realistic, we need to do multi-layer hard forks together and not leave anyone behind ideally. However, we can also look at where we can direct some CL/EL-only upgrades where entire features aren’t blocked due to X feature that’s not ready for the n+1 hard fork.

The Lodestar team posted a retro on our blog with some of these suggestions/ideas here: Lodestar’s Pectra Retrospective and Future Fork Ideas

If you require some of that within this thread for the AI generator or persistence purposes @timbeiko , let me know and I’ll summarize a version to post in this thread.

3 Likes

The Teku team has shared a document reflecting on some of the points raised by Tim and also some observations from the team. The full retrospective document can be found here. Please read below a summary of our discussion:

  • It is impressive to see how much we have evolved in our testing capabilities, largely due to the support from EthPandaOps and the tooling they have developed. We cannot stop praising them for all their work and how much more productive we are because of their help.
  • It is really good to see many teams working together and collaborating towards making the protocol better. Interop events and Eth R&D workshops have been super valuable.
  • Scope creep has impacted our productivity and workflow and made us take longer to deliver value. We should try to have a better understanding of the features going into a fork earlier in the cycle, and converging into a stable scope sooner.
  • Ideally, every fork should have a clear goal (associated with the Ethereum roadmap). This can help core-devs to understand where to focus and also help us during the process of scoping and prioritizing features.
  • We need to rethink the way we make decisions about EIP inclusion in a fork. Ideally, we want more mature EIPs being scheduled, after teams have had time to understand their value, impact and implementation details.
  • The separation between ACDE and ACDC seems artificial and creates more harm than good. We believe that we should move towards having a unified call, to talk about scope, priorities and governance related to the protocol. Separate “technical” calls can be held between Execution teams and Consensus teams to discuss anything relevant just to one of the layers.

Thanks!

7 Likes

Reth team has gathered opinions on Pectra scoping and development internally, here is the aggregated summary our thoughts:

EIP Consideration

Currently, the EIP is considered ready when it has been summarised, the spec has been defined and sufficient number of community members have discussed it. Then the champion of the EIP brings it on a wider ACD discussion forum and it can be considered for inclusion. The crucial step that we think is missing from this process are test vectors. This would make implementation easier for every client as well as put the authors into the mindset of encoding and possibly discovering the edge cases that their change presents.

Scoping

Pectra scoping has effectively started after Dencun has landed on mainnet and lasted essentially for almost 9-10 months (until the last EIP inclusion). We are strongly opinionated that ACD collectively should:

  • normalize considering EIPs for inclusion for the fork after next (later referred to as fork n + 1)
  • freeze “major” EIPs for the fork n + 1 , by the time fork n lands on mainnet
  • consider committing to a fixed fork schedule (e.g. 2 forks a year)

Testing

We should define a clear set of requirements for clients to be included in devnet on launch. These must include passing all necessary engine Hive test and could include having run an interop devnet locally with 2 or more other clients.

Devnet Parallelization

At this moment, we think that devnet parallelization (simultaneously running devnets for fork n, n + 1, n + ...) is a complex change to the current ACD process and requires more discussions. While we are not against the change, we do think that it introduces a set of challenges/questions including but not limited to:

  • fork activation: should we run fork n + 1 devnet with changes from fork n or should it be an isolated devnet?
    • the former approach puts additional strain on developers to troubleshoot potential bugs from fork n on fork n + 1 devnet
    • the latter approach requires downstream rebase of the changes which is time consuming and might introduce additional bugs
  • what to do with interdependent EIPs across forks?

Summary

We think that first and foremost we should define and change the fork scoping process and are open-minded to future changes with regard to devnet testing.

7 Likes