More frequent, smaller hardforks vs. less frequent, larger ones

hardfork
ethereum-roadmap
#14

It is my current intuition that unless we make serious improvements in the way the EIPs are put it, there is little hope that Ethereum 1x will deliver what it is supposed to.

That’s how I’m seeing it too. Is it the EIP process that needs to be reviewed in addition to HF times? I know it sounds unintuitive but would a Committee on EIPs resolve any of those issues? Such as Security concerns, release times, inclusions or exclusions due to security concerns.

[ WIP ] -> [ DRAFT ] -> [ LAST CALL ] -> [ ACCEPTED ] -> [ FINAL ]
[ WIP ] -> [ DRAFT ] -> [ LAST CALL/REVIEWED ] -> [COMMITTEE  ACCEPTED] -> [PLANNED INCLUSION OF HF SCHEDULED DATE]-> [ FINAL ]

I’ll post it here since it relevant but to me looks like another topic for discussion.

1 Like
#15

Seconded, this is my concern and my fear as well. Combined with Higher standards for EIPs, I think we have the beginnings of what needs to be done to improve the EIP process and increase the pace of innovation.

I think testing is definitely one bottleneck in the existing process and we should discuss how to improve throughput - I think a “layered pipeline” approach as discussed here would make sense. Does the testing team have sufficient resources today? CC @holiman (I don’t see Dimitry here)

Another thought to consider: instead of a waterfall approach, would a leaner approach with multiple “sprints” maybe make sense here? It seems like Afri’s proposal, which we’ve been working with for Istanbul, works this way - we’re pursuing multiple EIPs at the same time, with hard deadlines for EIP submission, implementations, testing, etc. and whatever makes it into the next upgrade goes in, whatever doesn’t must wait until the following one.

3 Likes
#16

I think we get this from having planned HFs in advance, regardless of the size :slight_smile: At this point, we’re mostly debating the length of the sprint :stuck_out_tongue:

3 Likes
#17

Sufficient resources - I’d say no on that. Testing is a bit thankless task, and what complicates the matter is that it both requires very solid knowledge of EVM internals and Ethereum in general, and it’s also much case of not much happening, and then suddenly there is tons of work to be done, as we finalize eips.

So from that respect, I think having a steadier flow of smaller eips would be a definite win. Then it wouldn’t be this case of coordinating testcases for N new features which may or may have internal interaction in odd ways.

2 Likes
#18

Thanks for sharing! I’ve added this to the original post.

#19

One thing that came up in the AllCoreDevs call today is that the this could potentially change before Istanbul. When and how should we get consensus on whether we want to change this?

1 Like
#20

I think this should be an Informational EIP (of which we have very few). We can update 233 to say that dates are picked based on those guidelines. Let’s get it in and finalized @shemnon.

#21

On the all core devs call just now @holiman proposed an “EIP-centric” upgrade process rather than a hard-fork centric model like we have now. @AlexeyAkhunov also expressed concern about a deadline, like the May 17 Istanbul deadline, meaning that folks would have to rush their EIPs in before the deadline lest they “miss the boat” and have to wait 9 or 12 months until the next hard fork. I’m also concerned about people submitting half-baked “placeholder” EIPs before the deadline.

I like Martin’s idea a lot, especially in combination with smaller, more frequent hard forks. @shemnon’s proposal at https://github.com/shemnon/EIPs/blob/4e3069b4f9a30a639b142151dc6295f634712786/EIPS/eip-network_upgrade_windows.md looks reasonable to me.

2 Likes
#22

Advantages to fast enough upgrade schedule:

  • If a feature isn’t ready on time it’s not a huge slip to wait for the next upgrade.
  • Upgrades happen too fast for the mob to get mobilized.
  • Discussion of EIPs gets pushed further back into the process where it belongs.
  • Upgrades may become as uneventful as Apple point releases.

Disadvantages:

  • Developers are testing more than developing.
  • The mob never stops screaming that they weren’t consulted.
  • More work and coordination for miners.
6 Likes
#23

Testing is a thankless task for developers. It’s a job that real testers like to do.

#24

From AllCoreDevs:

Nick Sawinyh @sneg55 09:06

More work and coordination for miners.

It’s not a big deal actually for most of mining pools.

Peter Salanki @salanki 13:00
+1 on Nicks comment. Updating node software for mining pools is very painless. Takes 5 minutes for me

Greg Colvin @gcolvin 14:34
Then we are down to the mob, which I don’t care about, and the lack of testing resources.

2 Likes
#25

+1 Lack of testing resources.

#26

Top developers and engineers know to own their code. They know that the engineering includes testability, including all the code for testing, keeping tests reliable, efficient, maintainable, extensible, etc.
Yes, other eyes and engineers are required for producing robust code, but the culture needs to be more on ownership rather than on other people/testers to find the bugs or security issues.
There are engineers that come from such cultures but usually they are very comfortable in their big company jobs.

I’m not writing this to argue but to clarify your points further. +1 to a lack of engineers on fronts like testing, security, performance…

1 Like
#27

Over the years I’ve seen many patterns on the relationship of development and testing. But when I have worked with professional test engineers I’ve found that they are simply much better at it than I am, as you would expect.

When I have been sole owner and tester of my code I can do fine. I spent three years writing one system that manifested only one bug in the next seven years in production. But I’m more productive when I can concentrate my efforts where I am most skillful, and let test engineers concentrate their efforts where they are most skillful.

This is independent of big vs. small company. The best test engineer I ever worked with was at a tiny startup. But it’s true that no big company I worked at failed to have professional test engineers on staff. Lots of them.

What’s been hardest here is maintaining ownership in an open-source environment that lets anybody on earth make PRs.

2 Likes
#28

So one of the questions at 1x Berlin related to the preferred cadence of 1.x network upgrades. 6 month vs 4 month was the point where consensus broke down. As an advocate of 6 month the big advantage I see for client developer is that it will keep the clients focused to one upgrade at a time. With a 4 or 3 month cadence there will start to be some overlap between the upgrades before they are “done.”

Here’s a chart I put together with some cheesy placeholder names for future upgrades that illustrates the overlap. EIPs is the deadline for EIPs to be considered. Clients is the client development soft deadline. Testnet and Mainnet are respective hard fork dates. These landmarks are presuming we keep the same Istanbul landmarks for future network upgrades.

6 month Cadence

Date Istanbul ‘Asiago’ ‘Brie’ ‘Cheddar’
May 2019 EIPs
Jun 2019
Jul 2019 Clients Kickoff
Aug 2019 Testnet
Sep 2019
Oct 2019 Mainnet
Nov 2019 EIPs
Dec 2019
Jan 2020 Clients Kickoff
Feb 2020 Testnet
Mar 2020
Apr 2020 Mainnet
May 2020 EIPs
Jun 2020
Jul 2020 Clients Kickoff
Aug 2020 Testnet
Sep 2020
Oct 2020 Mainnet
Nov 2020 EIPs
Dec 2020
Jan 2021 Clients
Feb 2021 Testnet
Mar 2021
Apr 2021 Mainnet

4 month Cadence

Date Istanbul ‘Asiago’ ‘Brie’ ‘Cheddar’
May 2019 EIPs Kickoff
Jun 2019
Jul 2019 Clients
Aug 2019 Testnet
Sep 2019 EIPs Kickoff
Oct 2019 Mainnet
Nov 2019 Clients
Dec 2020 Testnet
Jan 2020 EIPs Kickoff
Feb 2020 Mainnet
Mar 2020 Clients
Apr 2020 Testnet
May 2020 EIPs
Jun 2020 Mainnet
Jul 2020 Clients
Aug 2020 Testnet
Sep 2020
Oct 2020 Mainnet
Nov 2020

With a 4 month cadence we need to lock in EIPs prior to a successful launch of the previous upgrade. And the hard fork window is the same time period where devs would need to work on client implementation for the next version. If something goes bad on the pending upgrade the next upgrade will be severely impacted as well.

Because of this mainnet launch and next upgrade overlap that I think anything less than 6 months is a higher risk cadence than we should be looking to take on.

3 Likes
#29

Makes sense. Though I would say that most of this is based on the our “conventional” way of doing things, like:

  1. Assumption that people working on the EIPs are the same people who are developing major clients - this does not need to be the case. Working Groups are the attempt to change that.
  2. EIPs are often under-researched when they hit the roadmap, and there is a large uncertainty of their implementability (effectively EIP process is used as a research process). If EIPs are higher quality and properly researched and prototyped, the risk of them being thrown out and causing knock-on effect are lower
  3. Test preparation is a fairly centralised process

Challenging these assumptions is something that would have benefits regardless of whether the cadence is increased. And therefore, it is something to strive for

3 Likes
#30

@shemnon thank you this is great! I still have to figure out how to pull this into a summary so we can gather everything — thanks for doing “homework”.

@AlexeyAkhunov I think we can move to a more continuous / faster cycle, it will take time for people & processes to adjust rather than time to get work done!

Essentially “kanban” where EIPs are pushed through to ready and then bundled into releases.

1 Like
#31

To formalize what I said on AllCoreDevs on 26 April, for @AlexeyAkhunov’s point 1 if a fork goes poorly, like Constantinople did (with both testnet and mainnet problems) the same devs that would implement the change for the next network upgrade would be the same people fixing the problematic deployments of the previous network upgrade. If things go well you are correct they will not be the same people working them.

If we get to a point where things are going smoothly at 6 months (both Istanbul and Asaigo) we can re-evaluate for 4 months (with Brie and Cheddar or Cheddar and Derby overlapping). But both Byzantium and Constantinople were not what I would consider smooth. Alexey’s points 2 and 3 are the indicators we will need to see to consider a 4 month cycle.

And to be clear, these cheesy names are placeholders until we get more formal ones. But when talking about particular network upgrades in the future we need some mnemonic that implies monotonic order.

1 Like
#32

A couple of things on my mind for Asaigo -

  • Create a new “EIPs Proposed” checkpoint two to three months after kickoff. 6-7 months before mainnet.
  • Change the “EIPs” Proposed checkpoint to “EIP ready.” 4 months after kickoff and 5 months before launch.

This new checkpoint will be when we close the door to new EIPs for the upgrade, rather than 5 months before upgrade it will be 6-7 months. This will then give time to discuss and hash over the proposed EIPs for two to four all-core-devs calls. Anything that gets kicked out of the previous upgrade (such as EIP-1283 was) would automatically be considered as proposed.

EIPs don’t need to be well formed, they just need to be communicated as proposed with some reasonable bounds. Like “My fee market proposal as per my talk at Conference Y” or “State Fees steps W, X, and Z.”

The existing 5 months to go mark the EIPs proposed would need to be “ready” - i.e. someone would have been on ACD to champion the EIP, ACD had discussion on it on the call or in FEM (or some other appropriate forum) and the EIP would be in a condition it could go into last call, accepted, or final. The idea being it is ready for client implementors to implement without concern for it being incomplete.

The 3 month (client implementations soft deadline) and 2 month (Testnet launch) checkpoints would remain the same.

Date Istanbul ‘Asiago’ ‘Brie’ ‘Cheddar’
May 2019 EIPs
Jun 2019
Jul 2019 Clients Kickoff
Aug 2019 Testnet
Sep 2019 Proposals
Oct 2019 Mainnet
Nov 2019 EIP Ready
Dec 2019
Jan 2020 Clients Kickoff
Feb 2020 Testnet
Mar 2020 Proposals
Apr 2020 Mainnet
May 2020 EIP Ready
Jun 2020
Jul 2020 Clients Kickoff
Aug 2020 Testnet
Sep 2020 Proposals
Oct 2020 Mainnet
Nov 2020 EIP Ready
Dec 2020
Jan 2021 Clients
Feb 2021 Testnet
Mar 2021
Apr 2021 Mainnet

Future cheesy placeholder names:
‘Danbo’ - Oct `21
‘Edam’ - Apr '22
‘Fetta’ - Oct '23
‘Gouda’ - Apr '23
‘Hoop’ - Oct '23
‘Infossato’ - Apr '24

This will get us 5 years out, which is what, two major bear markets?

3 Likes
#33

I suggest to use Devcon locations past Istanbul:

  • (devcon 0: berlin?)
  • devcon 1: london
  • devcon 2: shanghai
  • devcon 3: cancun
  • devcon 4: prague
  • devcon 5: osaka

After 2.5 (?) years that would mean we need to have more devcons or switching naming convention :slight_smile:

2 Likes