Meeting Summary:
ACDT 95 was a meeting focused on reviewing DevNet performance and discussing Ethereum Layer (EL) topics. The team reviewed DevNet 8, where Joachim’s testing caused significant issues with Besu nodes missing 65% of proposals due to memory problems with 64KB contract deployments and jump tests. DevNet 9 was discussed briefly, highlighting challenges with node readiness, fork transitions, and caching inefficiencies that led to the decision to not continue running it. The group then discussed plans for DevNet 11, which will focus on coordinating with Layer 2s for fork transitions and testing watchtower updates. A major discussion centered on whether to include EIP 8253 regarding block access list compatibility, with most clients preferring to defer this to the Hecata fork rather than include it in the current Glamsterdam hard fork due to timing constraints and testing requirements. The meeting also covered benchmarking updates and real testing requirements, with Carlos presenting findings on Yochimnet versus state actor performance and gathering feedback on using Hive for real testing scenarios.
Click to expand detailed summary
The meeting began with technical discussions about breakout room setup, where danceratopz experienced login issues and Justin explained that CL breakout rooms would be automatically created while EL breakout sessions would continue in the same meeting. Pooja initially had trouble locating the Zoom link for the CL meeting, but Justin provided the link through a GitHub comment. The meeting was scheduled to start with a DevNet round followed by breakouts, with danceratopz noting that the agenda would be shared in chat.
The team discussed issues with DevNet 8, where Joachim’s testing deployed 64KB contracts with jump tests, causing most Besu nodes to fail and significantly impacting proposal processing. The team identified that Besu nodes were missing around 65% of proposals compared to other clients like Nethermind and Geth which had minimal issues. Ameziane and others are investigating memory usage problems, particularly related to debug trace calls, with some nodes being disabled to compare performance. The team also reviewed DevNet 9, which was intended as a non-finality test but experienced significant challenges including 30% network downtime during the gloss fork due to miscommunication about readiness and trunk branch deployment issues, along with Lighthouse nodes experiencing OOM errors due to caching inefficiencies and the large validator size of 4 million keys.
Stefan reported that DevNet9 experienced issues with nodes going offline after restarts and builder nodes struggling with subnetwork subscriptions, though the team decided not to keep it running due to client readiness and costs. The team discussed moving forward with DevNet11, which will focus on coordinating with Layer 2s for fork transition testing and ensuring clients are on trunk branches. The discussion included updates about a Prysm issue that needs to be merged into trunk, though it’s not yet resolved, and there were technical details about client checklist items and network configuration for DevNet11.
The team discussed plans for transitioning to a new network, with the main dependency being waiting for clients to have their unstable branches ready. Stefan suggested allowing at least 24 hours between the Genesis and Gloss Fork transitions, with coordination planned for L2s to ensure sufficient time for preparation. danceratopz indicated they would launch DevNet 11 towards the end of the week once the PRISM issue is merged, with an additional 1-2 days needed for the Gloss activation. The conversation ended with participants moving to breakout sessions, with the CL breakout being specifically mentioned.
The team discussed EIP-8253, which addresses an issue with block access lists and zero-nonce storage accounts. Most client teams, including Nethermind and Ifreex, expressed concerns about implementing the change so late in the process and preferred to defer it to the Hecata fork instead of including it in the upcoming Glamsterdam hard fork. The group agreed to make a final decision on Thursday after further client coordination. The team also briefly discussed reorg testing requirements, with consensus to develop this testing suite in Hive rather than creating a new tool. Finally, they noted that EIP-8037 state gas in debug traces would be discussed in the upcoming RPC standards call.
Next Steps:
- Stefan: Disable debug/trace for Besu nodes and leave it enabled on others to compare and find the root cause of the memory issue on DevNet 8.
- pk910: Set up DevNet 11, coordinating with Layer 2s for a smooth fork transition, waiting for all clients to have their trunk branches ready (especially Prism).
- pk910: Decide on the exact size and validator shares for DevNet 11.
- Jochem-brouwer: Add the spammer attack test case to the Steel Discord channel for Besu to debug.
- pk910: Forward the question about expressing attacks as yeast tests to Rafael for the benchmarking call.
- Client Teams (e.g., Nethermind, others): Coordinate async to decide if EIP 8253 should be included in Glimpse or deferred to Hecota, with a decision to be made on Thursday.
- Carlos (CPerezz): Bring up the results of using Yochimnet vs. state actor for Geth benchmarks in tomorrow’s benchmarking call.
- Carlos (CPerezz): Work on creating general reorg tests for EL clients, with a preference for using Hive as the basis.
- Stefan: Share the PR for clarifying the EVE-772 TX size for type 3 transactions and gather async feedback.
- Stefan: Discuss and finalize the state gas and depot traces proposal in the RPC standards call.
Recording Access: