Meeting Summary:
The team reviewed the status of DevNet environments and discussed plans for upcoming releases, including new image rollouts and testing schedules. Performance and benchmarking concerns were extensively discussed across different Ethereum clients, with particular focus on gas limits, sync times, and client-specific issues that need addressing. The team agreed to share benchmarking results, investigate performance optimizations, and follow a standardized approach for future testing scenarios to ensure compatibility across clients.
Click to expand detailed summary
The team discussed the status of DevNet 4 and plans for DevNet 5, with Barnabas reporting that DevNet 3 remains online and that they are rolling out new images for client teams to merge into their master branches. The team plans to launch DevNet 5 next week Tuesday, aiming for a 2-3 week testing period. Mario provided an update on VPO site tests, noting that a new release of EAT has been tagged and is currently building, with plans to run tests against all clients to verify fixes for the BPO issue that occurred in DevNet 4.
The team discussed gas limit testing updates, particularly regarding the proposed increase to 60 million gas. Ben raised the question of going to 60 million gas, and Justin from the Besu team expressed concerns about performance issues between 45 and 60 million gas. Kamil mentioned ongoing work on improving worst-case scenarios and integrating ESD tests and benchmark tests, which revealed some slow scenarios for Besu. The team agreed to share results and conduct further analysis to address performance issues, with Kamil planning to share data on the Besu team’s Discord channel.
The team discussed benchmarking challenges across different Ethereum clients, with a focus on snap sync failures and gas performance issues. Marius reported that Def Net 2 outline failed to snap sync within 70 hours, highlighting a need for clients to investigate their specific implementations. The group explored the current state of benchmark testing, where Mario explained they are working to unify tests into execution spec tests, though stateful tests in EEST remain challenging. Kamil noted that ecrecover benchmarks showed consistent performance of 60 megagas per second across clients, while Justin emphasized concerns about MOD exp precompile opcodes that are below 20 million gas, suggesting these should be addressed before considering gas limit increases.
The team discussed benchmarking results and performance concerns across different Ethereum clients. Kamil highlighted that Besu’s worst-case scenario of 19 megagases per second needs improvement, with a target to keep worst-case performance under 3-4 seconds. The team agreed to follow the firstst approach for new benchmarking scenarios to ensure compatibility across clients. Barnabas raised a PR regarding block number limitations in the minimal preset, which the team approved to include in full releases. Minhyuk from Sunnyside Labs shared a report on devnet updates. The team also discussed concerns about safe head behavior on Ethereum clients, which Justin and others will investigate further. Finally, Marius noted that Klaytn and Aragon were performing at 30-33 million gas for point evaluation precompile, which the team will review for potential optimization opportunities.
Next Steps:
- Client teams to merge their changes into main branches before next week’s Devnet 5 launch
- Kamil to share Besu benchmark test results with the team on Discord
- Kamil to share specific EC recover tests with the team for implementation in East
- Client teams to review and comment on Barnabas’s PR regarding bloated numbers testing
- Kamil to investigate point evaluation precompile performance issues with Keith and Aragon clients
- Client teams to analyze and address snap sync failures within 70h window
- East team to unify stateful tests in the benchmarking framework
- Client teams to consider using East benchmarking approach for new scenarios and tools
- Client teams to review and optimize performance for 9 opcodes and Modex precompile below 20 million gas
- Client teams to investigate and address performance issues for 60 million gas limit scenarios
Recording Access: