ACD Testing Call #47 - (Quick Notes)
Facilitator: Mario Vega
TL;DR
- Fusaka Devnet 3
- Nimbus and Lodestar are still being debugged; fixes are underway.
- Decision: Wait until both are stable before advancing.
- EIP-7825 Gas Limit
- Proposal to raise to 30M discussed; concern over app impact.
- Decision: Maintain current 16M gas limit on Devnet 3.
- Gas Limit Testing
- Stateful and large contract tests in progress; repo flag awaited.
- ModExp pricing PoC tool being developed; Geth change not expected soon.
- EIP-7951 (secp256r1) Benchmarking
- Besu results show high cost.
- Action: Mario to follow up with Besu team asynchronously.
- INTERVALS_PER_SLOT Replacement (EIP-4476)
- Adds slot timing config without protocol changes.
- Decision: No breaking changes; clients should begin prepping for implementation.
- PR-4477 Proposal
- Dev’s consensus that it’s too late for Fusaka.
- Decision: Do not proceed; no Devnet 4 for this. Plan shadow fork next.
- Sunnyside Devnet / PeerDAS Syncing
- Sync failures in perfect PeerDAS; investigation ongoing.
- Decision: Feature required, clients to report progress.
- Backfill Implementation
- Prysm, Lighthouse, and Lodestar in progress.
- Decision: Next ACD to revisit Devnet 4 for backfill.
Fusaka Devnet 3
Barnabas:
- Devnet 3 is mostly stable with ~90% participation.
- All 5 Blob Parameter Only (BPO) forks are activated and running smoothly.
- Most prior issues have been resolved.
- However, 2 consensus clients — Nimbus and Lodestar — are still facing issues.
- Caplin also involved in ongoing testing.
Lodestar:
- Matthew K is investigating genesis value sync and working on resolving the bug.
Nimbus:
- Update relayed by Agnish:
- Issue with the custody column related to hash functions in indices.
- EF test vectors are passing.
- EAS (Earliest Available Slot) was not moving — now fixed.
- General updates: MEV issues were reported and have been addressed.
- Issue raised on Eth R&D Discord - Nimbus and Geth having difficulty syncing (likely a peering issue).
- DAS Guardian fix expected within the next few days.
Bharath:
- Local transaction propagation was disabled on the RETH node, which caused transactions not to be streamed.
- Fixes of issues shared on the call.
- Also, under discussion with the RETH team to consider removing local transaction propagation entirely.
- Some complications related to spam filtering reported, and the root cause of transaction exclusion has now been identified.
Roman (RETH): Inquired about the RETH flag and its specific issue.
Bharath: Shared technical context also shared on Eth R&D Discord.
Nethermind (Marek):
- Nethermind worked with rbuilder as state provider on integration with state
- The state sync component is ready, but the EVM integration is still in progress.
- Additional testing is requested before full readiness.
- More details: Flashbots Multi-client Support Announcement
Conclusion & Next Steps
Mario:
- Overall, Devnet 3 is progressing well.
Barnabas:
- Aiming to ensure Nimbus and Lodestar are bug-free before moving forward.
Raising the Limit of EIP-7825 to 30 Million
Mario:
- Provided an update from last Thursday’s ACD discussion.
- Discussion link on Ethereum Magicians.
- Raised concerns that raising the gas limit may negatively impact certain applications.
Toni:
- Stated that tradeoffs favor keeping the current spec as implemented in Devnet 3.
Decision:
- Maintain the gas limit at 16 million for now.
Gas Limit Testing Updates
Barnabas:
- No major updates.
Kamil:
- Still working toward full state testing.
- Marcin is running additional tests and benchmarking.
- EEST is running on existing infrastructure for stateful testing.
- A test is live on a contract with a very large state size; waiting on repo to receive the necessary flag.
- Some tweaks are needed on Devnet.
- BloatNet is being prepared; results expected this week.
- Will continue iterating and updating.
Marcin:
- Developing a new tool to evaluate ModExp pricing.
- Differences in gas usage are expected; further investigation underway.
- Currently working on a PoC — actual testing still pending.
Parithosh (in chat):
- Geth team noted that upstreaming changes would take time, if at all. Not a short-term solution.
Mario:
- No Geth representative was present to elaborate further.
Louis:
- Related issue: execution-spec-tests#1976
- Related PR for configuring gas limits: PR #1983
Kamil:
- 60M gas limit update still requires more testing.
- Genesis file handling and stateful test considerations are ongoing.
Besu’s EIP-7951 Benchmarking Results
Justin (Besu): Updates shared in Comment #1648.
Mario:
- Initial results suggest the cost is too high. Will follow up asynchronously with the Besu dev team.
Replace INTERVALS_PER_SLOT with Explicit Slot Component Times
Justin Traglia:
- Introduced the PR to replace INTERVALS_PER_SLOT with explicit slot component times.
- Goal: Raise awareness now and propose merging within the next 3 months to begin implementation.
- This proposal adds additional config values; no substantial changes to protocol behavior.
- Believes there’s no reason to delay merging sooner.
Mario:
- Asked whether this introduces any breaking changes for Fusaka.
Justin T.:
- No breaking changes expected.
- Clients will need to update configurations, but should not impact Fusaka.
Pawan:
- Initial review looks good.
- Will revisit the PR for a deeper look.
PR-4477 Discussion & Decision
- Barnabas shared and requested decison on PR.
Feedback from devs on call & chat:
- Justin T.: Against including this change.
- Alex S.: Suggests leaving it for now.
- Pawan: Opposes the proposal.
- Barnabas: Notes that the proposal has received support from some CL devs (Potuz, Terence, nflaig).
- Toni W.: Feels it’s already too late to consider this change.
- Csaba K.: Also against; if needed, should be done with a future BPO.
- Barnabas: Opposes bundling this change with BPO.
- Mario: Including this would require an entirely new fork.
- Pari: Suggests deferring it to Glamsterdam.
- Raul: Reminds that BPOs were explicitly designed to avoid code changes.
- Agnish: Opposes inclusion; the proposal is too tricky and more data is needed from dynamic DA conditions with supernodes.
- nflaig (Nico): Agrees it’s too late for Fusaka.
Decision & Next step
- This PR will not move forward.
- Devnet 4 will not be launched.
- Focus will remain on stabilizing Nimbus and Lodestar before further progression.
- A shadow fork is being planned for next week. Barnabas encourages client teams to have established branches ready ahead of the shadow fork.
Sunnyside Update
Update on New Report:
- Successfully reached 60 blobs per block; observed impact on usage.
- Hit limits as expected per hardware requirements in EIP.
- Encountered genesis sync failures in perfect PeerDAS network setup.
Devs comments (from call & chat):
Manu:
- Provided explanation for Prysm issues on perfect PeerDAS.
- Actively working on a fix; expects resolution by end of the week.
Agnish:
- Requested clarity on the importance of syncing in perfect PeerDAS mode.
Pari:
- Clarified that Fusaka will not support PeerDAS with this in its initial release.
- This feature can be evaluated later, and if necessary, included in a future BPO.
Agnish agreed with this direction.
Mario:
- Agreed Perfect PeerDAS support is not required for Fusaka.
- May be revisited for future upgrades.
Agnish:
- Shared that reconstruction-based syncing offers more guarantees and would prefer that over relying on dynamic peer discovery.
- May need this feature.
Mario:
- Acknowledged that this feature might become necessary.
- Requested CL teams to confirm timelines for implementation as it may be a potential blocker for Fusaka.
Pawan:
- Currently does not have it implemented but is aware of the mechanism and will assess how long it would take to develop.
Decision:
Mario:
- This is important enough to be tracked closely.
- Requested the rest of the CL team to respond.
- Will raise the topic again on the next ACD call for broader CL input.
Note:
Sunnyside may spin up additional testnets to support further PeerDAS testing.
Backfill
Barnabas:
- Asked if any client implementations for backfill are in progress.
Manu:
- Prysm implementation is in progress.
Other Clients:
- Lighthouse and Lodestar are actively working on it.
Devnet 4 for Backfill?
Barnabas:
- Proposed the idea of a Devnet 4 focused on backfill testing.
- Suggested waiting for confirmation on ACD once client implementations are ready.
Backfill Implementation Challenges
Agnish:
- CPU usage is significant, depending on how quickly backfilling is expected.
Question raised: Should nodes provide slot-level data granularity while backfill is in progress?
Pawan:
- Mentioned they had to change the database structure to make Earliest Available Slot (EAS) accessible.
- Average backfill size observed: ~1 epoch.
- For their implementation, this is not a blocker.
Conclusion:
- Awaiting updates on perfect PeerDAS syncing before proceeding further.
Suggest any changes to notes here.