Post-Quantum (PQ) Interop #43, June 10, 2026

Agenda

  • devnet5
  • leanspec testing infrastructure
  • leanspec cleanup

Meeting Time: Wednesday, June 10, 2026 at 14:00 UTC (60 minutes)

GitHub Issue

Meeting Summary:

This meeting focused on updates and progress regarding DevNet 5 implementation across different client teams. Gajinder reported that Zim successfully merged DevNet 5 PR and conducted interop testing with multiple clients, though some messages were being published late and they were debugging this issue. Shariq from Veeam shared progress on running 24-node DevNet 4 with 24 validators and 6 nodes per subnet, addressing memory issues and working on finalization debugging. Ruslan from Kulim tested DevNet 5 interop with Zim and Islamda but couldn’t reach finalization between nodes. Pablo from Lambda provided updates on fixing RocksDB issues, porting Linux PR 833, and running interop devnets, while Bankat from Jin reported focusing on stabilizing DevNet 5 and addressing reports from recent runs. Thomas presented significant improvements to the Lean spec, including consolidating test infrastructure to reduce duplication, making tests more consumable by client teams, and adding a decorator to mock cryptography where not needed, which reduced CI test time from 30-60 minutes to just 4 minutes. The team discussed technical questions about attestation limits in blocks and performance metrics, with Parthasarathy requesting all teams to implement metrics for better observability and noting that only 12GB RAM would be available per container due to other services running on the same server. Kamil shared findings from testing 64 Lambda nodes, noting high bandwidth consumption on aggregation topics and slow block propagation, suggesting potential adjustments to Gossip Sub parameters and block propagation intervals.

Click to expand detailed summary

The team provided updates on DevNet 5 and DevNet 4 implementations across different clients. Gajinder reported that Zim successfully merged DevNet 5 PR and is now running interop tests with four subnets, though some messages are being published late. Shariq shared that Veeam ran successfully with 5 nodes on one subnet but encountered incompatibilities when testing with other clients, and they are working on a 24-node DevNet 4 setup with memory issues now resolved. Ruslan tested DevNet 5 interop with Zim and Islamda but couldn’t reach finalization between nodes, while Mihir reported on Lantern’s multi-subnet testing and efforts to address gossip decoding errors. Pablo updated on Lambda’s shadow changes, including fixes for RocksDB issues, LINVM updates, and work on execution layer integration with IterX.

The team provided updates on DevNet 5 alignment and maintenance efforts. Bankat reported progress on landing the lean VM bump and stabilizing DevNet 5, while Mercy mentioned they are testing Net 5 changes with plans to publish an image soon. Thomas presented improvements to the lean spec, including consolidating test infrastructure to reduce duplication, making tests more consumable by client teams, and optimizing CI performance by mocking cryptography where not necessary, resulting in test run times reduced from 40 minutes to 4 minutes.

Emil announced that ZK allocation improvements are now available to everyone, eliminating the need for a global allocator setup. Thomas discussed consolidating the DevNet 5 spec by clarifying existing features and identifying potential bugs, while noting no new features have been added recently. Mercy asked about the maximum number of attestation data entries per block, and Thomas confirmed it’s a maximum limit, not a requirement. Parthasarathy reported ongoing testing on DevNet 5 with 32 node Zim and 4 subnet testnet, identifying issues and requesting teams to implement a specific metric to improve aggregation slot performance and observability.

The team discussed stabilizing DevNet 5 and simulating with a larger number of validators as the main priority for the next week or two. Kamil reported testing with 64 Lambda nodes and identified issues with high bandwidth consumption on aggregation topics due to duplicate messages and slow block propagation latency of around two seconds for P95 propagation. Kamil mentioned these findings were discussed with the P2P team and suggested potential tweaks to Gossip Sub parameters or changes to block propagation intervals for future DevNets.

The team discussed potential solutions for improving slot time and propagation in DevNet 5, with Kamil suggesting adjusting intervals based on propagation speeds of attestation and block data. Thomas proposed using a simulator to fine-tune parameters once more stability is achieved in DevNet 5, and to safely increase the number of validators. Mercy inquired about client teams building Type 2 multi-message aggregate outcomes and measuring workload, to which Parthasarathy responded that he would investigate and follow up with specific metrics. Parthasarathy also requested the latest DevNet 5 image for testing interop, which Mercy confirmed would be published after local testing.

Next Steps:

  • Gajinder: Continue debugging late message publishing issues on multi-subnet runs and start multiple subnet runs with other clients on DevNet 5.
  • Shariq: Test interop against other clients on DevNet 5 after LeanVM compatibility fixes, and continue debugging finalization burst issues on the 24-node DevNet 4 setup before adding other clients.
  • Ruslan: Investigate and resolve the issue where Kulim cannot reach finalization with itself, and the issue where Islamda and Zim are not finalizing on DevNet 5 in a two-node minimal scenario.
  • Mercy: Publish a DevNet 5 image for Lighthouse as soon as local testing is complete.
  • Mercy: Ping Parthasarathy with the specific metric details regarding Type 2 multi-message aggregate block sign time workload measurement.
  • Bankat: Continue stabilizing Jin for DevNet 5, work through reported issues from the DevNet 5 TG channel and logs, and land the scoped fix before resuming runs alongside other DevNet 5 ready clients.
  • Parthasarathy: Look into the block sign time metric for multi-message aggregates and respond to Mercy after reviewing it.
  • Parthasarathy: Implement the aggregation slot timing metric in Zim and request all client teams to implement the same metric to help isolate aggregation publishing delays.
  • All client teams: Ensure container memory limit is set to 12GB when testing locally to match the long-running testnet environment constraints.
  • All client teams: Implement the aggregation slot timing metric as requested by Parthasarathy to improve observability in DevNet 5 runs.
  • Kamil: Continue testing with 64+ nodes, gather more numbers on bandwidth and block propagation latency, and explore GossipSub parameter tuning to reduce duplicate messages and improve propagation performance.
  • Thomas: Revisit slot timing and interval parameters (e.g., attestation vs. block propagation intervals, slot time) using the simulator once DevNet 5 is more stable, to fine-tune configuration for larger validator counts.

Recording Access:

YouTube recording available: https://youtu.be/fGbUwP7jQ04