EIP-7495: SSZ StableContainer

EIPs
21
22

Overall this is a nice addition to SSZ and accomplishes its objectives.

As far as formatting of the EIP, it looks like care was given to ensure that large sections can be copy/pasted directly into the ssz document in the consensus specs. Good job there.

I do wish that the Variant type was mentioned earlier in the document, given its importance. And that it had a format that was similarly able to be copied to the ssz document.

Also feels like some sort of motivation for Variant is missing. eg:

While StableContainer provides a flexible structure for forward and backward compatibility of ssz containers, in practice, having known versions of containers, with known subsets of fields are useful, eg: a datastructure where a field is available after a fork boundary but not before. To provide this type safety as well as more efficient serialization, Variant[S] is defined.

1 Like
23

Have added a Variant mention in the abstract as well: Update EIP-7495: Add `Variant[S]` to abstract by etan-status · Pull Request #8511 · ethereum/EIPs · GitHub

The Variant used to share serialization as well, because for EIP-6493 it is never serialized. Only EIP-7688 prompted the more compact serialization. Agree that with its increased relevance, it deserves an earlier mention.

1 Like
24
25

Updated to rename Variant to MerkleizeAs to better describe what it does. Furthermore, improved explanations and rationale for more clarity, and extended on tests.

Thanks @wemeetagain for the crazy amount of help. Feel free to also link the TypeScript implementation here :slight_smile:

26

Typescript implementation here:

1 Like
27

Looks like Update EIP-7495: Rename `Variant` -> `MerkleizeAs` and improve rationale by etan-status · Pull Request #8558 · ethereum/EIPs · GitHub got closed after I commented, pasting here for discussion:

IMO we should consider removing MerkleizeAs[“a container”] if the only benefit here is to support field reordering.

Also I’m just if reordering of fields is helpful in general, or if its just asking too much from ssz. Is there a case where reordering of fields during serialization is required, and not just a nice-to-have?

Removing the bitvector for serialization by creating a “subtype” is helpful and has tangible benefit. It follows logically from motivation to StableContainer definition. It makes MerkleizeAs serialization more efficient and on-par with Containers.

But reordering of fields feels more handwavey and wrt spec, looks more like a feature ‘because we can’, something that ‘falls from the implementation’. My recommendation would be keeping this EIP more restricted (removing all field reordering). And adding an additional EIP that extends functionality of MerkleizeAs, or defines a new thing with that functionality – where the functionality is matched by the motivation.

28

I love the idea for StableContainer!!! I have to admit thought naming convention for MerkleizeAs feels… less than ideal. How do you feel about StableContainer for the abstract over-arching container and InvariantContainer for the specific, unchanging, version of instead of MerkleizeAs. Code could look like InvariantContainer<BeaconState, Electra> or similar. What do you think @etan-status ?

1 Like
29

Thanks for the feedback; adjusted the name to Profile[B].

Furthermore, the spec was simplified:

  • Reordering of fields in Profile[B] no longer allowed
  • StableContainer[N] now requires everything to be Optional[T] to avoid duplicating logic from Profile[B]
  • Profile[B] now require B to be a StableContainer[N], for the cases that used Container the semantics are the same when not linking the types.
30

ssz_generic test vectors for this EIP added here:

1 Like
31

Added checks to remerkleable that ensure compatibility that only valid types can be created (no re-ordering, compatible field types, no extra fields, and so on).

Tests can be found here:

32

The serialization spec defines:

def is_active_field(element):
    return not is_optional(element) or element is not None

In the specification of StableContainer[N] it’s stated that:
" All fields of a StableContainer[N] MUST be of of type Optional[T]. Such fields can either represent a present value of SSZ type T, or indicate absence of a value (indicated by None). The default valueof an Optional[T] is None."

This makes the first part of is_active_field seem ambiguous - what does is_optional mean? If it means “of type Optional[T]”, this is true for all fields by definition, so do we actually want this?

def is_active_field(element):
    return element is not None
1 Like
33

That’s indeed correct! Have integrated the simplification:

1 Like