ERC-5750 Extra Data Parameter in Methods

Hi all, I am proposing an EIP-5750 to denote and designate the last parameters method as extra data.

Here is the pull request: EIP-5750: Method with Extra Data by xinbenlv · Pull Request #5750 · ethereum/EIPs · GitHub
To save you a click away, I add a snapshot here


eip: 5750
title: Extra Data Parameter in Methods
description: This EIP that defines an extra data parameter in methods.
author: Zainan Victor Zhou (@xinbenlv)
discussions-to: ERC-5750 Extra Data Parameter in Methods
status: Review
type: Standards Track
category: ERC
created: 2022-10-04

Abstract

This EIP that defines an extra data parameter in methods, denoted as methodName(... bytes calldata _data). Compliant method of compliant contract
can use the extra data in structural way to introduce extended behaviors.

Motivation

The general purpose of having a standard for extra data in a method is to allow further extensions for a existing method interface. For example, the safeTransferFrom already

  1. At the very least, Methods complying with this EIP, such as transfer and vote can add reasons in extra data, just like how GovernorBravo’s improvement over GovernorAlpha
  2. In addition, existing EIPs that has exported methods compliant with this EIP can be extended for behaviors such as using the extra data for endorsements or salt, nonce, commitments for reveal commit.
  3. Allowing one method to carry arbitrary calldata for forwarding a function call to another method.

Specification

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.

  1. Any compliant contract’s compliant method MUST have a bytes(dynamic size) as its LAST parameter of the method.
function methodName(type1 param1, type2, param2 ... bytes calldata data);

Rationale

  1. Having a dynamic sized bytes allow for maximum flexibility for arbitrary additional payload
  2. Having the bytes specified in the last naturally compatible with the calldata layout of solidity.

Backwards Compatibility

Many of the existing EIPs already have compliant method and all compliant contracts of such EIPs are already compliant.

Here are an incomplete list

  1. In the EIP-721 the following methods are already compliant:
  • function safeTransferFrom(address _from, address _to, uint256 _tokenId, bytes data) external payable; is already compliant
  1. In the EIP-1155 the following methods are already compliant
  • function safeTransferFrom(address _from, address _to, uint256 _id, uint256 _value, bytes calldata _data) external;
  • function safeBatchTransferFrom(address _from, address _to, uint256[] calldata _ids, uint256[] calldata _values, bytes calldata _data) external;
  1. In the EIP-777 the following methods are already compliant
  • function burn(uint256 amount, bytes calldata data) external;
  • function send(address to, uint256 amount, bytes calldata data) external;
  1. In the EIP-2535 the following methods are already compliant
function diamondCut(
        FacetCut[] calldata _diamondCut,
        address _init,
        bytes calldata _calldata
    ) external;
  1. In the EIP-1271 the following methods are already compliant:
  function isValidSignature(
    bytes32 _hash,
    bytes memory _signature)
    public
    view
    returns (bytes4 magicValue);

Security Considerations

  1. If using the extra data for extended behavior, such as supplying signature for onchain verification, or supplying commitments in a commit-reveal scheme, the security best practice shall be followed for that particular extended behaviors.
  2. Compliant contract shall also take into consideration the information of extra data will be shared in public and circulate around mempool, so specific caution shall be paid for replay-attack, front-run/back-run/sandwich attacks.

Copyright

Copyright and related rights waived via CC0.

Posting some good questions from @SamWilsn from the EIPs PR

@SamWilsn

Why does this EIP need to exist? I think this is a great recommendation, but we don’t want the EIPs repository to become a catalogue of Solidity best practices. You need to make the case in your Motivation for why this pattern needs its own standard.

@Pandapip1

I see no reason why best practices shouldn’t be standards. I could see the type being changed to informational, though.

Here is author answer

Just to be clear, this EIP is not a “recommendation” or “best practice”. EIP intend to serve as a a protocol/standard specifying format of method and their behavior. Unlike a lot of other ERCs that specifying the name and all parameters, this EIP specifies only the last data and its format, which makes it maximally compatible, both future and backward.

It could be confusing, so let me make a metaphor here:

  • If someone says: “I’d recommend you drive your car on the right side of the road because it has benefit of we bump into each other less” it’s an informational EIP.
  • If someone says, “For all standard-compliant road, let’s designate its right side of the road to be reserved for forward traffic, so we can future agree to which side to build traffic lights, ramps, lightings, fire stations…”, it’s an ERC EIP

This EIP serves for the standardizing purpose: There are a thousand ways to extend methods, but this EIP ask to designate certain place and format for extending behaviors. EIPs like EIP-5453(#5453) and EIP-5732(#5732) can all benefit from this EIP if they can rely on that a parameter in a given method is being designated for extending behavior.

Can you elaborate on this part of the EIP?

Having the bytes specified as the last parameter makes this EIP compatible with the calldata layout of solidity.

@frangio thank you for the question

The Solidity language structure the calldata layout as follows, as documented in 1, 2:

All in all, a call to the function f with parameters a_1, ..., a_n is encoded as

function_selector(f) enc((a_1, ..., a_n))

Thus, if we choose the bytes that will be designated for extension to locate in the last one of all parameters of a function, then we can use that in nesting data, we will be able to assume the same location of data in all methods that conform this rule.

But if we don’t designate the last bytes to be extaData, but alternatively choose the second from last field as a standard, then this is what could happen

One of standard’s implementation choose to put a string field after extraData

function foo(uint8 param1, bytes calldata extraData, string param2);

another of standard’s implementation choose to put a uint8 after extraData

function bar(uint8 param1, bytes calldata extraData, uint8 param2);

Then it’s not very easy nor gas efficient to figure out where extraData locates in such scenario.

But if we choose the last one, both implementations will have to be

function foo(uint8 param1, string param2, bytes calldata extraData);
function bar(uint8 param1, uint8 param2, bytes calldata extraData);

Then we can reliably always know the last parameter (meaning, zero from ending byte), are the extraData field.

One more real world example is for “EIP Endorsement”, we could choose to say: if this last 32 bytes is keccak256(“SOME_MAGIC_WORD”), we will interpret the extending data as endorsement and try to parse it, see EIP-5453 Endorsement (WIP)

It also helps when we try to do nested calls, e.g.

SomeERC721 is ERC5679MintAndBurn {
  
  function burn(from, extraData) {
    // send a call to another function via parsing extraData
    // then 
    _burn(from, []);
  }
}

Hope these descriptions and helps me explain myself more clearer, if not I will create some more deployable contract examples soon and get back with those examples

1 Like

Interesting proposal @xinbenlv

Functionally, is the requirement that any implementing contract must treat the bytes parameter as optional and non-critical?

@devinaconley not necessarily,

When a contract.method has last bytes and use the first N bytes, the remaining of bytes after Nth are always optional and non-critical.

For example, in one of the Commit-Reveal implementation, this particular line

uses first 0-32 bytes for salt, the the remaining bytes after 32th byte could be used for extension behavior.

Given a function foo(uint x, string s, bytes extraData) there is no guarantee that extraData will be the last thing in calldata. More generally, if there are multiple dynamic types there is no guarantee of their ordering in calldata. Do you agree with this? See Use of Dynamic Types in the Solidity docs on ABI encoding.

I am not sure if I follow your word here:

Given a function foo(uint x, string s, bytes extraData) there is no guarantee that extraData will be the last thing in calldata.

Based on solidity’s calldata (in the context of calldata vs storage, not to be confused with other meaning of calldata) the structure of

foo(uint x, string s, bytes extraData);

based on the encoding rule: function_selector(f) enc((a_1, ..., a_n))

will be a concatenation of

  • d1. method selector = keccak256("foo(uint256,string,bytes)")[0:3]
  • d2. encoded uint x which is x
  • offsetOfD3 for location of d3
  • offsetOfD4 for location of d4
  • d3. encoded string s which is padded32(length(s)) || uint256 of each bytes of s's content
  • d4 encoded bytes extraData which is extraData's length per 32bytes || content of extraData padded last one to full 32bytes

The overall becomes d1 || d2 || offsetOfD3 || offsetOfD4 || d3 || d4.

(Updated with frangio’s correction.)

This is not correct. In the simplest case, the encoding will be:

d1 || d2 || X || Y || d3 || d4
                      ^X    ^Y

With X and Y the offsets where d3 and d4 respectively start.

Although this is the canonical encoding that Solidity will produce, the decoding routines support other non-canonical encodings such as:

d1 || d2 || X || Y || d4 || d3
                      ^Y    ^X

As a consequence, it is not possible to predict the location of a dynamic bytes type in calldata.

@frangio
Oh, you are right. Thanks for the correction, yes there are offset X and Y to indicate place of such dynamic types in tail parts.

You made a very good point that I didn’t think of when drafting this EIP, which is with non-canonical encoding, it’s possible to even have reverse location of bytes, or some trick in having overlaps for different data fields (maybe, due to malicious TX or for saving gas cost maybe?). So there is no guarantee of last bytes being physically last one.

This fact you point out did weaken one of the rationales for mandating extraData as last one so the remaining rationale is majorly due to conventions.

For non-canonical encoded contract this could be even a potential security issue if interacting clients/contract assumes so. An action item for me is to think of how to add this as a security considerations. I will send a PR.

I appreciate your feedback and let me know if there are other advice.

Given this, I think the justification for this ERC has lost a lot of weight. I’m not sure that I see the point in standardizing this.

@frangio If there is no value of this standard, this EIP could just get ignored and quietly stay in the corner.

I hold the views that there are still many features that could utilize EIP-5750 and demonstrating having an extension field have a lot of benefits. I am working on a few ideas that actually requires EIP-5750 and will invite/notify you to review when those are ready for reviews.