EIP-4804: Web3 URL to EVM Call Message Translation

[MidnightLightning]

Regarding the “resolve modes” for this standard, it currently looks to the resolveMode function to determine what the contract is capable of. Has there been any consideration done to include ERC165 as part of how a gateway determines what the contract is capable of? For widely-used standards like ERC20, ERC721, and ERC1155, if a contract presents an ERC165 interface that advertises it supports one of those standards, that could help make more robust “default” responses?

For example, if a request is made for a contract with no function call in the path, and the gateway tries calling a resolveMode function and it returns “auto”, it could then try calling a supportsInterface function on that contract, and if it returns true for ERC20, then the gateway could return an HTML “card” representation of that contract’s name, symbol, and total supply (the known static metadata options for ERC20 contracts).

This could also affect not just the default return for a contract, but for contracts like ERC721, could make a call like web3://collectionaddress/100 return an HTML representation of token ID 100 from that NFT collection (showing image, properties, current owner, etc.)

Would there be any interest in having “auto” mode be “smarter” by using ERC165 interfaces?

[qizhou]

Many thanks for the suggestions! I agree that supporting some widely-adopted standard such as ERC-20, ERC-721 will increase the UX. One concern that from my head is that this creates further dependency of ERC-4804 with other ERCs. Further, this breaks a goal of ERC-4804, which translates an web3:// to a single EVM message (rather than multiple messages and display them in an organized way).

Note that web3:// is highly composable - we can create a web3://, which returns an HTML and further display the information of ERC-20/721. What do you think?

[qizhou]

A lot of great thoughts on improving ERC-4804. A couple of my thoughts:

@MidnightLightning

As an artist and developer, I’m keenly interested in the image/SVG handling. Sounds like handling of URLs ending in .svg is in progress but not quite implemented yet? From my experience exploring existing smart contracts that do on-chain image-generation, for string or bytes returns, there’s a few ways they format the results:

• Raw String (e.g. MoonCatRescue: web3://0x91cf36c92feb5c11d3f5fe3e8b9e212f7472ec14/accessorizedImageOf/1289 ): a string that starts with “<svg”
• Data URL-style: “data:[<mediatype>][;base64],” prefix in front of the string (e.g. SVG string starts with “data:image/svg+xml,<svg”)
• Token URI embed (e.g. Loot: web3://0xff9c1b15b16263c61d017ee9f65c50e4ae0113d7/tokenURI/100): JSON blob (possibly with application/json data URL prefix, possibly base64-encoded) that contains an image property that is an SVG string (possibly with image/svg+xml data URL prefix, possibly base64-encoded).

The “Raw String” style would need a suffix on it in the Web3 URL, or would have to use some form of “magic bytes” processing to guess at the file format (which would likely have varying levels of success being automatic). Data-URL-formatted strings could be automatically unwound by the gateway server and have a good level of confidence in what format the output is supposed to be (for standard mediatypes). This would help many different output types, including text formats (e.g. Autoglyphs web3://0xd4e4078ca3495de5b1d4db434bebc5a986197782/draw/100)

Great! We also found that using suffix of the last argument is not enough to render contract data in “auto” mode. This is a very good summary. One idea that we are discussing is to add a query string to specify the MIME type. E.g., a way may be

.../100?mime=image/svg+xml (Note that / between images and svg should be escaped) or
.../100?mime.types=svg

For data URL, unfortunately, I cannot find an MIME for data URL itself. Do you have any suggestions on that?

@MidnightLightning

Or, to be more specific, the Web3 URL format could create a specific way to indicate a specific property of a JSON result is desired? Some possible URL formats for “sending argument ‘100’ results in a string output that is JSON-encoded. Grab the ‘image’ property of that JSON object and return it, formatted as an SVG”:

• .../100.json/image.svg
• .../100.json#image.svg
• .../100.json?result=(image:image/svg+xml)

That first format treats JSON files as if they were folders in a file hierarchy, which seems to fit with the other goals of this initiative?

I personally won’t complicate the standard with further parsing features in JSON or any other format (e.g., HTML). The reason is that we could use other tools to decode the data in a composable way like

curl web3://contractAddr/method?returns=(string,string,string) | jq '.[1]'

so that we could use the composability of the tools and keep ERC-4804 as simple as possible. What do you think?

@MidnightLightning

One other thing to consider is using the Accept HTTP header as a way to deduce the request’s intent, and also as a way to determine desired failback methods. A request that comes with a header of

Accept: image/svg+xml, application/json;q=0.9, text/html;q=0.8, */*;q=0.5

is then very clear that it desires SVG data primarily, but if the gateway cannot serve that, JSON is second-choice, followed by HTML, and it failing all that, it would accept any other format the gateway can serve. That behavior is less to do with the URL itself, so not sure if that should be part of this standard, or a different standard on gateway behavior? The “Accept HTTP Header” is usually described as part of how the http:// schema is defined, so in that case it would fit that it should be called out in this standard that defines the web3:// schema how it handles request headers, and not just the URL itself?

This is a great suggestion! We are currently thinking about adding an extension to ERC-4804 to support customized headers (See EIP-6944 and EIP-5219). Definitively we can take this into account to make the standard more powerful!

[qizhou]

A proposal to enhance MIME type of Web3 URLs in auto model is here EIP-7087: MIME Type For Web3 URL in Auto Mode

[qizhou]

A Chrome-extension to support web3:// is developed by an anonymous community developer here GitHub - ComfyGummy/chrome-web3: Chrome extension for web3:// (ERC-4804) URLs

I have tested it and it runs very well even in very early stage. The work is super fantastic with very deep understanding of Chrome browser and http! It also provides some gif to ease the use of the extension!

[qizhou]

Also, we have a talk about web3:// URL (ERC-4804) at EthCC. Please come if you are interested in it!

[MidnightLightning]

@qizhou

For data URL, unfortunately, I cannot find an MIME for data URL itself. Do you have any suggestions on that?

“Data URL” doesn’t have a mime-type itself, since it’s a way of specifying mime types for a payload (similarly, “this file has a three-letter extension separated from the filename by a period to specify its type” doesn’t have a mime-type itself).

I would suggest that “Data URL” style payloads be parsed by the gateway as a good way to guess the mime-type. Therefore for queries like .../100 (that don’t specify a mime-type), the gateway has a pretty confident guess at what the mime-type is (same as how double-clicking on a file with a “.PNG” extension, the operating system has a pretty good guess it’s an image file and has a default action to take in that case). With that idea, .../100?mime=image/svg+xml still returns SVG (same as explicitly right-clicking a “.PNG” file and choosing “open with > picture viewer”; it doesn’t hurt anything, it’s just more verbose call than .../100.

@qizhou

I personally won’t complicate the standard with further parsing features in JSON or any other format (e.g., HTML). The reason is that we could use other tools to decode the data in a composable way

Sure, that seems good, to have a few interlocking standards like that.

[qizhou]

Thanks for the comment. The explicit MIME and dataurl support is proposed here EIP-7087: MIME Type For Web3 URL in Auto Mode. Please take a look

[nand]

Hi!

Adding the info here as well on the ERC-4804 “umbrella” :

I have implemented ERC-6944 (adding ERC-5219 as a mode to ERC-4804 web3://) on :

• The parsing and executing package GitHub - nand2/web3protocol: Parse and execute ERC-4804 web3:// URLs
• The proof of concept EVM Browser GitHub - nand2/evm-browser: Web browser with support of the EIP-4804 web3:// protocol, which can show on-chain websites hosted on Ethereum and all others EVM chains.

I have published a demo ERC-6944 website at web3://web3-mode-5219.terraformnavigator.eth/ (Terraform NFT explorer) whose source code is at https://etherscan.io/address/0x2b51a751d3c7d3554e28dc72c3b032e5f56aa656#code

[nand]

Hi @qizhou

Question: I have a smartcontract method in which I return a large complex structured data (struct of arrays of struct of strings, ints, …). I want to access that via a web3:// call in auto mode that will return that to me as JSON, but I think I am stuck :
1/ Using returns=(…) does not allow structs && arrays
2/ Generating myself the JSON string in the smartcontract is consuming waay too much gas (and is conceptually wasteful).
Do you see a way to do that, or shall we discuss an extension to the syntax for returns=(…) that handle structs && arrays, as another extension?

I propose this extension spec (while remaining backward compatible) :

• ?returns=(uint256[]) for an array of uints e.g. [[1, 3]]
• ?returns=(uint256[],string[]) for a return of 2 args: an array of uints and an array of string e.g. [[1, 3], ["blah", "bloh"]]
• ?returns=((uint256[],int)) for a return of a single tuple/struct containing an array of uints and an int e.g. [[[1, 3], 5]]
• ?returns=(string,(uint256[],int)) for a return of 2 args: a string, and a tuple/struct containing an array of uints and an int e.g. ["blah", [[1, 3], 5]]
• ?returns=((uint256[],(int,address)) for a return of a single tuple/struct containing an array of uints and a tuple/struct of an int and a address e.g. [[[1, 3], [3, "0xabcd"]]]
• ?returns=((uint256[],string)[]) for a return of an array of tuple/struct containing an array of uints and an int e.g. [[[1, 3], "bb"], [[2, 5], "aa"]]

This above returns everything into JSON arrays. I would like to also propose named vars, so that it returns objects. It would be a varname: prefix to field defs, such as:

• ?returns=(myattr:(mylist:uint256[],blah:string),bloh:string) which would returns {myattr: {mylist: [2, 4], blah: "boo"}, bloh: "baa"}

If this looks good to you, we could add in on ERC-7087 and broaden its scope?

[MidnightLightning]

I believe the current “Example 2” in the EIP is incorrect. It currently reads:

Example 2

web3://cyberbrokers-meta.eth/renderBroker/9999

The protocol will find the address of cyberbrokers-meta.eth from ENS on chainid 1 (Mainnet), and then call the address with “To” = “0x…” and “Calldata” = “0x” + keccak("view(uint256)")[0:4] + abi.encode(uint256(9999)).

The example URI given is /renderBroker/9999, but the paragraph after it translates that to calling a view(uint256) function rather than the renderBroker(uint256). I think that paragraph should be updated to be:

The protocol will find the address of cyberbrokers-meta.eth from ENS on chainid 1 (Mainnet). This address exists, and has a contentcontract text value set to 0x8bb9a8baeec177ae55ac410c429cbbbbb9198cac. It will therefore use that address as the “To” value. The contract at 0x8bb9... does not have a resolveMode() function, so the function to call is renderBroker(uint256): “Calldata” = “0x” + keccak("renderBroker(uint256)")[0:4] + abi.encode(uint256(9999)).

[MidnightLightning]

Example 1

web3://w3url.eth/

The protocol will find the address of w3url.eth from ENS in chainid 1 (Mainnet), and then the protocol will call the address with “From” = “0x…” and “Calldata” = “0x2F”.

This should instead be:

The protocol will look up the metadata of w3url.eth from ENS in chainid 1 (Mainnet). This domain exists, and has a contentcontract text value set to w3q-g:0xEbcA4860ebBe969E9Bc42643fcb437879dBDa9C6. The prefix on that value indicates it’s on the W3Q Galileo blockchain (Blockchain ID 3334). So the protocol will call then address with “To” = “0xEbcA…” and chainid = 3334.

For “Calldata”, the code of contract w3q-g:0xEbcA... is not published on the Galileo explorer, so unclear if it has a resolveMode function or not. If it does exist, I believe the “Calldata” = “0x2F” is accurate, but if it does not have that function, it should be “Calldata” = “0x” (no calldata; same as Example 4)

[MidnightLightning]

Example 3

web3://vitalikblog.eth:5/

The protocol will find the address of vitalikblog.eth from ENS on chainid 5 (Goerli), and then call the address with “From” = “0x…” and “Calldata” = “0x2F” with chainid = 5.

This example does not appear to work. Looking up the ENS record for “vitalikblog.eth” on Goerli yields an address of 0xc4fbA3740f95d25B2196C9437fDb005359296D36, which is not a smart contract address at all. So that should be “Calldata” = “0x”, but wouldn’t return any value?

That ENS record has a “web3” text value set to arb-nova:0xe4ba0e245436b737468c206ab5c8f4950597ab7f. EIP6821 does not mention that text value, but the name seems to fit here? That then makes the result the same as Example 4 (calling that address on Arbitrum Nova).

[qizhou]

Hi @nand ,

Returning an array should be already supported in the current ABI standard, e.g., for the following code

pragma solidity >=0.8.2 <0.9.0;

contract Test {
    struct Data {
        uint256 value;
        string str;
    }

    function getValue() public pure returns (uint256[] memory) {
        uint256[] memory m = new uint256[](3);
        m[0] = 1456;
        m[1] = 1235;
        m[2] = 8673;
        return m;
    }

    function getData() public pure returns (Data memory) {
        Data memory dt;
        dt.value = 5823;
        dt.str = "abeff";
        return dt;
    }
}

You should be able to call getValue() and get the array of uint256 via returns=(uint256[]). Please see the following link in the gateway as an example: https://0x8a7297d333b0243c766029ac31a712eef6451846.3334.w3link.io/getValue?returns=(uint256[])

For structure, I am checking how ABI encodes/decodes a structure, but please let me know if you have any findings.

For the feature of named vars, my concern is that it has not been adopted in existing ABI encoding/decoding practice and we need to cover all concern cases of the format - for example, what if one the name is missing such as ?returns=(myattr:(mylist:uint256[],string),bloh:string) where blah is missing?

[qizhou]

Hi @MidnightLightning

Many thanks for pointing out the typos. We do find a couple of typos/issues with the existing EIP-4804 standard, and an amendment can be found here Update EIP-4804: Add more details for the examples for EIP-4804 by qizhou · Pull Request #6860 · ethereum/EIPs · GitHub (Feel free to comment on it!) Unfortunately, there is no way to amend a finalized EIP according to the current EIP process, so we may have to create another one (likely reuse EIP-6860) to override EIP-4804.

[nand]

@qizhou

You should be able to call getValue() and get the array of uint256 via returns=(uint256[]). Please see the following link in the gateway as an example: https://0x8a7297d333b0243c766029ac31a712eef6451846.3334.w3link.io/getValue?returns=(uint256[])

Ah ok I haven’t realized arrays were supported. Will modify the web3protocol lib to properly support it.

@qizhou

For the feature of named vars, my concern is that it has not been adopted in existing ABI encoding/decoding practice and we need to cover all concern cases of the format - for example, what if one the name is missing such as ?returns=(myattr:(mylist:uint256[],string),bloh:string) where blah is missing?

They are indeed not part of the ABI encoding itself, but as the spec says : Despite the fact that names are intentionally not part of the ABI encoding, they do make a lot of sense to be included in the JSON to enable displaying it to the end user.

https://docs.soliditylang.org/en/latest/abi-spec.html#handling-tuple-types

Indeed, when working with web3:// calls that returns complex structures, it would be too bad to manually have to map the returning arrays into structs.

what if one the name is missing : I would say, if one or more of the name are missing, we fallback to returning arrays only with no names.

Allowed characters for the names : Since any valid string is a valid key in JSON, I suggest we do the same, and we would require quotes if the name contains : , ( ) e.g. returns=("a:1":uint) would return {"a:1": 1234}. If quotes are used and we want to include a quote inside the key, the quote should be escaped as in regular json, e.g. returns=("\"a\":1":uint) would return {"\"a\":1": 1234}

What do you think?

[nand]

Hi!

In accordance with @qizhou, ERC-6860 was written to formalize, add details, and do small fixes and tiny changes to ERC-4804.

You can view it here :

ERC-6860: Web3 URL to EVM Call Message Translation

Please let us know if you have comments!

2 implementations of ERC-6860 were made :
Golang: GitHub - web3-protocol/web3protocol-go: Parse and execute ERC-4804 web3:// URLs
Javascript: GitHub - web3-protocol/web3protocol-js: Parse and execute ERC-4804 web3:// URLs
Both are using implementation-independent tests in toml files located at : GitHub - web3-protocol/web3protocol-tests: Parse and execute ERC-4804 web3:// URLs

EVM Browser, using the js implementation, can be used to view web3:// URLS : GitHub - nand2/evm-browser: Web browser with support of the EIP-6860 web3:// protocol, which can show on-chain websites hosted on Ethereum and all others EVM chains.

Next is coming : Curl like apps to easily test and experiment with web3:// URLS : GitHub - web3-protocol/web3curl-js: Curl-like app for ERC-4804 web3:// URLs

And next is some serious documentation too.

[nand]

The web3:// curl app is now there, and with useful verbose modes (level 1, 2, 3)!

Useful to understand how a web3:// query is processed.

Example (level 1 verbose) :

web3curl -v "web3://vitalikblog.eth/css/misc.css"

will output:

* Fetching URL web3://vitalikblog.eth/css/misc.css
* Parsing URL ...
* Host domain name resolver: ens
*   Domain name being resolved: vitalikblog.eth
*   Resolution chain id: 1
*   Resolution type: contentContractTxt
*   contentcontract TXT record: arb-nova:0xe4ba0e245436b737468c206ab5c8f4950597ab7f
*   Result address: 0xe4ba0e245436b737468c206ab5c8f4950597ab7f
*   Result chain id: 42170
* Contract address: 0xe4ba0e245436b737468c206ab5c8f4950597ab7f
* Contract chain id: 42170
* Resolve mode determination... 
> 0xdd473fae
< 0x6d616e75616c0000000000000000000000000000000000000000000000000000
* Resolve mode: manual
* Contract call mode: calldata
* Calldata: 0x2f6373732f6d6973632e637373
* Contract return processing: decodeABIEncodedBytes
* Contract return processing: decodeABIEncodedBytes: MIME type: text/css
*
* Calling contract ...
* Contract address: 0xe4ba0e245436b737468c206ab5c8f4950597ab7f
> 0x2f6373732f6d6973632e637373
* RPC provider used: https://nova.arbitrum.io/rpc
< 0x0000000000000000000000000000000000000000000000000000000000000020...0000000000000000000000000000000000000000000000000000000000000126
*
* Decoding contract return ...
* HTTP Status code: 200
* HTTP Headers: 
*   Content-Type: text/css

[returned data (CSS here)]

[MidnightLightning]

In the “Specification” section, the technical definition of web3Schema gives three options: ethereum-web3:, eth-web3:, and web3:

However, immediately following it (in the “human-friendly” description of those terms), web3Schema is defined with two options: web3: and w3:

These two descriptions contradict each other. web3: is present in both, which makes it seem like the primary/default option (and indeed all the examples use it), but what of the other three options? The other three are only present in one of the definitions or the other, but not both. Are all four valid?

IANA registrations have been approved for web3: and w3:, but not the other two. So is the first part of the Specification wrong, and ethereum-web3: and eth-web3: are not valid schemas to use for this standard?

[qizhou]

ethereum-web3: and eth-web3: tries to follow ERC-831: URI Format for Ethereum, but ERC-831 is Stagnant. So in our latest updated standard ERC-6860, both ethereum-web3: and eth-web3: are removed in the schema (and unfortunately since ERC-4804 is finalized, so we have to replace it rather than modifying it).