Yes, though a simpler solution could be to have a variable block size limit. Block builders are capable of adding complexity to their algorithms.
The problem with block constraints is that it results in discrimination, one of the problems this change is trying to address. Gas and gas price (now priority fee) are supposed to be fair ways to evaluate whether a transaction can or should go into the block. Discrimination here means that a transaction might pay a fair priority fee but not be accepted due to outsized calldata. This isnât fair because the transaction is being treated as if it uses more gas than it does, diluting its priority fee. If blocksizeGas is a constrained resource, there should be a market for it, similar to blobgas.
A question that I have here is how much gas is made available for the actual execution of the opcodes.
Currently, that amount is pre_7623 = tx.gas - 21000 - data_cost - create_cost - access_list_cost. If data_floor_cost is the new floor cost of call data (as per this EIP), do we now make post_7623 = tx.gas - 21000 - data_floor_cost - create_cost - access_list_cost available for execution?
If yes, would we be under-funding some transaction executions?
I imagine a scenario where we passed post_7623 gas for opcode execution but during the course of execution, the execution gas used built up enough to cross the threshold. This means that we should really have passed pre_7623 gas for opcode execution. Since post_7623 < pre_7623 we have essentially underfunded the execution in a way. This could be the difference between success vs OutOfGas in some cases.
The computation uses gas_used which is only known after execution. So this EIP can cause out-of-gas after execution would have otherwise succeeded, by retroactively increasing the calldata cost. While the available gas could be computed ahead of time, the spec does not currently do this.
The internal gas of the transaction is computed by taking the higher gas cost, such that 99% of transactions (those donât reach the floor) get more refund.
We must make sure that we donât run out of gas after execution, otherwise clear DoS.
So thereâs a possibility that a transaction that ran out of gas, gets a gas refund. And that it wouldnât have run out, if the refunded gas was available for execution.
Ethereum dapp builder here, I think we shouldnât make calldata more expensive, here is why:
Many L2 app builders such as us have started building on L2s like Optimism when they still sent all L2 data into Ethereumâs calldata
We have protested against it, but Optimism and others have then started to post data into blob storage making the data we post on L2 âdata availableâ but not âretrievable into perpetuity.â
In essence chains like Optimism have pulled the rug under our application as they have gotten rid of a guarantee they priorly passed on to L2 dapp devs.
This is somewhat understandable as theyâre NOT putting user funds at risk, since these can be settled to L1 using Ethereumâs blob storage.
But here is why a calldata cost increase is a problem for non-ERC20 type of applications on Ethereum:
Our application asks users to store delegations from a userâs regular Ethereum wallet to a delegated in-app wallet on Optimism. If youâve ever used Farcaster, you might be aware that any interaction on there is essentially a signed Ethereum EIP-712 message. This is enabled because Farcaster delegates a signer on Optimism allowing that signer to represent your identity on the app. So Farcaster, but us as well, we falsely have relied on the L2 data to be available in perpetuity (and because L1 calldata was just too expensive)
Now, core devs and ETH researchers donât seem to be aware of this type of application being built on Ethereum and so in the recent past a few updates have been made that end up hurting these types of non-financial apps.
Generally speaking, any app that must store any kind of data which is not directly related to settling ERC20 tokens on L1 from L2 is affected. These apps need to be able to post cheaply into L1 calldata (or another solution like it must be found). That is because when posting into L1âs calldata, this creates a timestamp whose creation isnât controllable by users. Consumer protocol like Farcaster and Kiwi News rely on this timestamp and the data to be retrievable into perpetuity to come to our own consensus between nodes.
I want to make everyone aware that:
L2 data is fundamentally not exitable from L2s as blobs become non-retrievable for a regular person
Since weâre building a protocol on top of Ethereum/Optimism, we were relying on this timestamped and into perpetuity retrievabe data to be accessible from a regular Ethereum node.
Since OPâs move to EIP-4844, weâre basically reliant on the good will of the Optimism sequencer operator to allow us access to this data as weâre still allowing users to delegate their keys on L2.
Why do I think this matters for objecting increasing the calldata on L1?
Many application builders have had wrong ideas about what problems L2s are solving for them. The simple fact of the matter is that any type of data where the app builder relies on the data to be retrievable from an Ethereum node in perpetuity MUST BE L1 calldata. L1 hasnât been a priority so for these types of app developers L1 calldata is, most likely, already considered too expensive. But making it more expensive basically kills their use cases as then building a protocol on top of Ethereum becomes much harder.
Why calldata and not blobs?
Blobsâs retrievability on nodes is only possible in the short term. Iâm not aware of a workflow that would make it possible for a Farcaster or Kiwi News node operator to download all blobs ever stored on Ethereum and if I read the docs correct, then blobs will be pruned from nodes after some time. That said, I think it is also unproductive to suggest that these apps âcan just load blobs from an archival service.â Iâm aware that centralized blob archival services for blobs exist. But we ourselves are building a protocol, and so to make it reliant on the good will of a blob archival provider isnât any better than being reliant on the good will of the Optimism sequencer operator. We want to base our protocol on Ethereum. Otherwise, if we base it on a centralized operator (be that the Optimism sequencer or a blob archival provider) our guarantees to users will be much worse.
The usefulness of calldata is that the data is timestamped (without the submitter being able to control the timestamp) and retrievable in perpetuity from a regular Ethereum node with a pretty good guarantee that any standard interface provider like Infura and Alchemy also provide access to the data and that we donât have to rely on any single entity.
I think it should be prioritized creating a space where L2 users can cheaply post small amounts of metadata that can be timestamped and stored into perpetuity. We must mitigate the damage that is currently done on L2s by unaware application builders (who store data on L2s thinking it is permissionlessly retrievable in perpetuity) and we must broadly teach that L2 data isnât as exitable from L2s as are ERC20 tokens.
I hope you can consider this feedback to make a decision. Iâm happy to provide more detailed information. A disclaimer: Iâve said that Farcaster is also reliant on using this âideal timestamped calldata, retrievable into perpetuity,â it still is.
However, from what I know, theyâre now actively working on Snapchain, which is most likely going to solve this problem for them. Iâd prefer to keep building Kiwi News on Ethereum, and so I think itâs important that this information gets circulated and understood to make wise decisions at the level of the core devs. Thank you.
To clarify, applications that still want to post data on-chain via calldata can continue to do soâit just might become more expensive. This cost adjustment is primarily aimed at reducing worst-case limits without impacting the majority of users. Itâs a trade-off between âmaking a small nr of tx more expensiveâ vs âDoS resistanceâ.
When data is posted as calldata, it becomes part of the history, which places a burden on every node. In contrast, blobs are pruned after approximately two weeks, offering a more scalable solution.
While pure data transactions via calldata may become costlier, there are proposals, such as EIP-7639, that could eventually enable pruning of calldata as well. Ethereum has always prioritized horizontal scaling and reducing hardware requirements, so this direction should come as no surprise.
The âtimestampâ argument applies equally to blobs. Your application could choose to store data in blobs, retain all blobs locally without pruning (which I do personally, and I assume most companies can do as well), and still make the dataâincluding timestampsâavailable for verification.
For our dapp, to delegate keys pre EIP-4844 a user would roughly pay 0.1 USD to delegate their key. Most users accepted this.
Post EIP-4844 with OP posting into blobs this cost as gone to 0.01 USD. However, I have described above why that is non-sustainable for us and what the security implications of OP posting into blobs storage are.
Now let me simulate what itâd cost on todayâs mainnet (napkin math ahead):
We need to post 3x 32bytes into calldata. We also have to pay the transaction envelope cost of 21k gas
Weâve maximally optimized gas costs of our key delegation contract down to 37k gas when posting on OP. We donât push the key into readable storage for other contracts or users, itâs plainly calldata and emitting an event (details). All this to say, thereâs very little optimization potential for us to bring costs down.
Farcaster has a more complex identity model where they also store the delegated key in storage (so for them itâll be much more expensive) - At todayâs 10gwei gas rate (the post EIP-4844 average gas cost was 10gwei), putting 3x 32bytes into calldata on L1 is $1.44. Thatâs unacceptably expensive as a cost to just get a consumer logged into a dapp!
Farcaster has already implemented a model where the user has to pay up to 1 USD when they want to login to another app using their Warpcast-internal delegation.
Many users on Warpcast have specifically complained about this being too costly for the use case. There are ways to look up the complaints using Warpcastâs search or even asking someone from the Merkle Manufactory team. Iâm sure theyâll confirm that this is an issue, if they havenât yet said publicly anyways.
Farcaster developers are moving away from Ethereum Optimism through the Snapchain proposal. Since in the Snapchain message timestamping is also harder to user-control this will allow them to put the delegations on the Snapchain, leaving Ethereum.
With Kiwi News, we also donât know were to go frankly. But Ethereum or the L2s donât seem to serve our use case at all. I however have the impression that weâre very Ethereum aligned. Why is Ethereum not serving us? We (consumer apps like FC and us) have a lot of growth potential through onboarding consumers. And our architectures are very aligned with Ethereumâs vision of decentralization. Putting on my Ethereum investor hat, itâs suboptimal that we seem to want to just ignore this use case
I want to stress here that I think Ethereumâs treatment of users who need to store small amounts of metadata into calldata is whatâs most likely leading them to implement alternatives, not the other way around. Mind you, the 1.44 USD that is shown on the Metamask screenshot is the cost today, at LOW gas fees. Your proposal wants to increase this.
Iâm aware that it puts a burden on everyone. But thereâs also a benefit to it. E.g. we and Farcaster are creating a new online space for people to curate and publish content. Do you want to continue to post on X? I donât.
If you read my original response youâll find that blobs are not a functionally perfect substitute of calldata for us. Blobs are only a perfect functional substitute for rollups who can settle their balance before blobs are pruned on the nodes. Obviously the Ethereum community has set a blobs pruning target that fits rollups. Blob space was custom made for rollups. Also, us and comparable projects have never been contacted about a prune duration we would prefer.
That said, our and Farcasterâs delegation keys live on peopleâs apps, so alternatives to the Warpcast and the Kiwi News client. Those keys have a long lifetime: Basically they keys that have been delegated for Kiwi News to upvote a story 1.5 years ago today still have to be retrievable through Ethereum calldata. Thatâs why Iâve been saying âin perpetuity.â
But I want to stress this again, our keys in our usersâ app live way beyond the duration that blobs are stored on Ethereum nodes. Hence we cannot use blobs. Iâve also argued extensively for other reasons why blobs cannot be used, e.g. that theyâre not easily permissionlessly retrievable after pruning.
The practical requirements of application builders suggest we may need to reconsider this approach.
I have been using Ethereum since 2018-2019 as an application builder and up until very recently the invariant that all data will stick around and be retrievable with an archive node has been solid (maybe not in ETH researcher circles, that may be).
Iâve never been a fan of EIP-4444 and state expiry proposals etc. When I talk to other builders I often get approval with that opinion. Dapp builders have informed opinions as we have to work with the actual system in the trenches and make it work for users!
If we were building an application, maybe. But weâre not building an application, weâve built a protocol using set reconciliation over a patricia merkle tree. So has Farcaster. If we want our nodes to come to a deterministic eventual consistency over who has posted what then theyâll have to have all the same source of truth of which key is allowed to post on behalf of which root key and during what time intervals (again: Happy to provide even more details here). Thatâs why us and Farcaster have been leveraging Ethereum to store key delegations. The key delegations must be stored in perpetuity retrievable for all Farcaster and Kiwi News nodes and they must be timestamped such that the user cannot back date their delegations. If this isnât the case then all sorts of bad things happen in our protocol. E.g. someone could claim to have casted a message in 2021 etc.
Again, you cannot suggest to use blobs here:
Weâre actually talking about the proper functioning of the p2p set reconciliation algorithm that reconciles messages on Farcaster and Kiwi News nodes.
E.g. my Kiwi News node has been running since 1.5 years.
A new Kiwi News node might come online today. Itâll want to download all messages from the last 1.5 years from my node.
If we were to look up key delegations in blobs, then my node would have received all blob related key delegations and could have retained them after they were pruned. No problem so far.
However, that new Kiwi News node which just came online CANNOT trust my node to understand who has delegated their keys. They can also not look up long gone key delegations in a permissionlessly decentralized way because blobs archives are only provisioned by centralized actors. And additionally, you have to consider that today all a new node operator has to do is enter a Alchemy or Infura RPC URL to the .env file. If you actually want people to download blobs from centralized actors this comes with all sorts of problems for us.
If you look at Farcasterâs Snapchain proposal over their original architecture, youâll find that the Snapchain which moves away from eventual consistency is much more complex than their earlier version.
The first time I saw FCâs architecture I was amazed by it as I had looked for a scalable dapp architecture that would also allow to scale the metadata that usually accumulates in dapps. I truly think that a generalization of the FC set reconciliation (including a few tweaks here and there) can get us really far towards the goal of a scalable dapp infrastructure for data.
Think about it. Weâve created the equivalent to rollups for data. All we have to store are key delegations. Itâs highly efficient for how much data you can store on these networks. And rollups cannot be used for data as Iâve already mentioned (because rollups cannot exit data). So why would you hinder this use case by increasing calldata costs? This type of architecture should be promoted and Ethereum should serve it, as it has served rollups! We should not let Farcaster escape into building their own thing! We should serve them
If you combine this sentiment with a proposal to increase costs, then this is Bitcoiner-style zero sum thinking.
When I worked at ascribe.io in 2015, the Bitcoiners also wanted to shut us down for storing art on the blockchain. Exactly the same argument. âIt burdens everyone.â This was right before the blocksize wars.
In either case, Bitcoin was ill-equipped to handle Ascribe, and it was sometimes met with outright hostility in the crypto community. On a technical level, Ascribe capitalized on a standardized function called OP_RETURN, which enables a user-defined sequence of up to 40 bytes to be written onto the blockchain. This allowed for things like links â pointing to images and longer texts living off-chain â to be stored in Bitcoin transactions. DaubenschĂźtz wrote about the project in a recent newsletter for NFT curation protocol JPG.space: âAscribe was built on Bitcoin ⌠by signing up via email and password, a hierarchical deterministic wallet was derived from the federation wallet; all ownership transactions were stored in an inventive protocol called SPOOL, the âSecure Public Online Ownership Ledgerâ(12). Similar to the well-known ERC721 NFT standard, the SPOOL encodes ownership information on-chain(13); but it was a hell of a mess as Bitcoin really wasnât meant for any of this.(14)â Indeed, issues occasionally arose with Bitcoin validator nodes rejecting the transactions for containing too much data; some hardcore Bitcoin purists even called OP_RETURN transactions âspammyâ or blamed them for bloating the network (15). As their anxieties attest, Ascribe was a critical intervention for attempting to broaden the blockchainâs functionality beyond straight-up financial exchange.
Iâm excited about Ethereumâs potential to scale beyond rollups! We already went from zero to one
What Ethereumâs implementation of the rollup roadmap has shown is that if we can identify a use case that clogs up L1 mainnet, and we design a better, cheaper, alternative solution, we scale Ethereum, bring down costs to many users and developers, allow more people to use the technology, and we broaden its appeal. Thatâs amazing! Meanwhile we have managed to leave Ethereum L1 largely unaffected. AND we have scaled DeFi and tokens and while we have preserved all L1 guarantees (as of now).
Iâd hence propose that if we want to scale L1 throughput, instead of curbing economic activity with restrictive policy proposals, we should come up with solutions that make it economically unviable to still use the L1 resource because thereâs a better alternative. A great example of this is rollups using blobs, which have produced a 91% cost benefit over storing their data in perpetuity in calldata (https://dune.com/0xRob/blobs).
So to unclog L1, I generally propose the following method to develop Ethereum:
Analyze what causes L1 clogging
Design an economically more viable solution and adjust it until developers use it
When developers move, L1 un-clogging happens
repeat
I think scaling this type of thinking in the culture of Ethereum developers could also lead to a broad enthusiasm about its culture. Maybe you know: https://www.solanaroadmap.com/
Iâd be excited if we could do this our own way too! Investors would love it too probably
And just to make sure: blobs != calldata for all use cases of calldata, so please donât suggest again to use blobs, thank you.
The rationale behind this EIP goes beyond simply saying, âitâs a burden for everyone, so weâre making it more expensive.â Large blocks pose a significant DoS risk, and ensuring the networkâs security is of utmost importance. Itâs a delicate balance between allowing larger blocks to enable certain types of applications and safeguarding the network, and this decision is far from black and white. For instance, storing HD videos directly on-chain is already impractical. While making calldata extremely cheap could make it feasible, doing so would drastically increase hardware requirements for nodes or require longer block times. In most cases, itâs more sensible for blockchains to host large datasets externally (e.g., via IPFS) and only store timestamped proofs on-chain.
With the gradual increase in gas limits, weâve seen how vulnerable the network can be to large blocks, and EIP-7623 addresses this by mitigating that risk. The vast majority of users wonât notice any difference. Among those affected, most are only writing small amounts of data into calldata, so theyâll see only a slight increase in costs (=negligible since 100 more gas is nothing in perspective of paying 21000 gas already).
Here are two recent instances where large blocks caused issues:
The comparison to âstoring HD Videoâ is a strawman argument, no one is proposing to do that.
Blobs are for ensuring the availability w.r.t. the fraud proof window, they are not for legit calldata use cases.
Additionally, this requires changes to the gas pricing APIâs, of which I do not know one provider who is even working on this (I would assume there are some at least). With EIP1559 there was a readiness checklist for downstream services signalling readiness for the new changes (e.g. Wallet providers supporting the usage). Do you have such a check list? I maintain a Gas Reporting Registry, GitHub - sambacha/gas-reporting: Reference documentation on every gas price API and all the different formats so this would be helpful to know and to provide additional information regarding the changes!
Thanks @Nerolation for responding to all these questions, I can appreciate both sides of the argument.
I agree, this is crazy. We just want our 3x 32 byte calldata to be preserved to do onchain delegations of Ethereum addresses, and at the same time we need to make this transaction consumer friendly. I have been pretty clear about this in earlier posts already. Not HD videos. Blobs are fundamentally not a consumer grade transaction anyways.
Blobs are for ensuring the availability w.r.t. the fraud proof window, they are not for legit calldata use cases.
Sorry for the accusation but it is my assessment that ETH researchers are pretty clueless or willfully against other use cases but token trading. Iâve brought this issue up many times already. There are many use cases of Ethereum beside cheap token swaps, but nobody seems to care. At the end of the day this is probably a failure from us who rely on long-term retrievability of data etc. The DeFi lobby on Ethereum is very strong, for good reasons
Youâre rightâstoring HD videos was a poor example, as it pushes the argument to the extreme.
Ultimately, though, itâs a trade-off between allowing larger blocks and enabling messaging through calldata.
This is just gaslighting, and it doesnât help in being taken seriouslyâwhich, to be clear, I have and still do. We both know that large blocks pose a significant DoS risk. On top of that, until we implement regular history pruning, every full node has to store it on disk, further driving up hardware requirements.
Nobody wants regular history pruning but token swap applications
This is just gaslighting, and it doesnât help in being taken seriously
You didnât give a shit about my concerns and the proposal has been moved further in the Pectra approval process. I have been pretty respectful so far. Also in person at Devconflict where my concerns were cast aside too. Why should I now care about an etiquette that you impose? Maybe take peopleâs concerns seriously next time before they start to violate your etiquette
I take othersâ concerns seriously, but thereâs another perspective in this discussion: some advocate for making calldata more expensive to mitigate the growing DoS risk that comes with increasing gas limits.
History expiry shouldnât come as a surpriseâit has been under consideration for years. For instance, Vitalikâs well-known roadmap diagram highlighted it years ago.
There will never be complete consensus on any topic. At some point, after weighing the pros and cons and going through the ACD process, a decision has to be made.
Iâm not surprised by state expiry. Iâve always been against it. As you can read from my above remarks, I have already built applications with data retrievability in perpetuity in mind since 2015, so Iâm quite aware that there are plans for history expiry, and Iâm still against it today. Vitalik is just one person. Heâs our leader but I donât agree with everything he proposes.
There will never be complete consensus on any topic. At some point, after weighing the pros and cons and going through the ACD process, a decision has to be made.
You should consider projecting the minority by eliciting their use cases. Many people will just stop building on Ethereum without a complaint beforehand. L2s are already useless to my application so I have moved off of Ethereum. Everyone sleepwalked into this. DeFi has a much stronger lobby. But people who want to etch stuff in the chain arguably donât have a strong lobby or billion dollar companies who can commission research and influence.
Iâm wondering how will that play in the mid-term with STARK/PQ-secure proofs verification.
All of the proving systems known atm that are PQ-secure depend on having large proofs.
If we blow-up CALLDATA cost, how do we anticipate verifying these proofs when we donât need any sort of DA solution (we just need to verify the proof to trigger certain SC function). So basically, no blob-related stuff to solve this.
It will become unreasonable to have apps as we have today that rely on G16 verification being âdecently cheapâ.
