My initial impression was that this is worse than just making transient storage patterns cost-effective for SSTORE/SLOAD. But since we have to pay storage gas up-front, the larger issue is that REVERT does not refund (see EIP-3978).
I would rather the gas costs be fixed than for two opcodes to go toward this. The new opcodes won’t fix existing gas injustices like UniswapV2’s re-entry lock. These opcodes just create new complexity to isolate a small portion of new code from the problem, rather than addressing the problem itself.
Thinking about potential use cases beyond making existing patterns more efficient I think it quickly becomes clear that dedicated opcodes are necessary. The upfront cost of setting a zero slot to a non-zero value combined with the cap to refunds is simply too high.
I don’t think you can efficiently use storage as transient storage because fundamentally nodes have to read from disk to check whether or not you’re overriding a value meaning that effort has to be priced in. With transient storage you’d get this fresh plot of data that can safely be modified throughout the transaction without needing to read anything from disk.
A different approach to transient storage in general would be to allow contracts to mark storage slots as impermanent at deployment, via EOF. You’d then change how SSTORE is priced by first checking whether the slot that’s being read / written to was marked as impermanent allowing you to safely know that the default value is 0 and that the final value does not have to be written to disk at the end of a transaction. But this feels more complex vs. just adding separate opcodes
How do you propose gas costs should be fixed? How is this related to refunding on revert?
As far as I can tell, the “real problem” is that storage is fundamentally a persistent data structure and resource, and using it for transient purposes is almost guranteed to be inefficient.
Revert is the main case in which storage writes are unfairly priced; we are paying for storage that will not occur, similar to your transient case. There is an EIP (EIP 3978) to fix this, but it needs work.
How is transient a storage just a band-aid? Do you disagree that storage accessing opcodes cannot safely have the same cost as standalone transient storage opcodes?
Yes (thought you shouldn’t ask questions in the negative; it makes answers ambiguous)
I don’t see why UniswapV2’s re-entry lock should cost more gas than equivalent transient opcodes (besides COLD_STORAGE_ACCESS). Neither change the state.
Here’s why I believe that transient storage opcodes are necessary or at least a better approach to alternatives, let me know what you disagree with and why because I don’t fully follow your argumentation so far:
The goal of good opcode gas pricing should be to fairly represent the practical runtime cost for nodes to process certain operations in the EVM
Storage is persistent meaning that changing / reading from it requires expensive disk operations (I assume that the state trie is stored on disk for the most part)
If you use persistent storage for transient storage you still have the overhead of at least reading from disk to ensure you’re not changing state, this cost needs to be accounted for
Conversely if you have dedicated transient storage operations you never have to worry about existing state at the beginning of a transaction, meaning you don’t need to do any disk operations
So fundamentally, even if you allow for full refunds, dedicated transient storage operations can achieve a lower cost than using persistent storage for transient storage because with SLOAD / SSTORE you need to check you’re not messing up some existing state.
Concretely let’s look at a reentrancy lock using storage and what operations need to be done:
Initial lock check (cold SLOAD): Read value from storage. Value at slot could’ve already been set and thus disk needs to be checked. Cached after initial retrieval to reduce subsequent cost.
Changing the state of the lock (warm SSTORE, change from initial value): The storage value can only be changed in the cache initially, however saving it later may require a disk write, therefore gas needs to be charged upfront
Subsequent lock check upon reentrancy (warm SLOAD): Simply retrieve state from cache, low cost.
Resetting lock state after method execution completes (warm SSTORE, value reverting): Only change state in cache. However by resetting to the original value a potential write to disk is averted, therefore the upfront gas cost can be refunded.
The example shows that even if you remove the cap on refunds there’ll always be the cost of the cold SLOAD which needs to look at the state trie. Am I missing something?
An alternative approach to dedicated transient storage opcodes would be to mark certain slots as transient via EOF or other opcode and then adjust the cost of the SLOAD / SSTORE if they’re interacting with transient slots, however I don’t like this approach as it packs even more complexity into the accounting and use of SSTORE / SLOAD which are already complicated enough on their own.
Definitely agree that something like this would be cleaner (but not exactly this, I think it’s too complex). Starting to appreciate this perspective.
Yes it can be trivially handled, but the point made by @chriseth is that it can also not be handled. The fact that it’s optional creates a new ability that smart contracts do not have currently. Specifically, a contract will now be able to check if it has previously executed as part of the same transaction. (Update: I was wrong, this is not entirely new! It can already be implemented by abusing CREATE[2] and SELFDESTRUCT, though it would become impossible again after EIP-4758.)
This is why Nikolai (rest in peace ) brought up TXID:
I am personally a fan of transient storage and want to see it happen, but I haven’t seen this newly enabled expressivity discussed in this thread except for Nikolai hinting at it, and I think it deserves more discussion.
Another similar angle is that transient storage at the tx level breaks referential transparency. Although the EVM is stateful, CALL is surprisingly pure except for just a few bits of state, which would need to be extended to include transient storage according to the current proposal. An alternative design would only propagate transient storage “downwards” but not “upwards”, i.e. reverting changes to it when returning from a call. Not sure if this alternative was discussed either.
Some use cases of transient storage EIP-1153 that work best with the transaction-level version of transient storage (rather than transient storage just for subcalls, which must use callbacks):
ERC20 temporary approve: approve a spender for a single transaction
Why not EIP-1363 approveAndCall? That requires the spender to support a callback or an intermediary contract which is more expensive and does not work well for approving multiple tokens
KYC: call a kyc contract to verify your identity, and then call other KYC-gated functions on other contracts that also check you’ve verified your identity in that transaction. This might be used with transient storage because you only want to prove your identity for a single transaction (rather than associate your identity to the account for all time, including e.g. after loss of a private key)
NFT royalties or ERC20 fee-on-transfer: pay a fee to unlock transfers for that token ID or token amount for a single transaction. Other designs of FOT or NFT royalties break composability, e.g. fee on transfer tokens may take fees either from the recipient or the sender balance. This avoids the breakage due to transfer fees. Also allows fees to be paid by a third party or in a different token.
If you propose everything should be a callback, you get callback hell. Imagine you want to temporarily approve an ERC20 to pay some NFT royalties to accept an offer on a KYC-gated NFT marketplace. Or, imagine that you want to approve 3 different tokens and then call into a contract. Both require 3 separate callbacks. How do you order them, and how do you construct calldata for passing through the several callbacks? How do you verify callbacks all came from the right contracts? Callbacks should be avoided unless necessary, and it’s not accurate to say all these use cases are supported via callbacks in the subcall-only transient storage version.
Yeah, while initially call-scoped transient storage sounded to me like it was just as good, because you can just use callbacks, I agree now that callbacks are not a good solution and tx-scoped transient storage would enable some valuable patterns to be implemented in a much nicer way.
At the risk of not making any friends here, I’d like to voice some concerns about this from the Solidity perspective (beyond those already voiced by @chriseth above).
While it may not be a huge effort to implement this on the execution layer in clients, this change induces a severe increase in the complexity of the semantics of the EVM that I’m not sure is fully appreciated in the discussion so far. Just some points I haven’t seen discussed exhaustively so far:
No matter what we do on the language level itself, in its current specification, this will be abused for in-memory mappings that are meant to be semantically call-local, but which will nonetheless bleed through on the transaction-level, which will increase the danger of reentrency bugs.
In general (as others have commented already), I’d expect transient storage to increase the complexity of any kind of static analysis and manual code analysis / auditing significantly.
I’d also expect the induced increase in semantic complexity and the reduction of the purity of EVM calls to cause pains for formal verification, in perpetuity.
On the language level alone, we’d need to consider whether transient storage warrants a new serialization format, which would significantly increase the complexity of the language semantics, and even if not, an additional data location means increasing the testing surface by an order of magnitude (the EVM already has an unhealthy amount of distinct data locations as is). In general, proper language support for this is non-trivial.
As far as use cases are concerned, transient storage still seems like a hacky catch-all solution to me - for reentrancy locks introducing an entire new address space of data is utter overkill; passing complex data structures between calls seems like an issue to be addressed in how calldata works instead; using it as a replacement for in-memory maps is downright dangerous etc. pp.
All that being said, if this makes it to a hardfork, we will of course have no choice, but to implement support on the Solidity level - but I sincerely hope you have thought this through properly and properly consulted with all the affected parties before jumping at this - to me personally, to be honest, seeing given the state of the discussion, it seems a bit premature to be considering this a candidate for inclusion in Shanghai.
EDIT: changing “seeing” to “given” since apparently otherwise it’s assumed that I just saw this for the first time and am randomly commenting out of the blue.
I don’t have much else to contribute after @ekpyron’s post. But honestly I think this is just so exaggerated. You’re seriously listing KYC as a reason to change EVM semantics heavily?
Regardless of whether it should go into any hard fork, it does look like the discussion has not taken into account all the potential problems this might bring. For the people championing the EIP, are you really sure that you’ve thought through the problems mentioned above, and that you’re actively fine with it instead of just ignoring it because it brings you the feature you want?
I don’t think more work for the clients and compilers is a problem per se, of course with progress come the changes and they need to be supported. But we need to be sure (or at least convinced) that this is for the good of the EVM, and not just a hack that will make a few happy and the EVM worse.
edit:
Just adding a final point: many of the listed problems will not go away by simply choosing “tx level or call level or whatever else”. There are clear pros and clear cons about transient storage. Those are likely not going to change. In the end there needs to be a decision on which outweighs which. Either the problems are too much, or we’re fine with them given the pros we get.
Given the unanimous app developer support, it’s quite silly for solidity compiler devs to join the discussion this late to tell their users they shouldn’t have it for their own protection. It’s the app developer’s responsibility to ensure the security of the contracts they write, not the compiler or other tools, which is why the arguments that bugs could be written with it are unconvincing.
It’s still much more expensive than memory, and in my experience in-memory maps are rarely, if ever, useful in smart contracts. Especially maps costing as much as 100 gas per read/write. You are better off iterating over the entire list up to a certain size of list for 50+ entries. So, yes you’re right, but it’s almost never useful, so this particular concern is insignificant in comparison to all the other ways you can be vulnerable to reentrancy.
Very general concern, please provide examples. Having written and read code that uses transient storage possibly more than anyone so far I don’t think it makes code any harder to read, audit, etc…
It’s exactly the same as regular storage, you either clear it at the end of a transaction or very intentionally don’t clear it for future calls. Forgetting to clear transient storage has the same reentrancy issues as forgetting to clear regular storage used transiently. It’s not actually a reentrance bug that in a later transaction the slots were 0 again (although it may cause some other kind of bug, depending on your use case).
“Reduction of purity”, could you explain? And please be specific with the pains it causes?
My experience with formal verification, having tried to do FV with Uniswap V3, is that it’s already impractical for stateful contracts. That said, the difference here is now you have to consider subsequent calls across transactions as well as subsequent calls in the same transaction. Meaning transient storage slots can either have a value set from a previous transaction (same as storage) or 0 values at the beginning of a transaction. That doesn’t seem like a huge difference, and it won’t even matter for contracts where formal verification is actually practical.
It depends heavily on your current abstraction, according to @big_tech_sux on Twitter it was added to Vyper in ~20 lines of code.
Do the storage refunds used for this today strike you as elegant? Considering they have a 20% cap, in order to get rid of gas token, and you still have to read from storage a value that is always 0? Please suggest an alternate solution if you are going to call this one overkill. I would love to support a simpler solution that supports the required use cases, enables parallel execution, allows simplification of storage refunds, etc.
I’ve championed for this since late last year, discussed with @ethchris as early as February this year. It’s a bit late to be getting up to speed publicly on the discussion, as your concerns are more premature than the EIP. The EIP is already implemented in several clients with passing comprehensive cross-functional EVM tests. Last I checked it’s further along in terms of implementation and specification than any other Shanghai CFI EIP.
There have been many opportunities to suggest alternatives (to transient storage specifically) from the core solidity team, and I still haven’t seen any suggested from the team. Asking that you please provide specific examples and solutions in your feedback, rather than general concerns at this point.
it’s quite silly for solidity compiler devs to join the discussion this late to tell their users they shouldn’t have it for their own protection
This is disingenuous and borderline disrespectful. This is an open discussion and anyone is entitled to share their opinion at any point in time.
it was added to Vyper in ~20 lines of code.
Have you actually verified that? Here’s the commit that added it: https://github.com/vyperlang/vyper/compare/master...transient
Yes, 20 lines of code, but not a single test. I checked the tests directory, not a single reference to transient. That may be fine for a PoC, but doesn’t look ready for production imho. (which is ofc understandable, to not add full support until it’s actually confirmed)
It’s a bit late to be getting up to speed publicly on the discussion
Why are you trying to gatekeep this discussion? I disagree that the concerns are more premature than the EIP, since a few haven’t even been properly answered to, only conveniently “forgotten”, as seems to be a common way of getting rid of EIP criticism. The EIP being implemented in different clients is probably a requirement but it’s far from being sufficient for something to go in, is it not?
There have been many opportunities to suggest alternatives (to transient storage specifically) from the core solidity team
Why should the Solidity team have to provide better alternatives to it? At this point a few members are simply voicing unresolved concerns that may worsen the EVM, and your attempt to call them “resolved” and say “it’s too late” is weird, to say the least. None of us here get to make those decisions alone.
It’s rather silly to claim that this it’s late for bringing this up, if for example the issue for static analysis has come up as early as 2018 (EIP-1153: Transient storage opcodes - #25 by Arachnid). Issues don’t just vanish if ignored. Same goes for @chriseth’s concerns. As for the reduction in the purity of EVM calls @frangio explained that already above.
As for non-trivial language support: I can of course hack in support of the opcode in a few lines of code without tests, we have a PR doing the same, that doesn’t mean that it suddenly becomes a properly tested high-level language construct just like that. But as @leonardoalt already pointed out, implementational difficulty is not the major issue here, I’d just want to clarify that considering that to be trivial is a misrepresentation.
And, of course, compiler developers, static analysis and formal verification are major parts in smart contract security and should of course be consulted in evaluating such a change that, after all, targets to become a high-level language construct.
Are there any statements of these changes not causing issues and being perfectly fine and manageable from people doing static analysis or formal verification or auditors? My main point is that a change like this IMHO requires active positive confirmation from these parties and I don’t see any of that.
Lists of developers that are in support here and here.
Mark Tyneway from Optimism as well as Arbitrum (posted above, also consulted DZack23 on twitter) are in support.
If you don’t consider the developers that are writing the code responsible for the majority of the Ethereum chain usage ‘unanimous’, I don’t know what to tell you. If you haven’t seen these lists before, then that’s exactly my point about joining the discussion late.
I’m pointing out that if you are saying it’s a question of tradeoffs, you should give more weight to user feedback
I am asking you to get up to speed privately and then post examples and specific feedback, rather than general concerns. I have responded to every single point in @ekpyron’s post as well as @ethchris’s, that’s pretty far from gatekeeping the discussion I think. An open discussion is not the same as an open and productive discussion.
Is everyone aware of the potential problems, new complexity etc and are OK with that, or just interested in gas savings? Ofc I can’t say for sure either way, and won’t assume, but tweets that likely didn’t warrant the proper diligence rather sounds like the latter. Also the software like clients and compilers adding code support to them doesn’t mean support for it getting in, as can be seen with the Solidity PR also being there.
Huh? Why is that?
With all the deserved due respect to Nick and Frangio, screenshots of texts and single sentences of the form “sounds simple” hardly show diligence.
I’d for example be interested in the position of people working with formal EVM semantics, like e.g. Runtime Verification, or such. Carrying another kind of state with different semantics through calls of course adds to complexity there. Be it manual or not, it basically doubles the effort to prove invariants of a smart contract, since they’re contingent on the position in a transaction call chain. Is all of that managable? Theoretically, sure. Is it trivial? Depending on what you do maybe, maybe very much not. Is that enough reason for being sceptical about this EIP? For me it is, in general it may, but doesn’t have to be.
But anyways, I actually don’t quite understand the aggressiveness. Of course, I’m aware of this having the support of application developers - but that doesn’t mean that it has to universally be considered as a good move (especially in any fixed particular variant).
As for memory mappings: I’ve even seen this mentioned as use case and I think you’re underestimating the cost of iterating lists. I don’t see the cost as the limiting factor here, until it’s basically back at a cold storage read of a zero - a reduction of the address space would be, either for the opcodes or with the EOF-based marking of storage slots as alternatives.
Storage repricing and refund adjustments are also still alternatives that have come up often. Which are complex and not nice, sure, but for modelling storage not merely as disk space, but as cached disk space, which accurately reflects what happens in clients, I don’t see them going anywhere regardless. Would also be interested how this could no longer or better be done with Verkle trees.
In any case, I’m mainly pointing out that this EIP doesn’t only have fans (and the response IMHO actually reaffirms that this is more than necessary). It’s not like you will and have to have unanimous support in the end, but as I said originally, I hope the decision takes the price of this in terms of complexity into account (which surely is unarguably not just zero :-)).
To clarify: This by no means intends to disqualify anyone’s opinions. Everyone on the “support” list is ofc 100% knowledgeable and capable of judging this. However I do believe that scattered support like screenshots and tweets are different from structured support, but that’s just my opinion. I’m completely fine with being on the “wrong” side here
) brought up TXID:
