True, but why are you storing values in the mapping unless you later need to iterate through the map? I can’t think of a use case that uses a mapping but doesn’t store a list of keys. EDIT: you can also just count the number of set keys, which is more efficient, and check that it’s 0 (require the caller to do something that unsets the keys)
Sidenote, an iterable map data type would be cool to have in solidity, rather than doing this separately.
One case it breaks is a call to a hypothetical ERC20#temporaryApprove that only lasts one transaction, and then another call to a separate contract that calls #transferFrom (which looks for a transient approval before a regular approval). In the context of 3074, this might be the cheaper version of approve and swap in a single transaction (for tokens that would support it)
If the SSTORE_RESET refund were greater, there would be no need for transient storage; you could just reset the storage and its cost would be reduced.
In addition, reverts are mispriced (and should refund), which prevents persistent storage from cheaply implementing transient storage.
Hey everyone, I visualised the adjusted EVM with transient storage. Let me know if I missed something. It helped me a lot to explain the aim of EIP-1153. Maybe some of you can make use of it.
After reading EIP-1153 again: one strange thing about EIP-1153 is that it doesn’t get reverted on REVERT or OOG in its own frame. How a robust mutex is implemented if it may leave the mutex activated by an error? This can be abused by an attacker to censor a certain contract.
Let say a contract A calls another trusted contract T which uses a transient mutex.
The attacker can perform a transaction that calls a contract A but blocks the execution of a targeted child contract T by performing these steps:
Call to T with very little gas so that it raises OOG during the execution of T, leaving the mutex active.
Call A which will try to call T and that call will fail because of the mutex.
So this brings the stack overflow security problem back, but now using broken mutexes.
Interaction with reverts should be be mandatory.
At least transient memory writes should be reverted if the contract owning it reverts!
The original version of EIP-1153 did not interact with reverts. However, I edited the EIP so that it interacts with reverts in the exact same way that regular storage does. So it indeed works as you want it to, and the EIP-1153 text plus all existing implementations reflect that:
If a frame reverts, all writes to transient storage that took place between entry to the frame and the return are reverted, including those that took place in inner calls. This mimics the behavior of persistent storage.
Dropping a link here to the EIP-1153 implementation project board: EIP-1153 · GitHub
My initial impression was that this is worse than just making transient storage patterns cost-effective for SSTORE/SLOAD. But since we have to pay storage gas up-front, the larger issue is that REVERT does not refund (see EIP-3978).
I would rather the gas costs be fixed than for two opcodes to go toward this. The new opcodes won’t fix existing gas injustices like UniswapV2’s re-entry lock. These opcodes just create new complexity to isolate a small portion of new code from the problem, rather than addressing the problem itself.
Thinking about potential use cases beyond making existing patterns more efficient I think it quickly becomes clear that dedicated opcodes are necessary. The upfront cost of setting a zero slot to a non-zero value combined with the cap to refunds is simply too high.
I don’t think you can efficiently use storage as transient storage because fundamentally nodes have to read from disk to check whether or not you’re overriding a value meaning that effort has to be priced in. With transient storage you’d get this fresh plot of data that can safely be modified throughout the transaction without needing to read anything from disk.
A different approach to transient storage in general would be to allow contracts to mark storage slots as impermanent at deployment, via EOF. You’d then change how SSTORE is priced by first checking whether the slot that’s being read / written to was marked as impermanent allowing you to safely know that the default value is 0 and that the final value does not have to be written to disk at the end of a transaction. But this feels more complex vs. just adding separate opcodes
How do you propose gas costs should be fixed? How is this related to refunding on revert?
As far as I can tell, the “real problem” is that storage is fundamentally a persistent data structure and resource, and using it for transient purposes is almost guranteed to be inefficient.
Revert is the main case in which storage writes are unfairly priced; we are paying for storage that will not occur, similar to your transient case. There is an EIP (EIP 3978) to fix this, but it needs work.
How is transient a storage just a band-aid? Do you disagree that storage accessing opcodes cannot safely have the same cost as standalone transient storage opcodes?
Yes (thought you shouldn’t ask questions in the negative; it makes answers ambiguous)
I don’t see why UniswapV2’s re-entry lock should cost more gas than equivalent transient opcodes (besides COLD_STORAGE_ACCESS). Neither change the state.
Here’s why I believe that transient storage opcodes are necessary or at least a better approach to alternatives, let me know what you disagree with and why because I don’t fully follow your argumentation so far:
The goal of good opcode gas pricing should be to fairly represent the practical runtime cost for nodes to process certain operations in the EVM
Storage is persistent meaning that changing / reading from it requires expensive disk operations (I assume that the state trie is stored on disk for the most part)
If you use persistent storage for transient storage you still have the overhead of at least reading from disk to ensure you’re not changing state, this cost needs to be accounted for
Conversely if you have dedicated transient storage operations you never have to worry about existing state at the beginning of a transaction, meaning you don’t need to do any disk operations
So fundamentally, even if you allow for full refunds, dedicated transient storage operations can achieve a lower cost than using persistent storage for transient storage because with SLOAD / SSTORE you need to check you’re not messing up some existing state.
Concretely let’s look at a reentrancy lock using storage and what operations need to be done:
Initial lock check (cold SLOAD): Read value from storage. Value at slot could’ve already been set and thus disk needs to be checked. Cached after initial retrieval to reduce subsequent cost.
Changing the state of the lock (warm SSTORE, change from initial value): The storage value can only be changed in the cache initially, however saving it later may require a disk write, therefore gas needs to be charged upfront
Subsequent lock check upon reentrancy (warm SLOAD): Simply retrieve state from cache, low cost.
Resetting lock state after method execution completes (warm SSTORE, value reverting): Only change state in cache. However by resetting to the original value a potential write to disk is averted, therefore the upfront gas cost can be refunded.
The example shows that even if you remove the cap on refunds there’ll always be the cost of the cold SLOAD which needs to look at the state trie. Am I missing something?
An alternative approach to dedicated transient storage opcodes would be to mark certain slots as transient via EOF or other opcode and then adjust the cost of the SLOAD / SSTORE if they’re interacting with transient slots, however I don’t like this approach as it packs even more complexity into the accounting and use of SSTORE / SLOAD which are already complicated enough on their own.
Definitely agree that something like this would be cleaner (but not exactly this, I think it’s too complex). Starting to appreciate this perspective.
Yes it can be trivially handled, but the point made by @chriseth is that it can also not be handled. The fact that it’s optional creates a new ability that smart contracts do not have currently. Specifically, a contract will now be able to check if it has previously executed as part of the same transaction. (Update: I was wrong, this is not entirely new! It can already be implemented by abusing CREATE[2] and SELFDESTRUCT, though it would become impossible again after EIP-4758.)
This is why Nikolai (rest in peace ) brought up TXID:
I am personally a fan of transient storage and want to see it happen, but I haven’t seen this newly enabled expressivity discussed in this thread except for Nikolai hinting at it, and I think it deserves more discussion.
Another similar angle is that transient storage at the tx level breaks referential transparency. Although the EVM is stateful, CALL is surprisingly pure except for just a few bits of state, which would need to be extended to include transient storage according to the current proposal. An alternative design would only propagate transient storage “downwards” but not “upwards”, i.e. reverting changes to it when returning from a call. Not sure if this alternative was discussed either.
Some use cases of transient storage EIP-1153 that work best with the transaction-level version of transient storage (rather than transient storage just for subcalls, which must use callbacks):
ERC20 temporary approve: approve a spender for a single transaction
Why not EIP-1363 approveAndCall? That requires the spender to support a callback or an intermediary contract which is more expensive and does not work well for approving multiple tokens
KYC: call a kyc contract to verify your identity, and then call other KYC-gated functions on other contracts that also check you’ve verified your identity in that transaction. This might be used with transient storage because you only want to prove your identity for a single transaction (rather than associate your identity to the account for all time, including e.g. after loss of a private key)
NFT royalties or ERC20 fee-on-transfer: pay a fee to unlock transfers for that token ID or token amount for a single transaction. Other designs of FOT or NFT royalties break composability, e.g. fee on transfer tokens may take fees either from the recipient or the sender balance. This avoids the breakage due to transfer fees. Also allows fees to be paid by a third party or in a different token.
If you propose everything should be a callback, you get callback hell. Imagine you want to temporarily approve an ERC20 to pay some NFT royalties to accept an offer on a KYC-gated NFT marketplace. Or, imagine that you want to approve 3 different tokens and then call into a contract. Both require 3 separate callbacks. How do you order them, and how do you construct calldata for passing through the several callbacks? How do you verify callbacks all came from the right contracts? Callbacks should be avoided unless necessary, and it’s not accurate to say all these use cases are supported via callbacks in the subcall-only transient storage version.
Yeah, while initially call-scoped transient storage sounded to me like it was just as good, because you can just use callbacks, I agree now that callbacks are not a good solution and tx-scoped transient storage would enable some valuable patterns to be implemented in a much nicer way.
At the risk of not making any friends here, I’d like to voice some concerns about this from the Solidity perspective (beyond those already voiced by @chriseth above).
While it may not be a huge effort to implement this on the execution layer in clients, this change induces a severe increase in the complexity of the semantics of the EVM that I’m not sure is fully appreciated in the discussion so far. Just some points I haven’t seen discussed exhaustively so far:
No matter what we do on the language level itself, in its current specification, this will be abused for in-memory mappings that are meant to be semantically call-local, but which will nonetheless bleed through on the transaction-level, which will increase the danger of reentrency bugs.
In general (as others have commented already), I’d expect transient storage to increase the complexity of any kind of static analysis and manual code analysis / auditing significantly.
I’d also expect the induced increase in semantic complexity and the reduction of the purity of EVM calls to cause pains for formal verification, in perpetuity.
On the language level alone, we’d need to consider whether transient storage warrants a new serialization format, which would significantly increase the complexity of the language semantics, and even if not, an additional data location means increasing the testing surface by an order of magnitude (the EVM already has an unhealthy amount of distinct data locations as is). In general, proper language support for this is non-trivial.
As far as use cases are concerned, transient storage still seems like a hacky catch-all solution to me - for reentrancy locks introducing an entire new address space of data is utter overkill; passing complex data structures between calls seems like an issue to be addressed in how calldata works instead; using it as a replacement for in-memory maps is downright dangerous etc. pp.
All that being said, if this makes it to a hardfork, we will of course have no choice, but to implement support on the Solidity level - but I sincerely hope you have thought this through properly and properly consulted with all the affected parties before jumping at this - to me personally, to be honest, seeing given the state of the discussion, it seems a bit premature to be considering this a candidate for inclusion in Shanghai.
EDIT: changing “seeing” to “given” since apparently otherwise it’s assumed that I just saw this for the first time and am randomly commenting out of the blue.
I don’t have much else to contribute after @ekpyron’s post. But honestly I think this is just so exaggerated. You’re seriously listing KYC as a reason to change EVM semantics heavily?
Regardless of whether it should go into any hard fork, it does look like the discussion has not taken into account all the potential problems this might bring. For the people championing the EIP, are you really sure that you’ve thought through the problems mentioned above, and that you’re actively fine with it instead of just ignoring it because it brings you the feature you want?
I don’t think more work for the clients and compilers is a problem per se, of course with progress come the changes and they need to be supported. But we need to be sure (or at least convinced) that this is for the good of the EVM, and not just a hack that will make a few happy and the EVM worse.
edit:
Just adding a final point: many of the listed problems will not go away by simply choosing “tx level or call level or whatever else”. There are clear pros and clear cons about transient storage. Those are likely not going to change. In the end there needs to be a decision on which outweighs which. Either the problems are too much, or we’re fine with them given the pros we get.
Given the unanimous app developer support, it’s quite silly for solidity compiler devs to join the discussion this late to tell their users they shouldn’t have it for their own protection. It’s the app developer’s responsibility to ensure the security of the contracts they write, not the compiler or other tools, which is why the arguments that bugs could be written with it are unconvincing.
It’s still much more expensive than memory, and in my experience in-memory maps are rarely, if ever, useful in smart contracts. Especially maps costing as much as 100 gas per read/write. You are better off iterating over the entire list up to a certain size of list for 50+ entries. So, yes you’re right, but it’s almost never useful, so this particular concern is insignificant in comparison to all the other ways you can be vulnerable to reentrancy.
Very general concern, please provide examples. Having written and read code that uses transient storage possibly more than anyone so far I don’t think it makes code any harder to read, audit, etc…
It’s exactly the same as regular storage, you either clear it at the end of a transaction or very intentionally don’t clear it for future calls. Forgetting to clear transient storage has the same reentrancy issues as forgetting to clear regular storage used transiently. It’s not actually a reentrance bug that in a later transaction the slots were 0 again (although it may cause some other kind of bug, depending on your use case).
“Reduction of purity”, could you explain? And please be specific with the pains it causes?
My experience with formal verification, having tried to do FV with Uniswap V3, is that it’s already impractical for stateful contracts. That said, the difference here is now you have to consider subsequent calls across transactions as well as subsequent calls in the same transaction. Meaning transient storage slots can either have a value set from a previous transaction (same as storage) or 0 values at the beginning of a transaction. That doesn’t seem like a huge difference, and it won’t even matter for contracts where formal verification is actually practical.
It depends heavily on your current abstraction, according to @big_tech_sux on Twitter it was added to Vyper in ~20 lines of code.
Do the storage refunds used for this today strike you as elegant? Considering they have a 20% cap, in order to get rid of gas token, and you still have to read from storage a value that is always 0? Please suggest an alternate solution if you are going to call this one overkill. I would love to support a simpler solution that supports the required use cases, enables parallel execution, allows simplification of storage refunds, etc.
I’ve championed for this since late last year, discussed with @ethchris as early as February this year. It’s a bit late to be getting up to speed publicly on the discussion, as your concerns are more premature than the EIP. The EIP is already implemented in several clients with passing comprehensive cross-functional EVM tests. Last I checked it’s further along in terms of implementation and specification than any other Shanghai CFI EIP.
There have been many opportunities to suggest alternatives (to transient storage specifically) from the core solidity team, and I still haven’t seen any suggested from the team. Asking that you please provide specific examples and solutions in your feedback, rather than general concerns at this point.
it’s quite silly for solidity compiler devs to join the discussion this late to tell their users they shouldn’t have it for their own protection
This is disingenuous and borderline disrespectful. This is an open discussion and anyone is entitled to share their opinion at any point in time.
it was added to Vyper in ~20 lines of code.
Have you actually verified that? Here’s the commit that added it: https://github.com/vyperlang/vyper/compare/master...transient
Yes, 20 lines of code, but not a single test. I checked the tests directory, not a single reference to transient. That may be fine for a PoC, but doesn’t look ready for production imho. (which is ofc understandable, to not add full support until it’s actually confirmed)
It’s a bit late to be getting up to speed publicly on the discussion
Why are you trying to gatekeep this discussion? I disagree that the concerns are more premature than the EIP, since a few haven’t even been properly answered to, only conveniently “forgotten”, as seems to be a common way of getting rid of EIP criticism. The EIP being implemented in different clients is probably a requirement but it’s far from being sufficient for something to go in, is it not?
There have been many opportunities to suggest alternatives (to transient storage specifically) from the core solidity team
Why should the Solidity team have to provide better alternatives to it? At this point a few members are simply voicing unresolved concerns that may worsen the EVM, and your attempt to call them “resolved” and say “it’s too late” is weird, to say the least. None of us here get to make those decisions alone.
) brought up TXID: