Moody and me have talked in the meantime. I’m still not convinced it is worth the complexity, but I understand some of the use-cases now.
Having spent some more thoughts on this, one thing that caught my eye is that this is a transaction-based feature instead of a call-based feature. Until now (please correct me if I’m wrong), the only way an “inner call” can be distinguished from an “outer call” inside the EVM is using tx.origin and maybe some aspects of selfdestruct. And both tx.origin and selfdestruct have been heavily criticized because of that, for example with regards to account abstraction.
Please excuse me if this has been discussed before, but one problem I can see (and mind me, it is a complicated feature) is that if we reset transient storage to zero only at the end of the transaction, we can no longer group two transactions together.
For example, if you have a reentrancy lock and people do not reset it (because they want to save the gas) and rely on it being reset at the end of the transaction, you will no longer be able to call this contract for the rest of the transaction at all.
We can solve this problem by resetting the transient storage of a contract whenever a call returns from the contract and the same contract is not in the current call stack. It would eliminate some other use cases, but it would feel much cleaner to me.
You could also check the code length of caller is greater than zero. You can’t do this for code running in a constructor
I’m not sure if the exact criticisms of those opcodes applies here. Could you elaborate or share a link?
This is interesting, but it might make transient storage harder to implement in the clients, since you need to keep track of the depth of the call stack by address. I don’t think it would introduce any additional risk, but the contract is already capable of resetting storage before returning from the calls (and also if the call reverts, all transient storage writes within it are reverted as well.) The 100 gas per TSTORE saved is probably not worth optimizing, and actually more explicitly represents the cost of clearing the slots. IMO this should be handled by the smart contract, not by the opcode, because it is trivial to handle within the smart contract.
If you use a transient mapping and don’t store the keys, the contract cannot reset it. Also this is just something people will forget about - “ah, it will be cleared at the end of the transaction anyway, I don’t need to do it”.
The current go-ethereum implementation curretnly keeps a full transaction log of all transient store operations - clearly an address per call would not be a big deal.
What I would be more interested in is which use-cases would not be possible any more in this modified version. Most cases I have seen call back into the caller, so those would be fine, as is the reentrancy lock obviously, but are there any where this does not happen?
True, but why are you storing values in the mapping unless you later need to iterate through the map? I can’t think of a use case that uses a mapping but doesn’t store a list of keys. EDIT: you can also just count the number of set keys, which is more efficient, and check that it’s 0 (require the caller to do something that unsets the keys)
Sidenote, an iterable map data type would be cool to have in solidity, rather than doing this separately.
One case it breaks is a call to a hypothetical ERC20#temporaryApprove that only lasts one transaction, and then another call to a separate contract that calls #transferFrom (which looks for a transient approval before a regular approval). In the context of 3074, this might be the cheaper version of approve and swap in a single transaction (for tokens that would support it)
If the SSTORE_RESET refund were greater, there would be no need for transient storage; you could just reset the storage and its cost would be reduced.
In addition, reverts are mispriced (and should refund), which prevents persistent storage from cheaply implementing transient storage.
Hey everyone, I visualised the adjusted EVM with transient storage. Let me know if I missed something. It helped me a lot to explain the aim of EIP-1153. Maybe some of you can make use of it.
After reading EIP-1153 again: one strange thing about EIP-1153 is that it doesn’t get reverted on REVERT or OOG in its own frame. How a robust mutex is implemented if it may leave the mutex activated by an error? This can be abused by an attacker to censor a certain contract.
Let say a contract A calls another trusted contract T which uses a transient mutex.
The attacker can perform a transaction that calls a contract A but blocks the execution of a targeted child contract T by performing these steps:
Call to T with very little gas so that it raises OOG during the execution of T, leaving the mutex active.
Call A which will try to call T and that call will fail because of the mutex.
So this brings the stack overflow security problem back, but now using broken mutexes.
Interaction with reverts should be be mandatory.
At least transient memory writes should be reverted if the contract owning it reverts!
The original version of EIP-1153 did not interact with reverts. However, I edited the EIP so that it interacts with reverts in the exact same way that regular storage does. So it indeed works as you want it to, and the EIP-1153 text plus all existing implementations reflect that:
If a frame reverts, all writes to transient storage that took place between entry to the frame and the return are reverted, including those that took place in inner calls. This mimics the behavior of persistent storage.
Dropping a link here to the EIP-1153 implementation project board: EIP-1153 · GitHub
My initial impression was that this is worse than just making transient storage patterns cost-effective for SSTORE/SLOAD. But since we have to pay storage gas up-front, the larger issue is that REVERT does not refund (see EIP-3978).
I would rather the gas costs be fixed than for two opcodes to go toward this. The new opcodes won’t fix existing gas injustices like UniswapV2’s re-entry lock. These opcodes just create new complexity to isolate a small portion of new code from the problem, rather than addressing the problem itself.
Thinking about potential use cases beyond making existing patterns more efficient I think it quickly becomes clear that dedicated opcodes are necessary. The upfront cost of setting a zero slot to a non-zero value combined with the cap to refunds is simply too high.
I don’t think you can efficiently use storage as transient storage because fundamentally nodes have to read from disk to check whether or not you’re overriding a value meaning that effort has to be priced in. With transient storage you’d get this fresh plot of data that can safely be modified throughout the transaction without needing to read anything from disk.
A different approach to transient storage in general would be to allow contracts to mark storage slots as impermanent at deployment, via EOF. You’d then change how SSTORE is priced by first checking whether the slot that’s being read / written to was marked as impermanent allowing you to safely know that the default value is 0 and that the final value does not have to be written to disk at the end of a transaction. But this feels more complex vs. just adding separate opcodes
How do you propose gas costs should be fixed? How is this related to refunding on revert?
As far as I can tell, the “real problem” is that storage is fundamentally a persistent data structure and resource, and using it for transient purposes is almost guranteed to be inefficient.
Revert is the main case in which storage writes are unfairly priced; we are paying for storage that will not occur, similar to your transient case. There is an EIP (EIP 3978) to fix this, but it needs work.
How is transient a storage just a band-aid? Do you disagree that storage accessing opcodes cannot safely have the same cost as standalone transient storage opcodes?
Yes (thought you shouldn’t ask questions in the negative; it makes answers ambiguous)
I don’t see why UniswapV2’s re-entry lock should cost more gas than equivalent transient opcodes (besides COLD_STORAGE_ACCESS). Neither change the state.
Here’s why I believe that transient storage opcodes are necessary or at least a better approach to alternatives, let me know what you disagree with and why because I don’t fully follow your argumentation so far:
The goal of good opcode gas pricing should be to fairly represent the practical runtime cost for nodes to process certain operations in the EVM
Storage is persistent meaning that changing / reading from it requires expensive disk operations (I assume that the state trie is stored on disk for the most part)
If you use persistent storage for transient storage you still have the overhead of at least reading from disk to ensure you’re not changing state, this cost needs to be accounted for
Conversely if you have dedicated transient storage operations you never have to worry about existing state at the beginning of a transaction, meaning you don’t need to do any disk operations
So fundamentally, even if you allow for full refunds, dedicated transient storage operations can achieve a lower cost than using persistent storage for transient storage because with SLOAD / SSTORE you need to check you’re not messing up some existing state.
Concretely let’s look at a reentrancy lock using storage and what operations need to be done:
Initial lock check (cold SLOAD): Read value from storage. Value at slot could’ve already been set and thus disk needs to be checked. Cached after initial retrieval to reduce subsequent cost.
Changing the state of the lock (warm SSTORE, change from initial value): The storage value can only be changed in the cache initially, however saving it later may require a disk write, therefore gas needs to be charged upfront
Subsequent lock check upon reentrancy (warm SLOAD): Simply retrieve state from cache, low cost.
Resetting lock state after method execution completes (warm SSTORE, value reverting): Only change state in cache. However by resetting to the original value a potential write to disk is averted, therefore the upfront gas cost can be refunded.
The example shows that even if you remove the cap on refunds there’ll always be the cost of the cold SLOAD which needs to look at the state trie. Am I missing something?
An alternative approach to dedicated transient storage opcodes would be to mark certain slots as transient via EOF or other opcode and then adjust the cost of the SLOAD / SSTORE if they’re interacting with transient slots, however I don’t like this approach as it packs even more complexity into the accounting and use of SSTORE / SLOAD which are already complicated enough on their own.
Definitely agree that something like this would be cleaner (but not exactly this, I think it’s too complex). Starting to appreciate this perspective.
Yes it can be trivially handled, but the point made by @chriseth is that it can also not be handled. The fact that it’s optional creates a new ability that smart contracts do not have currently. Specifically, a contract will now be able to check if it has previously executed as part of the same transaction. (Update: I was wrong, this is not entirely new! It can already be implemented by abusing CREATE[2] and SELFDESTRUCT, though it would become impossible again after EIP-4758.)
This is why Nikolai (rest in peace ) brought up TXID:
I am personally a fan of transient storage and want to see it happen, but I haven’t seen this newly enabled expressivity discussed in this thread except for Nikolai hinting at it, and I think it deserves more discussion.
Another similar angle is that transient storage at the tx level breaks referential transparency. Although the EVM is stateful, CALL is surprisingly pure except for just a few bits of state, which would need to be extended to include transient storage according to the current proposal. An alternative design would only propagate transient storage “downwards” but not “upwards”, i.e. reverting changes to it when returning from a call. Not sure if this alternative was discussed either.
Some use cases of transient storage EIP-1153 that work best with the transaction-level version of transient storage (rather than transient storage just for subcalls, which must use callbacks):
ERC20 temporary approve: approve a spender for a single transaction
Why not EIP-1363 approveAndCall? That requires the spender to support a callback or an intermediary contract which is more expensive and does not work well for approving multiple tokens
KYC: call a kyc contract to verify your identity, and then call other KYC-gated functions on other contracts that also check you’ve verified your identity in that transaction. This might be used with transient storage because you only want to prove your identity for a single transaction (rather than associate your identity to the account for all time, including e.g. after loss of a private key)
NFT royalties or ERC20 fee-on-transfer: pay a fee to unlock transfers for that token ID or token amount for a single transaction. Other designs of FOT or NFT royalties break composability, e.g. fee on transfer tokens may take fees either from the recipient or the sender balance. This avoids the breakage due to transfer fees. Also allows fees to be paid by a third party or in a different token.
If you propose everything should be a callback, you get callback hell. Imagine you want to temporarily approve an ERC20 to pay some NFT royalties to accept an offer on a KYC-gated NFT marketplace. Or, imagine that you want to approve 3 different tokens and then call into a contract. Both require 3 separate callbacks. How do you order them, and how do you construct calldata for passing through the several callbacks? How do you verify callbacks all came from the right contracts? Callbacks should be avoided unless necessary, and it’s not accurate to say all these use cases are supported via callbacks in the subcall-only transient storage version.
Yeah, while initially call-scoped transient storage sounded to me like it was just as good, because you can just use callbacks, I agree now that callbacks are not a good solution and tx-scoped transient storage would enable some valuable patterns to be implemented in a much nicer way.
At the risk of not making any friends here, I’d like to voice some concerns about this from the Solidity perspective (beyond those already voiced by @chriseth above).
While it may not be a huge effort to implement this on the execution layer in clients, this change induces a severe increase in the complexity of the semantics of the EVM that I’m not sure is fully appreciated in the discussion so far. Just some points I haven’t seen discussed exhaustively so far:
No matter what we do on the language level itself, in its current specification, this will be abused for in-memory mappings that are meant to be semantically call-local, but which will nonetheless bleed through on the transaction-level, which will increase the danger of reentrency bugs.
In general (as others have commented already), I’d expect transient storage to increase the complexity of any kind of static analysis and manual code analysis / auditing significantly.
I’d also expect the induced increase in semantic complexity and the reduction of the purity of EVM calls to cause pains for formal verification, in perpetuity.
On the language level alone, we’d need to consider whether transient storage warrants a new serialization format, which would significantly increase the complexity of the language semantics, and even if not, an additional data location means increasing the testing surface by an order of magnitude (the EVM already has an unhealthy amount of distinct data locations as is). In general, proper language support for this is non-trivial.
As far as use cases are concerned, transient storage still seems like a hacky catch-all solution to me - for reentrancy locks introducing an entire new address space of data is utter overkill; passing complex data structures between calls seems like an issue to be addressed in how calldata works instead; using it as a replacement for in-memory maps is downright dangerous etc. pp.
All that being said, if this makes it to a hardfork, we will of course have no choice, but to implement support on the Solidity level - but I sincerely hope you have thought this through properly and properly consulted with all the affected parties before jumping at this - to me personally, to be honest, seeing given the state of the discussion, it seems a bit premature to be considering this a candidate for inclusion in Shanghai.
EDIT: changing “seeing” to “given” since apparently otherwise it’s assumed that I just saw this for the first time and am randomly commenting out of the blue.
) brought up TXID: