I have discussed this issue with some security companies before, and the solution is similar to @Dexaran’s. Security companies can issue audit SBT to their audited contracts. which means that each security company or person has their own corresponding SBT contract. We can verify whether the contract belongs to this company by using the url metadata of the contract and the /.well-know/contract.json file of the official company domain name
As a wallet (or an explorer), you can enumerate these contracts to display which people/companies have audited the contracts that your user is interacting with and whether they are relatively secure.
As @rmeissner mentioned, this ERC focuses on standardizing what auditors should sign, rather than defining the registry. The goal is to ensure consistent verification across the ecosystem.
I totally get where you’re coming from as I share same sentiment. That said, there’s a solid reason to standardize the data format for posting audits on-chain, and it aligns with the SEC’s recent thinking. While the SEC is primarily concerned with reliable financial information, you could argue that audits indirectly fall under this umbrella. Take a look at this document:
Notice: […] in practice the source code implemented on blockchains are in machine-readable format, may not conform to public descriptions of the code
So, creating a standardized, public, and immutable registry for audits of a given protocol makes sense. But a standard is only as good as the infrastructure built around it (gh2source2bytecode compare, documentation2implementation compare, does audit audit actual bytecode or github etc.). That’s mostly going to be off-chain though. Personally I call it “GenslerProtocol”. So yeah, this EIP does have value, especially since there’s at least one clear use case that would benefit from a robust data format.
Many people suggest going with a registry instead of the current proposed implementation. That’s precisely what we’ve been working on for the past couple of years at Trustblock.
Compared to what we’ve built, the current proposal has its strengths and weaknesses.
The main problems I see so far:
Expect protocols to add this to their codebase, which is far from trivial
Synchronicity? Auditors have to submit audits upon each request, which is very limiting in terms of usage
Handling upgrades
Findings are missing but still valuable information for protocols to act upon.
I would be interested how this problem is differently handled by the 2 approaches.
The current ERC proposes a way that could be used in any registry and without any onchain interaction necessary.
The challenge we see with registries is that it is still a very centralized approach (which has its benefits). Therefore the ERC aims to create a building block to make audit information available and verifiable onchain, not to build such registries itself.
Synchronicity? Auditors have to submit audits upon each request, which is very limiting in terms of usage
Overhead wise the ERC aims to keep it minimal. Many auditors are already signing their audits as part of the process and the additional overhead to create and sign the onchain representation should be quite low (would be interesting where you see this becoming a blocker).
When it comes to publishing the representation and pushing it onchain, then this can be done by anyone. Auditors could upload this side-by-side with their audit files (similar how checksums are published) and anyone could make use of these. This way the overhead to interact with any 3rd party is not required.
The benefit of the representation over a registry IMO is that we can have many different registries that utilize the audit representation. This has multiple benefits.
No single entity is in control over the audit representations
The user can select from different registries one that matches his needs. E.g. the user might select a registry for smart contracts that are audited only by specific auditors.
The registry can combine the representation with other criteria
I would be interested how this problem is differently handled by the 2 approaches.
In our case, we have one registry per chain, and these registries contain auditors’ wallet addresses that are allowed to publish audits. We whitelist wallets for now, but the best would be to have governance voting auditors on and off.
Therefore the ERC aims to create a building block to make audit information available and verifiable onchain, not to build such registries itself
I agree with you it’s clearly out of scope for this EIP, but I think it’d still be interesting to actually work together on an audit registry ERC that we could also integrate on Trustblock. The main reason for a registry is that it will create many more use cases for audits on-chain, according to our research through these couple of years, than if the verification has to be made synchronously with the auditor. With the data being directly accessible, protocols can be used for various composability purposes. Moreover, storing on-chain also guarantees the immutability of audits.
The challenge we see with registries is that it is still a very centralized approach (which has its benefits).
Audits are delivered by a single entity, so they will always be centralized by nature, right? However, if you meant that registries would be controlled by a single entity (like us for example), then I think it depends on the registry functioning, for example in our case only auditors can publish audits so we don’t control the data.
Overall
Per my knowledge and seeing how many auditors work, I have not seen many auditors sign their audits (not sure exactly what you meant there) nor keep track of contract addresses they have audited. So, if they want to support this, they must change how they do their business. I believe specifying exactly which contract addresses audited is the right thing, but that isn’t the reality right now
If they want to upload old reports, they must find the exact addresses they audited. When they do new ones, they have to ask for deployed contract addresses after an audit, verify them, and add them to the report
Preparing audit representation and signing. Depending on the tooling used, it can be automated or simplified. Still, it is an additional step
Display their wallet addresses or public keys so that others can verify audits
So they should have the right incentive behind it. I know it is not exactly the scope of this EIP, but that’s the practical aspect of it.
If protocols want to support this and accept only, let’s say, tokens with audits into their system, they have to:
Pre-select auditors they trust. It means that they have to get their wallet addresses/public keys to verify the signature belongs to the auditor they trust
I agree with you. The way you would implement registries is very interesting and different from the way we implemented them in our system so far.
In our case, we have one registry per chain, and protocols can preselect which authorized auditors they want to get audits’ data from.
Another significant advantage in favor of registries is that if the audits were to be stored on-chain, we could make them immutable, which is super helpful to balance trust relations further between users, auditors & protocols.
The TLDR is that solves the issue with sites such as CoinMarketCap, Etherscan, CoinGecko, etc because they don’t publish accurate data about auditors even when you follow all their forms. Hence, they are a point of centralization that filters real information. On the other hand, we think that this problem should not be limited to audits and a way to add metadata in general could be interesting.
The main problem with this ERC is that it only allows for “positive” audits that say the contract is secure.
I own a security auditing company (https://audits.callisto.network/), I’m an auditor myself and in many cases it is really important to say “Warning! This contract is not secure” while others are saying it is secure.
It is not secure to pretend “if at least one company said that the contract is secure = we label it as secure” .
Also audit records must not be immutable by any means. Imagine a contract was audited, then a vulnerability is discovered but there is already a signed audit report that states that the contract is fine. Anything can happen or be discovered AFTER the audit report so this needs to be accounted.
Just for the sake of reference: I tried (and failed) to create an on-chain registry of smart contract audits in 2008 as a part of SolidStamp service. Some of my learnings and ideas are still available at:
After all, the proposal seems to be to see if a protocol was audited.
We can achieve that more simply, in fact:
Auditors (personal or firm, whatever) create NFT to represent the audit information. Additional information like the issued date and a report link can be included in ERC721Metadata.
Maintain an address registry of auditors on-chain.
You need an agreement of the auditors on this metadata and how to issue that nft.
What you described is not the purpose of the nft the purpose is to create a basis that allows you to create such nfts. I.e. could you say that you have a nft contract where you can submit an erc-7512 object which will mint the nft with some basic information (like auditor). This then can be used in different protocol to perform security check (or other logic based on the audits).
I recently wrote a deep dive on ERC-7512 for anyone interested in a comprehensive overview of the cases for and against the proposal (thanks to everyone who contributed to the discussion thread–found many of the insights useful during my research): A Guide To ERC-7512: On-Chain Representation For Security Audits
Another possible initial step, rather than doing audits (which are complex) do DeFiSafety protocol reviews. These are fixed format, have a pass/fail and are designed for reading by the general public. This could be implemented quickly with audits brought in later. An example report is at : Liquity - detailed report | DeFiSafety
While we’re here, it seems like we might as well provide the bytecode hash as part of the onchain representation so that users of these systems can verify that the deployed code matches the code that was audited.
This could be represented as an array of structs containing the address of the contract and its corresponding bytecode hash.
Ethereum 2077’s analysis of ERC-7512 suggested extending ERC-7512 to aggregating information about other security measures (e.g., bug bounties and formal verification) vs. providing only audit data to investors doing due diligence on protocols.
The article even mentions DeFi Safety’s protocol reviews as an example of how this idea can be implemented, with the caveat that the product has switched to a subscription model vs. as being publicly accessible (which was the case circa 2022 IIRC). Here’s the relevant quote from the article’s conclusion:
ERC-7512 moves us a step closer to building trust in on-chain applications and may inspire more efforts to standardize other aspects of the security review process. For instance, given projects now increasingly adopt a “defense-in-depth” approach to protocol security—bug bounties, formal verification, audit contests, incident monitoring, and more—a system that aggregates (verifiable) information about a project’s various security measures in the same location (as opposed to this information being fragmented across multiple websites and dashboards) would do wonders for investors, users, business development (BD) teams conducting due diligence for DeFi protocols. (DeFi Safety used to have a similar service but recently switched to a revenue model.).
