Making DeFi SAFU — Secureum #3
Security Audits For Users
Losses from DeFi hacks amounted to a shocking $238M in 2020 which was ~41% in value of all blockchain-related incidents.
DeFi projects seem to increasingly rely on external security audits as a stamp of security approval typically justified by lack of sufficient in-house smart contract security expertise and accelerated time-to-market aspirations. While I believe this audit-as-a-silver-bullet approach is untenable in the long run, this post is a thought experiment towards making DeFi “SAFU” by breaking down Security Audits For Users (SAFU).
The goal of audits is to alert projects, typically before launch, of potential security-related issues which need to be addressed to improve security posture, decrease attack surface and mitigate risk. Audit is not a security warranty of “bug-free” code by any stretch of imagination but a best-effort endeavour by trained security experts operating within reasonable constraints of time, understanding, expertise and of course, decidability.
Audits are necessary (for now at least) but not sufficient. There is risk reduction but residual risk exists because of several factors. Furthermore, not all audits are equal — it greatly depends on the expertise/experience of auditors, effort invested vis-a-vis project complexity/quality and tools/processes used.
A critical point often missed is that security companies execute audits for clients who pay for their services. Engagements are therefore geared towards priorities of project owners and not project users/investors. Audits are not intended to alert potential project users of any inherent risk. That’s not their business/technical goal.
Nevertheless, the conjecture is that project users/investors may be basing their risk on mere completion of audits and incorrectly assuming no/minimal risk thereafter without bothering to, or having the expertise (understandably so) to evaluate audit report contents. But what if someone with slightly more security expertise does an audit report breakdown for the benefit of less security-savvy users as a public good (not as an endorsement/indictment or financial/investment advice)?
My hypothesis is that SAFU may help make DeFi a bit safer.
Audit reports illustrate security issues with descriptions of underlying vulnerabilities, severity/difficulty, potential exploit scenarios and recommended/resolved fixes. They also provide subjective insights into code quality, documentation and testing. The scope/depth/format of audit reports varies across auditing teams but they generally cover similar aspects. Evaluating them requires a reasonable level of expertise in smart contract security.
Most projects publish their audit reports for transparency and perhaps also to market their security due-diligence for reassuring user/investor concerns. Audit reports are also publicly shared (with client consent) by teams such as Trail of Bits (ToB), ConsenSys Diligence (also see BlockchainSecurityDB), OpenZeppelin and Quantstamp.
For this experiment, I decided to pick the latest smart contract audit published from ToB which happened to be the Origin Dollar audit and turned out to be an interesting one because it was unfortunately exploited right around the audit’s end. This post breaks down its audit reports and draws attention to key report highlights in the context of the exploit and incident response.
Context
Project Overview: Origin Dollar (OUSD), a stablecoin project from Origin Protocol, was initially launched on September 23, 2020. OUSD is an ERC-20 token backed by three stablecoins USDT, USDC and DAI to keep its value close to $1. Users mint OUSD by converting these three stablecoins which in turn are deployed to DeFi protocols such as Compound, Aave, Uniswap, Balancer, and Curve to generate yields using different strategies.
Exploit Overview: On November 17, 2020, OUSD suffered a flash loan exploit resulting from a missing validation check in one of their smart contracts that allowed a reentrancy vulnerability. This resulted in a loss of around $7M. The attack’s detailed timeline and incident response has been documented in their blog post.
ToB was finishing up their audit when the attack happened. The project had however already launched publicly in September and despite cautioning (as shown below) in their announcement about the forthcoming audit and inherent risk, users deposited funds and got exploited.
What’s impressive is the swift incident response and the apparent operational/organizational changes introduced thereafter. As explained in their post-mortem, their root-cause analysis takeaways were:
After the attack, they fixed the missing validation and reentrancy vulnerability (and other issues) as suggested by the ToB audit. They conducted another audit with Solidified, are engaging with Certora for smart contract formal verification and plan to engage with OpenZeppelin for auditing upgrades.
They integrated Slither/Echidna in their CI/CD process, check PRs more stringently for security aspects and review other DeFi hacks more carefully. They’ve introduced automated monitoring for large/failing transactions along with a fast-pause feature where two multisig holders can pause minting/redeeming to respond to any similar future incidents. They’re also working with Nexus Mutual and Cover Protocol for DeFi insurance.
They have since then relaunched with ongoing compensation and proclaimed deeper commitment to security.
Audit Overview: There have been two OUSD audits so far. The first one was with ToB which was nearing finish when the attack happened. There was a second one with Solidified after that. (Two others related to their governance Origin Token aren’t considered here.)
The ToB audit was a four person-weeks effort with two auditors completing it in two weeks from November 2-17, 2020. They reported 23 issues of which nine were High-Severity (including the one exploited) and one Medium-Severity. Rest were low/informational/undetermined severity. The overall assessment was that contracts have several unaddressed issues and were not deployment ready at audit time. Since the initial audit, the project made the recommended fixes which were further reviewed by ToB.
The Solidified audit was performed by three people (duration not mentioned) and reported (on December 17, 2020) no major issues, three minor and two informational issues. Two minor issues were since resolved and one deemed a non-issue. One informational issue was resolved and another acknowledged.
Trail of Bits Audit Breakdown
The audit report is considerably elaborate. The Executive Summary concludes with a high-level recommendation that says:
Note the red flags raised on the High-Severity & Low-Difficulty vulnerability (TOB-OUSD-010). ToB defines (Appendix A) their High-Severity category as: “Large numbers of users, very bad for client’s reputation, or serious legal or financial implications” and Low-Difficulty as: “Commonly exploited, public tools exist or can be scripted that exploit this flaw.” This dangerous combination means that an attacker can easily exploit the vulnerability to cause serious damage which was exactly what happened.
Out of the 23 vulnerabilities identified, nine were of High-Severity. Several best-practices were not followed. The overall recommendation was that contracts needed much more work on documentation, validation, security assessments and therefore weren't ready for deployment. This conclusion itself is a show-stopper in my opinion but let’s dive deeper.
Project Dashboard: The project dashboard gives an excellent overview of the assessment:
A total of 23 vulnerabilities with nine High-Severity ones and discovered within four person-weeks effort is concerning.
Note that while the dashboard (and some places later) reports only eight High-Severity vulnerabilities, the mismatch appears to be because of TOB-OUSD-019 which is classified as Undetermined-Severity in most places but High-Severity in its description. This brings up another important aspect that Undetermined-Severity (Appendix A) means: “The extent of the risk was not determined during this engagement,” which implies that the vulnerability severity could be anywhere in the range from low-high, and therefore cannot be ignored.
Furthermore, note the nine Data Validation and eight Undefined Behavior vulnerabilities. Data validation vulnerabilities result from missing sanity checks on input data that may be attacker controlled and therefore necessitate greater attention. Similarly, undefined behavior means that triggering those vulnerabilities could result in potentially malicious unexpected results and hence should not be dismissed. Some other categories such as Standards, Timing and Auditing & Logging may not be as critical. Denial-of-Service will affect availability but is unlikely to result in fund loss.
Code Maturity: The report then presents a qualitative code maturity assessment across eleven classes and rating criteria:
Note the five classes with Weak rating (Appendix B: The component led to multiple issues; more issues might be present) and four classes with Moderate rating (The component had some issues) which seems concerning. The ratings are justified by corresponding vulnerabilities discovered. Classes of Front-Running and Key Management weren’t investigated.
Existence of privileged roles, inconsistent SafeMath, lack of validation for untrusted data, standards non-conformance, lack of extensive documentation, undocumented incident response plan and reentrancy issues are big red flags raised.
The presence of a reentrancy vulnerability is a lowlight in my opinion. Reentrancy has been well documented since the 2016 DAO hack. Many tools including ToB’s Slither warn about reentrancy. They can be investigated and fixed (using Checks-Effects-Interactions pattern or reentrancy guards) or ruled out (by confirming trusted contracts).
Coverage: The engagement covered the token, vault, strategies, oracles (Chainlink/Uniswap) and governance. Curve strategy and off-chain components were outside the scope.
Verification: Fuzzing tool Echidna was used to verify that OUSD followed ERC20 properties but it failed five of the seven tests. These failures verified two High-Severity vulnerabilities: TOB-OUSD-002 and TOB-OUSD-023.
Recommendations: A list of short-term and longer-term recommendations are presented. Seven of the 23 vulnerabilities (three High-Severity including the one exploited) could have been detected with ToB's Slither tool. A monitoring system could be useful in tracking critical transactions/events on all dependent contracts to alert any anomalous behavior.
Findings Summary & Details: Details are presented for all reported vulnerabilities highlighting their type/severity/difficulty, code segments, vulnerability descriptions, exploit scenarios and recommendations.
An observation here is that out of the nine High-Severity issues, two others besides the one exploited are also Low-Difficulty: (1) TOB-OUSD-008 may lead to undefined behavior (potentially malicious) because proposal transactions can be executed separately due to missing access control checks (2) TOB-OUSD-019 may lead to undefined behavior because of several function calls that lack return value checks leading to unexpected results and undermining correct system functioning.
Fix Log: This (Appendix E) indicates that all except three of the 23 reported vulnerabilities have been fixed. One Informational-Severity issue remains unfixed while two High-Severity Data-Validation vulnerabilities — one that may allow DoS and another that breaks a common EIP-20 invariant — have been partially fixed. This is good news but ideally should have happened before project launch.
Solidified Audit Breakdown
This report is relatively lightweight compared to ToB's. The brevity might have to do with the duration (not specified) and that this audit happened after ToB.
Tables highlighting code maturity and issues do not raise any red flags:
One observation is that a sixth informational note: “Staking contract may run out of funds” at the end, which seems important, appears to have been skipped in earlier summaries.
Furthermore, given that this audit happened after ToB, it would have helped to independently verify ToB's findings and fixes, which doesn’t appear to be the case.
Conclusion
Security hindsight, after an exploit, is certainly 20/20 and regretful. In my opinion, even without considering the exploited vulnerability, the audited project (before fixes) may have been high-risk for users when ToB’s report clearly says that the assessed contracts were not security-ready for deployment. The number/nature of reported vulnerabilities arguably justify that recommendation.
This was coincidentally an easy project to consider because ToB’s audit report unequivocally recommends fixing the many security issues before deployment and the project was also unfortunately exploited to justify the audit’s recommendation. My guess is that projects would generally be much harder to objectively evaluate based on their audit reports alone. We may have to consider other approaches when audits do not say much or don’t exist.
Nevertheless, breaking down DeFi Security Audits For Users (SAFU) may still be worthwhile as a public good but certainly should not be interpreted as an endorsement/indictment of projects/auditors or as financial/investment advise. DeFi SAFU should be a collective community responsibility.
I hope you found this somewhat useful. Thanks for reading and look forward to your comments and feedback.