Daniel Von Fange
On August 31st, 2021, an attacker stole 19.6 million dollars worth of ETH and AMP from Cream lending pools. This attack was notable because it was a successful reentrancy attack against contracts that did have reentrancy protection.
The very first big hack on an Ethereum smart contract was a reentrancy attack. Reentrancy attacks are THE classic Ethereum smart contract vulnerability. Consider a sample bank smart contract with this withdrawn method:
withdraw(address user, uint256 amount){
uint256 oldBalance = balances[user];
// Check
require(oldBalance >= amount);
// Calculate
uint256 newBalance = oldBalance - amount;
// Transfer funds
coin.transfer(user, amount);
// Store
balances[user] = newBalance;
}If an attacker was able to run their own code during the coin transfer in this withdraw, then the attacker could withdraw more than they had deposited.
Let’s see how this works. The attacker starts with $1,000 deposited, and requests a withdrawal of $1,000. The withdrawal function reads in the user’s balance, checks that the user has funds, and sets a temporary variable newBalance to $0. This “newBalance” variable is not a write to storage, rather it is only in memory during this execution of the withdraw function.
When the $1,000 is transferred to the attacker, the attacker gets to run code at this point in the middle of the withdraw function. Immediately, the attacker requests another withdrawal for $1,000. Because the storage keeping track of the user’s balance has not yet been updated, the withdrawal functions loads in the user’s balance as still $1,000 and permits another withdrawal to begin.
After the second transfer in the second withdraw is complete, then the newBalance in each running copy of the withdrawal function are stored. In each case, the newBalance being stored would be zero. The attacker has doubled their money, and could repeat the attack.
A core part of the AMP ERC20 token implementation is that during each transfer of AMP from one user to another, the token can run external code chosen by the receiver. This is very unusual, and allows code to be run in the middle of other contracts being executed. This is a classic opening for a reentrancy attack.
CREAM is a set of over-collateralized lending pools, like Compound and AAVE. A user deposits an amount of a crypto token, and is then able to borrow a smaller dollar value of a different crypto token.
A core part of the protocol is that a user can never borrow a greater amount than the user's collateral permits. Before making a loan, the CREAM codebase scans through each lending pool in the system to verify how much the user has borrowed from each and lent to each.
The CREAM borrow function looks like this (focusing just on the user balance and the transfer):
function borrowFresh(address payable borrower, uint borrowAmount) internal returns (uint) {
// ... snip ...
vars.accountBorrows = borrowBalanceStoredInternal(borrower);
vars.accountBorrowsNew = add_(vars.accountBorrows, borrowAmount);
// ... snip ...
doTransferOut(borrower, borrowAmount);
/* We write the previously calculated values into storage */
accountBorrows[borrower].principal = vars.accountBorrowsNew;
// ... snip ...
}It looks exactly like a textbook reentrancy example. So why wasn’t it hacked long before?
There are three common reentrancy defenses that smart contracts use.
If we jump back to our example, but move just one line of code, we can prevent a reentrancy attack:
withdraw(address user, uint256 amount){
// Check
uint256 oldBalance = balances[user];
require(oldBalance > amount);
// Calculate
uint256 newBalance = oldBalance - amount;
// Store
balances[user] = newBalance; // MOVED to before transfer
// Transfer funds
coin.transfer(user, amount);}If the call to an external function is after everything else in the function, then the attacker gains no advantage by calling the code from inside the transfer. This simple method of preventing attacks was learned painfully in the early days of Ethereum, and this defense is in the official Solidity documentation.
But sometimes you need to make multiple external calls, or there are unavoidable calculations that must be done after external calls. In such a case, the usual solution is to use a single contract storage slot as a lock. Whenever any state changing function in the contract is called, the lock storage is checked to see if other code is executing. If it is locked, the function will revert. If it is unlocked, then the function will write a locked value to the storage and continue normal execution. After running the code, the last step in the function will unlock the lock. By doing so, an attacker cannot executed any code in the same contract at the same time.
NonReentrant locks are incredibly common and are essentially standard for smart contracts these days. It's a simple tool that prevents a tremendous amount of mischief.
Lastly, a common method is to have a whitelist of trusted coins or contracts that your own contract is allowed to interact with. As long as these external contract are not attacking you themselves, and not calling out to external contracts themselves (in methods you use) this can be safe.
But it's very easy to get wrong. Either by forgetting to check somewhere inside your code that a coin is in your list before calling it, or by adding a trusted coin without checking it for reentrancy possibilities. It's also possible that a coin that was safe before could potentially be upgraded later to unsafe behavior.
As we already saw, the order of the CREAM borrow function was vulnerable to reentrancy. Furthermore, a coin that allowed an attackerto execute code during a transfer was added to the list of tokens supported by CREAM.
However, the CREAM lending pool contracts did have reentrancy locks on their contract functions. How did the contracts get hacked when they had protection here?
The CREAM lending system is made up of multiple contracts with one "CToken" pool contract for each asset supported. So CREAM has a lending contract for ETH, as it does for USDC, as it does for AMP, etc. Each one of these individual lending contracts has its own separate lock for protecting that individual contract against reentrancy.
However, the system as a whole is not protected because an attacker could be inside different lending pool contracts simultaneously.
In each round of this attack, the attacker put up one and a half million dollars of collateral, then borrowed AMP. During the borrow function for in the AMP pool contract, the AMP coin called the attackers code, allowing the attacker to start a second borrow of ETH. Because the AMP borrow had not written any record of the AMP borrow to storage yet, when the ETH borrow in the ETH pool contract checked with all pool contracts to see how much the attacker had already borrowed, it saw that the attacker had no debts and lots of collateral, and thus allowed a duplicate ETH borrow.
With the ability to get two borrows for a single set of colateral, it was game over. The Cream Post Mortem has an excellent writeup of the rest of the attack, beyond the core vulnerability.
- The Slither automated tool will check individual functions for state writes after external calls.
- Contract level reentrancy protection should be used - unless you have an extremely simple contract that can be proven not to need it.
- If you have an interlocking set of contracts, you might need either a global level of locking, or a way to funnel incoming transactions through a single contract.
- As DeFi gets more complicated, there may be reentrancy vulnerabilities found involving contracts from multiple parties, each individually protected, but systematically weak.