Following weeks of upward momentum, during which time the price of its coin multiplied more than five times in value, Solana hit a serious roadblock last week when its blockchain network went offline for more than 17 hours starting on September 14. Today, the Solana Foundation issued an initial postmortem of the event, and cast the root cause in a new light.
Most of the information in the initial overview report confirms what was tweeted last week, either through Solana’s official Twitter accounts or that of Solana Labs CEO Anatoly Yakavenko. In essence, the network was overwhelmed by a flood of incoming transactions, which the Foundation had previously pegged at up to 400,000 per second.
The flood started upon the launch of an initial decentralized exchange (DEX) offering, or IDO, that was hosted on the Solana-based decentralized finance (DeFi) protocol, Raydium. An IDO is a type of token launch that takes place on a DEX, rather than an initial coin offering (ICO) on a centralized exchange.
In this case, the IDO was for Grape Protocol, a widely-used Solana toolset for DeFi developers. The network was overwhelmed by transactions generated by bots, which were apparently trying to muscle their way to success in the token offering.
The transactions overwhelmed the Solana network’s distributed nodes, causing some to crash due to the amount of memory being used. The network stopped producing blocks when it could not ascertain consensus over the blockchain’s current status. Last week, Solana initially pegged “resource exhaustion” as the cause of the downtime.
While today’s report still lines up with last week’s information, Solana’s updated language is more pointed: “The cause of the network stall was, in effect, a denial of service attack,” the report reads.
While not a traditional DDOS attack, or distributed denial-of-service attack—the type that can knock out websites and other servers—Solana’s statement suggests that the end result was similar.
Ultimately, the network’s validators opted to update and restart the network, creating a hard fork from the last confirmed block. It took 14 hours, per Solana’s report, to coordinate the restart effort across the required 80% of validators, update the nodes, and get the network up and running again.
“Thank you to the validator community, engineers, and the whole Solana ecosystem for coming together to fix this problem,” the report reads. “On the rare occurrence that issues like this happen, it’s disruptive to everyone—and when you need to fix something on a decentralized network, it’s a true community project.”
The Solana Foundation plans to release a more robust technical postmortem in the coming weeks with help from the community.
Although the wider cryptocurrency market is down significantly today, dropping 10% in total market cap, Solana (SOL) has seen a larger price drop than many of its contemporaries. Right now, the Ethereum rival is down 18% over the last seven days at a current price of $140, per Nomics—a steep drop from the all-time high above $213 set earlier this month.