Smart contract blockchain Solana nearly froze Wednesday, Marinade Finance says

A routing failure at a major data center provider briefly knocked nearly 29% of Solana’s staked tokens offline, bringing the network close to a full halt, according to staking platform Marinade.

Because Solana stops finalizing transactions if more than one-third of staked tokens go offline, the incident left the network within about 20 million tokens of a freeze similar to a five-hour outage in February 2024.

The glitch, traced to a bad internet route originating at Teraswitch’s Miami facility and spreading to data centers in Europe and Asia, exposed the risk of relying on a single connectivity provider that controlled more than a quarter of all staked tokens.

Solana nearly froze on Wednesday morning when a routing glitch at a major data center provider knocked almost 29% of the network’s staked tokens offline, staking platform Marinade said. Finality, the point at which blockchain transactions become irreversible, stops if more than a third of the coins staked go dark. Staking refers to act of locking coins in a blockchain network to secure it in return for rewards.

Marinade said that the network came within about 20 million tokens of that threshold. Roughly 90 validators were hit and together lost 333 SOL in rewards, a relatively small sum that will be covered by “validator bonds.” “If delinquency had gone past a third, nothing finalizes for anyone holding SOL anywhere, and there's no bond for that.

The February 2024 halt took about five hours to restart,” Marinade said in an explainer post. Solana is one of the leading smart contract blockchains, with assets worth $4.3 billion locked in DeFi protocols operating on the network. The blockchain has built a reputation as a faster and cheaper alternative to industry leader Ethereum, but has faced several outages in the past.

The latest issue started with a bad internet route from Teraswitch’s Miami facility that then spread to data centers across Europe and Asia, cutting off validators in London, Amsterdam, Frankfurt, Singapore and Tokyo. North America stayed online. The company fixed the issue in about 10 minutes, and traffic was flowing again by 4:16 a.m.

UTC. One single network operator, identified as AS2032, controlled more than a quarter of all the tokens people had locked up to secure the network, which was more than the Solana-prescribed safety limit. Almost all of those tokens went offline at the same time. Other companies lost another 14 million tokens in the same short period.

Most of the affected validators, including the big one called Helius, stayed offline for the full 33 minutes because their backup systems never switched on. This whole event is a clear warning: if more than one-third of the network’s tokens ever go offline at once, the entire blockchain freezes for every single person holding SOL, and there is no quick way to fix the bigger damage that would follow.

Disclosure & Polices: CoinDesk is an award-winning media outlet that covers the cryptocurrency industry. Its journalists abide by a strict set of editorial policies. CoinDesk has adopted a set of principles aimed at ensuring the integrity, editorial independence and freedom from bias of its publications.

CoinDesk is part of Bullish (NYSE:BLSH), an institutionally focused global digital asset platform that provides market infrastructure and information services. Bullish owns and invests in digital asset businesses and digital assets and CoinDesk employees, including journalists, may receive Bullish equity-based compensation.