You Were Warned

Four weeks ago yesterday I posted The Things Are Winning about the IoT-based botnet attack on Krebs On Security. I wrote:

And don't think that knocking out important individual Web sites like KrebsOnSecurity is the limit of the bad guys capabilities. Everyone seems to believe that the current probing of the root servers' defenses is the work of China but, as the Moon Worm showed, careful preparation isn't necessarily a sign of a state actor. There are many bad guys out there who could take the Internet down; the only reason they don't is not to kill the goose that lays the golden eggs.

Last Friday's similar attack on Dyn, a major US DNS provider, caused many of its major customer websites to be inaccessible, including Twitter, Amazon, Tumblr, Reddit, Spotify, Netflix, PayPal and github. Dyn's DNS infrastructure was so overloaded that requests for name-to-IP-address translations were dropped or timed out. The LOCKSS team uses github, so we were affected.

It is important to note that these attacks are far from the largest we can expect, and that it is extraordinarily difficult to obtain reliable evidence as to who is responsible. Attackers will be able to produce effects far more disruptive than a temporary inability to tweet with impunity. Below the fold some commentary and useful links.

Although it appears that Mirai was not the only botnet technology involved in the Dyn DDoS, it is potentially far more powerful than we have seen so far:

One online tracker of Mirai suggests there at least 1.2m Mirai-infected devices on the internet, with at least 173,000 active in the past 24 hours.

In A New Era of Internet Attacks Powered by Everyday Devices David E. Sanger and Nicole Perlroth of the New York Times write:

the problem is quickly expanding: Cisco estimates that the number of such devices could reach 50 billion by 2020, from 15 billion today. Intel puts the number at roughly 200 billion devices in the same time frame.

So perhaps about 10-15% of 0.001%, or less than 0.0002% of the IoT has been mobilized, and it can generate terabit/sec DDoS. Mirai is a very crude tool according to a security researcher:

I am just surprised at how such a trivial attack code could be responsible for such a large DDoS. It really says a lot more about the state of IoT security than the specifics of the malware, ... If people still aren't changing default passwords and disabling telnet on Internet connected equipment in 2016 then we are heading to a future with more incidents like this happening.

The idea that getting people to change their IoT device passwords will fix the problem is laughable. Even in the unlikely event that you could get 99% of all IoT devices to use hard-to-guess passwords, you could still face attacks 5,000 times bigger than the current ones, maybe 5 exabit/sec.

Passwords can be changed, mitigating the vulnerability. But only a small proportion of IoT devices can have their software updated. Most IoT devices have known vulnerabilities, and all have unknown vulnerabilities (zero-days). Without the ability to update the software, they will be a threat as long as they are connected. How feasible would it be to get 99% of the un-patchable IoT devices disconnected? Even then the DDoS problem would not be fixed.

A more sophisticated tool than Mirai that used known vulnerabilities (such as the 12-year-old SSH bug) could create a botnet with say 20% of the IoT, a 100 exabit/sec DDoS capability. With the Shodan search engine, the source for Mirai and a set of known vulnerabilities, this is within the capability of ordinarily competent programmers. It could almost certainly take the entire Internet down.

Four years from now a similar fraction might generate between 300 (Cisco) and 1,300 (Intel) exabit/sec. It isn't realistic to expect an effective, widely-deployed solution to the IoT security problem in the next four years.

It is possible for ISPs to mitigate these attacks. Brett Glass reports:

We blocked incoming attacks by the Mirai worm ..., monitored our network for vulnerable camera systems that were attempting to participate in it (there was only one -- a cheap, Chinese DVR rebranded and resold by a company in New Jersey to one of our rural customers), and set up a honeypot to capture the code.

But most of them won't, because there's no economic incentive to be this careful and responsible. Just as many ISPs have failed to implement the basic mitigation of IP address spoofing, BCP38 filtering:

It costs money to install filters, albeit a very small amount, but it is not free. Nor is the labour capable of installing those filters cheap. Therefore it makes economic sense for this network operator to not install filters. No one is DDOSing their network, that’s someone else’s problem. This network operator can save money by not installing filters, and realize none of the loss associated with DDOS attacks.

Sanger and Perlroth write:

It is possible, investigators say, that the attack on Dyn was conducted by a criminal group that wanted to extort the company. Or it could have been done by “hacktivists.” Or a foreign power that wanted to remind the United States of its vulnerability.

It isn't just extraordinarily difficult to defend against attacks on this scale, it is also extraordinarily difficult to identify the perpetrators with sufficient certainty to exact punishment. Brian Kreb's Spreading the DDoS Disease and Selling the Cure is a fascinating account of his accumulating of circumstantial evidence that a Mr. Sculti (a student at Clemson) and a Mr. Wu (a student at UCSD) appear to be involved with Mirai:

Are either Mr. Wu or Mr. Sculti behind the Mirai botnet attacks? I cannot say. But I’d be willing to bet money that one or both of them knows who is. In any case, it would appear that both men may have hit upon a very lucrative business model.

The lucrative business model is, according to Krebs:

selling DDoS protection against the very DDoS-for-hire services he is courting with his domain registration service.

Just one of their customers was vDOS:

vDOS — a “booter” service that has earned in excess of $600,000 over the past two years helping customers coordinate more than 150,000 so-called distributed denial-of-service (DDoS) attacks

In this case the perpetrators were identified after their site was "massively hacked":

Two Israelis allegedly behind vDOS, both 18, were arrested after an FBI investigation. The site had been operating for four years. vDOS offered four retail tiers: from a $19.99 “bronze” plan to a $199/month “VIP plan”.

vDOS had excellent customer service. This is similar to the ransomware market:

Cerber is 2016's biggest name in ransomware. ... Cerber didn't get to the top just by being good at infecting computers, locking up people's files and blackmailing its victims for Bitcoin. The plucky ransomware is on the fast track to fame and fortune thanks to a hard-won reputation for top-notch customer service that wows its victims at every turn. At least that was the conclusion in security company F-Secure's summer report, Evaluating the Customer Journey of Crypto-Ransomware.

UK banks have given up and, as Jamie Doward at The Guardian reports, City banks plan to hoard bitcoins to help them pay cyber ransoms because:

the scale and ferocity of the attacks meant some banks were coming round to the view that it was cheaper to pay off the criminals than risk an attack. ... “From a purely pragmatic perspective, financial institutions are now exploring the need to maintain stocks of bitcoin in the unfortunate event that they themselves become the target of a high-intensity attack, when law enforcement perhaps might not be able to assist them at the speed with which they need to put themselves back in business.” ... The cost to businesses of an attack can far outweigh paying off the blackmailers: telecoms provider TalkTalk lost 101,000 customers and suffered costs of £60m as a result of a cyber attack last year.

Iain Thomson at The Register reports on a parallel example:

A study by the RAND Corporation, published in the Journal of Cybersecurity, looked at the frequency and cost of IT security failures in US businesses and found that the cost of a break-in is much lower than thought – typically around $200,000 per case. With top-shelf security systems costing a lot more than that, not beefing up security looks in some ways like a smart business decision.

Romanosky analyzed 12,000 incident reports and found that typically they only account for 0.4 per cent of a company's annual revenues. That compares to billing fraud, which averages at 5 per cent, or retail shrinkage (ie, shoplifting and insider theft), which accounts for 1.3 per cent of revenues.

So, as with patent trolls, short-term thinking turns the bad guys from a nuisance into a massively profitable business. How soon before hired lobbyists in Washington are watering down legislation to improve IoT security?

Suzanne Woolf on Dave Farber's IP list gets to the heart of the problem:

The entire infrastructure is at risk, from DNS providers through ISPs and CDNs and your favorite "cloud provider", because small, cheap, un-maintainable, almost unnoticeably low-profile devices are enormously easier to add to the network than they are to fix, and enormously easier to mobilize against others than to defend against.

It's a classic problem of asymmetric resource use, and the advantage right now is with the attacker.

Sanger and Perlroth conclude:

The answer may not come by Election Day, but the next wave of attacks very well could.

I would not be at all surprised if, on Nov 8th, a much larger DDoS were launched against US network infrastructure. If the recent
attackers were script kiddies, election day would be an attention-grabbing target. If they were DDoS for hire vendors like those behind the Moon Worm, or (allegedly) Messrs Wu and Sculti, it would be the best possible advert. If they were the Russians, it would cap their campaign of support for Trump by feeding his paranoia about rigging. Imagine Election Day with no Twitter, no New York Times, no CNN, no e-mail. It would be an extremely disruptive event.