Tuesday, August 16, 2016

OK, I'm really amazed

Ever since I read Maciej Cegłowski's What Happens Next Will Amaze You (its a must-read) I've been noticing how unpleasant the experience of browsing the Web has become. Ever since I read Georgis Kontaxis and Monica Chew's Tracking Protection in Firefox for Privacy and Performance I've been noticing how slow browsing the Web has become.

Because I work at Stanford I have a discounted subscription to the New York Times, so I'm that rarity on the Web, a paying customer. You would think they would try to make my Web browsing experience pleasant and hassle-free. So here I am, using my hotel's WiFi and Chrome on my totally up-to-date Google Nexus 9 tablet with no ad-blocker. I'm scrolling down the front page of the New York Times and I notice a story that looks interesting. My finger touches the link. And what happens next amazes me. In fact, it tips me over the edge into full-on rant mode, which starts below the fold. You have been warned, and I apologize for two rants in a row.

So, why do I pay the New York Times $7.50/month? Two reasons:
  • First, in any society it is useful to know what the governing elite wants you to think. That's why Soviet citizens read Pravda. Its why Chinese citizens read the People's Daily.  It's one reason I read the New York Times. For example, the governing elite wants you to believe the too-big-to-fail banks weren't to blame for the 2008 financial crash. Of course, they were to blame, but its useful to know you're not supposed to believe that.
  • Second, as well as the instruction in right thinking, the paper publishes work by reporters that I usually enjoy reading. For example John Markoff, or Gretchen Morgenstern.
Both of these reasons require that I read the articles. They do not require that I view, much less click on, the ads. Whatever their value to future scholars, to me every byte transferred by the ads, every pixel consumed by the ads, every second I spend waiting for the ads to load, is value subtracted. I'm not paying my $7.50/month for the ads, I'm paying it for the articles.

With that preamble out of the way, lets see what happened when I followed the link to the story:
  • The first thing that happened was, well, nothing. There was a brief pause until:
  • The URL in the headline bar changed from http://nytimes.com/ to the URL of the story, but I'm not seeing the story, I'm still seeing the content of the front page.
  • Then the top of the story became visible in the bottom 2/3 of the window with a large blank space above it. I started to read the article.
  • I got to the end of the text that was visible in the window, and scrolled down.
  • I continued to read for a short time, until:
  • Without me doing anything, the page jump-scrolled back to the top, so I saw the text I'd already read with an advert where the blank space had been.
  • So to continue reading, I scrolled back to where I had been. The page started fighting me. As soon as I lifted my finger, the page jump-scrolled back to the top.
  • I tried to scroll back again, but now the page was immovable. No matter what I did, the page had stopped scrolling. All I could see was the section of the page that was visible initially. And the ad.
The amazing thing is that I was never able to read the whole page. The advert was so insistent that my viewing it was more important than my reading the article that I was never able to read it.

I'm paying the New York Times so I can read the articles but their ad system is preventing me getting the content I HAVE PAID FOR!


Sigh!  Rant over.

This actually happened back in April while I was at the IIPC's General Assembly. I wrote the rant above, put it aside to add more information, then forgot about it. I tried re-visiting the page, but of course the ad system fed me a different, less obnoxious ad, so the experience wasn't repeatable. But last week I had a similar experience, when an ad on the New York Times grabbed the page I was reading away from me so I wasn't able to finish reading it. I had to kill and restart the browser to regain control. So I was reminded of this unfinished post.

Ad ecosystem
Lets assume for the purposes of argument that the New York Times is a trustworthy site whose Web developers test their code carefully and would never do anything to degrade the experience of their paying customers. And the New York Times has impenetrable security so could never get compromised by bad guys who would thereby get the ability to run JavaScript in customers' browsers.

The problem is that the trustworthy site sells the ability to run JavaScript in your browser to all sorts of entities that are much less trustworthy via a complex ecosystem infested with fraud that Cegłowski describes thus:
Today we live in a Blade Runner world, with ad robots posing as people, and Deckard-like figures trying to expose them by digging ever deeper into our browsers, implementing Voight-Kampff machines in Javascript to decide who is human. ...

The ad networks' name for this robotic deception is 'ad fraud' or 'click fraud'. ...

Ad fraud works because the market for ads is so highly automated. Like algorithmic trading, decisions happen in fractions of a second, and matchmaking between publishers and advertisers is outside human control. It's a confusing world of demand side platforms, supply-side platforms, retargeting, pre-targeting, behavioral modeling, real-time bidding, ad exchanges, ad agency trading desks and a thousand other bits of jargon.
No-one in this ecosystem, except the New York Times, cares about the reader's experience. If they think about depriving the reader of the ability to read the content they paid for by forcing an ad on them at all, they probably think its a great idea.

Because the system is so automated, and because there's no way for the reader to complain to the New York Times about an ad, especially because the reader no longer has control of their browser, and even if there were the problem is unrepeatable, the New York Times probably never even finds out that someone paid them to degrade the reader's experience. They are happily ignorant of why their customers are up to here with paying for this kind of miserable user experience.

Of course, I was lucky. The JavaScript I got was obnoxious, but wasn't actively hostile. Others weren't so lucky. Among the "trustworthy" sites found serving the Angler malware were:
Publisher Traffic (monthly)*
msn.com 1.3B
nytimes.com 313.1M
bbc.com 290.6M
aol.com 218.6M
my.xfinity.com 102.8M
nfl.com 60.7M
realtor.com 51.1M
theweathernetwork.com 43M
thehill.com 31.4M
newsweek.com 9.9M

* Numbers pulled from SimilarWeb.com.
It turned out that this was only one of three Angler malvertising campaigns. Similar attacks are routine. Many of them happen when systems in the complex ad ecosystem are compromised. So the trustworthiness of the New York Times as a place to visit depends on the trustworthiness and the security systems of everyone in the ad ecosystem. The New York Times probably doesn't know who the entities in the system they deal with are themselves dealing with, and who those entities are dealing with, and so on. And it certainly can't audit all their security systems to see whether they match the New York Times' (assumed to be) impenetrable security.

As John Oliver points out, news organizations are in big trouble because:
One news industry veteran, who has long since ditched newspapers for radio, once summed up the situation perfectly for me. "It's not that newspapers don't make money. They just don't make enough money for their greedy owners."

Historically, newspapers have had sky-high profit margins. At their peak in the late 90's, seven publicly traded media companies saw profit margins at nearly 30%. The now-defunct American Journalism Review even reported in October 1994 that the Warren Buffett-owned Buffalo News had a 34.6% profit margin.
They are in such desperate pursuit of these long-gone margins that they have sold off control of their user experience to the highest bidder. They wonder why people don't trust them and run ad-blockers. They refuse to supply content to those who do.
Trevor Pott on The Register's Sysadmin Blog sums it up:
I'm not turning off my browser's defences. It isn't going to happen. I've been bitten one too many times by malvertising and other web nasties and those shields are staying very, very up. ... Ad blockers aren't just for the lazy and the selfish. They are part of browser defence and are absolutely mandatory on today's web. There's no getting around it: advertising has been a vector for malware for far too long. The trust is comprehensively lost and it isn't coming back.
Publishers are caught between a rock and a hard place. Advertisers often demand that you use the ad networks they are familiar with. The ad networks see no percentage in ensuring clean ads. The publishers have little to no control. If they want to stay in business they need to advertise, and they need to use the advertising networks the advertisers demand.
This will not end well, as the arms race between Facebook and AdBlock Plus shows. First, as Casey Johnston writes at the New Yorker, if Facebook succeeds it is another nail in the coffin of the open Web:
To understand Facebook’s motivation, it helps to revisit Apple’s mobile ad blockers, which the company introduced last year. Apple’s mobile ad blockers, which have earned a lot of attention, work only on Safari’s mobile browser. But Apple’s entire mobile platform is built on closed-system apps, not Web-based sites. Ads served within these apps cannot be blocked by anything, not even Apple’s own ad blockers. So, by blocking Web-based ads, Apple’s move would, in theory, drive businesses away from the open mobile Web, where the future of advertising is shaky, and toward apps, where there are fewer business variables. It’s not a coincidence, either, that Apple makes money from apps through both sales and advertising, while it earns nothing from its mobile browser.

What was true of Apple then is even more true of Facebook now. Over the last few years, Facebook has seen rapid growth within its mobile app, where blockers don’t work and it can easily control the ad experience in the same way that Apple can control the app experience.
Second, as Cory Doctorow writes, if AdBlock succeeds they get to censor the Web:
If online services successfully make ads and content indistinguishable to attempts to block them, content becomes the only target to work on

A blunter tool than ad blocking as it mostly works now (if you let it block 𝓫𝓻𝓪𝓷𝓭, you'd never see your friends talking about 𝓫𝓻𝓪𝓷𝓭), this has far greater potential for censorship and other unpleasantness, once a successful provider ends up in a middle-man position akin to the one Ad Block currently has.

Imagine how easy it would be for Ad Block 2020 to influence an election if it had evolved to understand the political meaning of any given block of content—having long ago established the legitimacy of this approach by backing user-end controls in Facebook's war on them.
Yet again, it is lesser-of-two-evils time.


David. said...

In pursuit of their traditional margins, many newspapes erected paywalls. At the Financial Times, the Wall Street Journal, and the New York Times they have, despite becoming somewhat porous, been relatively successful. After all, the New York Times persuaded me to cough up.

But Mike Masnick at Techdirt points outh that Lots Of Newspapers Discovering That Paywalls Don't Work:

"Not surprisingly, more and more newspapers that bet on paywalls are discovering that they don't really work that well and were a waste of time and effort -- and may have driven away even more readers."

Full disclosure, I also pay for The Guardian (which doesn't have a paywall) and support a number of blogs.

PS - Any time you write "in pursuit of their traditional margins" you're probably writing about an industry that will never get them back.

David. said...

Today's malvertising story comes from Kaspersky Labs:

"By simply viewing their favorite news sites over their morning coffee users can end up downloading last-browser-update.apk, a banking Trojan detected by Kaspersky Lab solutions as Trojan-Banker.AndroidOS.Svpeng.q. There you are, minding your own business, reading the news and BOOM! – no additional clicks or following links required. And be careful – it’s still out there!

It turns out the malicious program is downloaded via the Google AdSense advertising network."

David. said...

Doc Searls has a great rant about how counter-productive it is for companies, especially Amazon, to bombard you with ads for the product you just purchased.

David. said...


"Hackers thought to be working for Russian intelligence have carried out a series of cyber breaches targeting reporters at the New York Times and other US news organizations, according to US officials briefed on the matter."

David. said...

Via Yves Smith at naked capitalism, a look at the declining revenues of newspapers, and the effects of the decline:

"a reduction in advertising revenues will reduce the quality of newspapers. Ultimately, this may result in a less well-informed public.

The year 2015 was perhaps the worst for the newspaper industry since the recession. According to the Pew Research Center (2016), in the US total advertising revenues (print and digital) among publicly traded companies declined by nearly 8%."

David. said...

Doc Searl's It’s People vs. Advertising, not Publishers vs. Adblockers makes a good point:

"nearly all press coverage of what’s going on here defaults to “(name of publisher or website here) vs. ad blockers.”

This misdirects attention away from what is actually going on: people making choices in the open market to protect themselves from intrusions they do not want.

Ad blocking and tracking protection are effects, not causes. Blame for them should not go to the people protecting themselves, or to those providing them with means for protection, but to the sources and agents of harm."

David. said...

At Techdirt, Leigh Beadon sums up the state of Internet advertising with Traffic Is Fake, Audience Numbers Are Garbage, And Nobody Knows How Many People See Anything. The headline sums it up.

On Digby's awesome blog, the equally awesome Spocko extends this to Twitter with On Debate Night Your Tweet Matters. A sample:

"But you know whose tweets will get picked up? The millions of TwitterBots controlled by a handful of people. They know how the keywords, counting and sorting tools of the media work. This means that when the media talks about "the public's reaction" It's not totally the public's reaction. This is a problem. How big is it?

One estimate from Twitter Audit is that 1 out of every 4 followers of Big Orange Hair are fake. And yes, both sides do it, Hillary Clinton has the same percentage of fake twitter followers as her opponent. You can be outraged or see it as "Bot Parity" for those accounts."

David. said...

Spotify joins the ranks of major malvertising sites.