The URLTeam is the ArchiveTeam subcommittee on URL shorteners. We believe that they pose a serious threat to the internet's integrity. If one of them dies, gets hacked or sells out, millions of links will stop working.In a fascinating keynote at Digital Preservation 2013, Hilary Mason dropped a hint to ask her after the talk about Bit.ly's Plan Z. So we did. It turns out that Plan Z is Bit.ly doing the right thing, for themselves and to some extent for the world.
Every time Bit.ly shortens a URL, they write a static redirect from the short to the long URL into Amazon's S3. Thus, if their service ever goes down, as for example it did when someone unplugged their data center, all they need need to do is to update the DNS record for bit.ly to point to S3, and the short URLs continue to resolve as they used to.
This is fine for Bit.ly itself and it is a big step towards being fine for the world. Of course, if Bit.ly were to go out of business, they would stop paying the S3 charges and for the bit.ly domain name, and their Plan Z wouldn't help the Web survive intact.
Archive Team has a group called Urlteam that works to back up URL shortening services, compiling a map from short to long URL for each and exposing the result as a torrent that can be downloaded and preserved. This is great, as it gets the data out of the custody of the service but, again, it doesn't on its own solve the problem of keeping the links intact after the service implodes.
If we can get it deployed, Memento is the missing piece of the solution. It is designed to ensure that links resolve even after the original target has gone away. As far as Memento is concerned, a shortened URL is no different from any other URL. If, after the URL shortening service dies, one or more of the backups can provide a site like Plan Z's that exports the static redirects and supports Memento then browsers that support Memento will continue to see the shortened URLs resolve as they originally did.
It does solve the problem on its own. With the data in secure places outside of bit.ly, it's trivial to make a website at a nonprofit like archive.org or at another location, and aim DNS at it for bit.ly so the redirections work. We've done some experimentation with this for urlteam and will continue to.
ReplyDeleteI'm all for more solutions, but we've got one. H's solution is short-sighted. I trust nothing about a company like bit.ly to follow through on this promise, and she left bit.ly 3 days later! Who's in charge of plan Z now?
Correct me if I'm wrong, Jason, but aren't you assuming that after bit.ly the company and the service go belly-up, you will be in control of the bit.ly DNS domain and thus be able to point it at your redirections? This seems unlikely. The use of Memento would avoid the need to control the DNS domain.
ReplyDelete