A tweet is not just a tweet – Linkrot and our digital history.

When I talk and give workshops I often have a slide stating my concern for the demise of stories on the web.

Of course some people use Twitter for just that. Disposable throw-away comments.

I doubt micro-bloggers and journalists always feel the same way. Or the people who use twitter instead of email. Or those serialising novels through these channels, seeding projects, replacing letter writing or sharing stories in general.

I have a few friends using online archiving solutions to back up their social platforms. They back up their status updates as they would their documents and multimedia. Though hardly ever with the conversations and context attached.

It’s not just an issue with Twitter of course. Connections are being severed continuously. And our current attempts to archive these digital histories can’t keep up.

As we invest so much of our cultural, conversational histories and stories, is it too much to ask that the companies hosting and archiving them guarantee that they will not only be there for future generations, but they will maintain the connections and metadata? Unless of course we choose to annoanamise or remove that data ourselves.

Recently I wanted to move my blog content so it would sit under one banner, Documentally.com. Richard Mackney made the migration painless, showing me that with a little preparation you could maintain all the old links by redirecting them to the new url’s.

It got me thinking again about all the times holes in the internet are formed when links die. Either through sites and pages moving or vanishing, or url shorteners failing to connect.

It’s not just this linkrot that’s a huge issue. It’s the lack of responsibility on behalf of the start-ups not offering some kind of guarantee. All the time asking for content creators and conversationalists to invest their time and stories into pages which may not be there next year.

Would we behave any differently if we realised we were sharing our thoughts with the digital version of invisible ink? Isn’t the archiving quality of all that is digital flaky enough already?

Those who used the asynchronous video conversational platform Seesmic will know just what I’m talking about. Thanks to someone on the inside tipping me off, I was able to download most of the 6000 videos I’d recorded. Many users were not so lucky.

Through no fault of @Kosso‘s the Phreadz servers were also switched off, losing countless archived conversations and the videos that made them. Kosso now implements a dropbox backup system for users who share multimedia through his platform (currently in closed beta) Mobods.

When the money ran out, 12Seconds.tv acted more responsible than many when they offered users a download of all their 12 second videos as a standalone website that could be re-hosted elsewhere.

I wonder what kind of back-up service the mighty Twitter or Facebook have in mind?

How can they guarantee anything of the sort after Twitter for instance installed a ‘central point of failure’ with it’s link shortening service. It’s reported that since 2001, 600 of a reported 1000 link shortening services have ceased to operate.

On the 8th of October 2012 Twitter’s t.co link shortening system experienced a 40 minute outage when after a phishing complaint an Australian hosting company put the t.co domain on hold.

As users, when we post and share content online, if we care about the stories embedded within our musings, we need to think more about web archiving solutions. Perhaps even insist platforms backup to the places already harvesting web content to keep it safe for the future.

I’d happily support the reinvention of the local library. A place I have sadly ignored for years. I’m thankful my local library has so far survived the cull but would they not also work well as a decentralised server farm for remote web data harvesting.

I’m just thinking out loud now. There are no doubt existing solutions and better to come. I fear we just don’t think about this enough. And there are so many reasons for us to care about archiving the web.

Despite it being the largest document ever written, It’s the collective works of countless international authors working in the worlds languages. It’s certainly growing faster than I can read it and this growth is not going to slow down even though huge portions of the web are continually disappearing.

Perhaps it’s down to the individual to become Librarian and Archivist of all that we produce and curate. I don’t think preservation and access is the most important concern in the minds of the patrolling stakeholders as they pace the perimeters of their walled gardens and data silo’s.

What will you have to show for the years of your life spent inputting data to the web?

A tweet is not just a tweet.  It’s a part of our digital heritage. A fragment of our culture, digitised.

Should it really be this ephemeral?


  1. says

    These are all very good points. Frankly, they currently don’t worry me that much. I like the ephemeral nature of some social networks, and I hate looking through old posts on most channels. The problem, I suspect, is that the future me will feel very differently about this, so it would be nice to have more of this type of information archived, and available.

    Also, the guy in the tweet at the top seems like a sensible, good-looking fellow.

  2. Roger Overall says

    Wow. This is an impressive piece of thinking, as well as being a sobering warning.

    Photographers have known this for a while, that digital can be a road to a pictureless and memory-deficient future. Discs corrode, hard-drives fail, cloud storage companies can vanish in … well, in a cloud. Digital is intangible. It is difficult to harness for the long term. What is the digital equivalent of stumbling across an pile of old, forgotten photographs in the attic?

    I think that any digital storyteller needs to realize that their stories will be lost unless they themselves do something about it. The platforms we use are not accountable to us. They are interested for the most part in traffic, numbers, subscribers, data – our data – to monetize.

    Thing is, humans are fickle creatures. We’ll broadcast, but we won’t back up. Maybe if we were forced to back up, we’d think more about what we push out on to the internet.

    Yet, would that be a good or a bad thing?

    • says

      Cheers for the comment Roger.

      The more I share in these spaces the more I appreciate what self hosting your own blog has to offer. WordPress has been around for a long time and yet I have only scraped the surface of it’s offerings. If something should happens to a hosting company it’s just a matter of shifting your (hopefully) backed up data.

      Although I use Disqus as the comment system I have seen it’s easy to migrate the data from the comments back into WordPress and the conversation to be had here will hopefully still be around long after certain other platforms have fallen by the wayside.

      This blog does have some battle scars though. If you click on the Seesmic graphic you will be taken to an old blog post from Dec 2007. A blog post with a gaping hole where a Seesmic video used to be.

      I tend to self host images now as well. I used to drop flickr embeds in here but there is always a niggling worry that they won’t be around for ever.

  3. says

    I was always told data does not exist, unless it is in two places.

    I use wordpress to back up my tweets and host my twitter pictures. WordPress is then automatically backed up weekly to dropbox and sugarsync, and therefore my pc.

    I also host my own url shortening service, so although through twitter these are reshortened to t.co’s, I will ensure my links will live as long as I want them too.

    I chose to set up these systems to ensure I kept copyright, and to ensure any companies that went bust / changed policies / were sold didn’t mean the end of my data. I’m increasingly pleased I took that path and will continue to develope my own services.

  4. says

    I’ve been doing this for a while – backing up my ancient email and IM conversations. I fully expect them to be of great interest to subsequent generations (once I’m elected supreme overlord).

    A few points to note.

    Facebook, as much as I dislike them, offer a really good backup of your content. You can request to download everything you’ve posted. It’s all there – images, video, text. Sadly (and somewhat understandably) it’s missing other people’s content.

    Twitter, shitty though they have been, expose the destination of their t.co links in their API. This means that if you do take a backup, you don’t have to worry about being unable to decipher the links. Whether HampsterDance.com will still be there in 10 years time is another question. (Aside, some Twitter clients are naughty and remove the t.co link and replace it with the real destination).

    One thing I wish I did was archive the comments I leave on other people’s websites. You’re using Disqus which means I can grab a copy – but I wonder how many sage words I’ve typed in to long-forgotten WordPress installs?

    Of course, all our content follows Sturgeon’s Law. But the remaining 10% is priceless.

  5. says

    Christian, you’ve written what I’ve been thinking about since your 2012 Orwell remembrance gathering. It was that series of posts that got me to thinking: “Hey, Christian’s been doing this for 4-5 years now. Let me pull up all those accounts of those occasions and re-live them.
    >>#1 Linkrot<>#2 Search<>#3 Digital: ephemeral irony<<
    You've cited numerous other services (Seesmic, Phreadz, etc.) that in the grand scheme of things lived but a "blink of the eye" – yet contain, for some, memories worth preserving longer than a "blink". And other services, even though they may still be in service, we may *choose* to move on to other services better suited to meet our morphing needs. It's the old Latin saying of "Caveat Emptor" or in this case "Caveat utitur!" (according to Google translate)
    This whole ephemeral/digital topic could be an entire blog theme for a multi-disciplinary community.
    Christian, thank you for this post that at least got me off my butt (arse) to chime in — in support of this very important & silently pervasive theme.
    -Jeff (@bundini)

  6. says

    In this complex and emergent digital world storing, memory, has become as important as being able to do a just in time recovering and visualizing of information.

    The same complexity that empowers us to add novelty (new content) to the system and creates innovation, promotes at the same time and inability to handle and organize it, especially from a central point. Don´t see this struggle slowing down anytime soon…

    I hope that in the future we share some kind of “containerization” (see http://vimeo.com/49392667), some kind of standard protocol or open graph standard for our digital content that most of us rely on and can build tools and services upon but I think that till the struggle between distributed novelty and the need for centralization/aggregation will continue.

    Maybe the “backup as we know it” for digital content will need to evolve into something else that we can not even imagine right now…

  7. says

    A fascinating post. To pick up on your final comments – I too have wondered at times about the possibilty and desirability (or not) of so-called digital immortality. I quite like the idea of a digital librarian or archivist, but I do wonder if the role should be more of a curator in a museum rather than a librarian?

    Most of todays tools and systems are merging the mundane with the substantial – posts written for posterity are often merged with mutterings for the moment. It is possible to record every moment of the day, but when does one have time to play it back? Would we like our descendants to gain a raw feed of everything that has happened or would we rather have the digital equivalent of finding a photograph album in the attic?

    Eventually there may be algorithms that can sift through someone’s entire tweet, status or photographic digital history and pick out the good bits. Eventually there may even be digital recreations of our current selves, based on our recorded digital outpourings. But I think I might like to choose my own version of my history rather than leave it to an algorithm.

    Archiving may be good for future history or sociology research, but I wonder if highlights, chosen by us today, might be best to preserve for the future by a digital caretaker or curator.

    Anyway, as I said a fascinating subject, and thank you for making me think about it again.

    (some previous thoughts here: https://emalliab.wordpress.com/2011/12/23/write-only-memories/)

  8. Pendant says

    I disagree. A tweet is just a tweet. I think Twitter is pretty much the perfect asynchronous communication tool. Valid in the existential now, and ephemeral thereafter, in exactly the same way that the spoken word is the perfect synchronous communication tool.

    Would you have everyone record their every spoken word throughout their lives, and then have all that data backed up in perpetuity on the grounds that it forms part of our cultural heritage? Of course not.

    And as for Facebook: the only sensible thing you can do with that is employ the litany against FB.

    I’m totally with you on linkrot, though. Some people just don’t seem to get it that it’s important to think about link longevity. I’ve been preaching that for years, but it still all falls on deaf ears: too many web wankers are too worried about making sure their content doesn’t look stale — so they change everything around, usually with scarcely any thought for those who have content that links to them and will have to deal with it if they don’t want broken links on their own content… /sigh.

    PS typo alert:
    “Unless of course we choose to annoanamiseanonymise or remove that data…”