Cliff Lynch has just published a long and very important article at
First Monday entitled
Stewardship in the "Age of Algorithms". It is a much broader look than my series
The Amnesiac Civilization at the issues around providing the future with a memory of today's society.
Cliff accurately describes the practical impossibility of archiving the systems such as Facebook that today form the major part of most people's information environment and asks:
If we abandon the ideas of archiving in the traditional preservation
of an artifact sense, it’s helpful to recall the stewardship goal here
to guide us: to capture the multiplicity of ways in which a given system
behaves over the range of actual or potential users. ... Who are these “users”
(and how many of them are there)? How do we characterize them, and how
do we characterize system behavior?
Then, with a tip of the hat to Don Waters, he notes that this problem is familiar in other fields:
they are deeply
rooted in historical methods of anthropology, sociology, political
science, ethnography and related humanistic and social science
disciplines that seek to document behaviors that are essentially not
captured in artifacts, and indeed to create such documentary artifacts
Unable to archive the system they are observing, these fields try to record and annotate the experience of those encountering the system; to record the performance from the audience's point of view. Cliff notes, and discusses the many problems with, the two possible kinds of audience for "algorithms":
- Programs, which he calls robotic witnesses, and others call sock puppets. Chief among the problems here is that "algorithms" need robust defenses against programs posing as humans (see, for example, spam, or fake news).
- Humans, which he calls New Nielson Families. Chief among the problems here is the detailed knowledge "algorithms" use to personalize their behaviors, leading to a requirement for vast numbers of humans to observe even somewhat representative behavior.
Cliff concludes:
From a stewardship point of view (seeking to preserve a reasonably
accurate sense of the present for the future, as I would define it),
there’s a largely unaddressed crisis developing as the dominant archival
paradigms that have, up to now, dominated stewardship in the digital
world become increasingly inadequate. ... the existing models and
conceptual frameworks of preserving some kind of “canonical” digital
artifacts ...
are increasingly inapplicable in a world of pervasive, unique,
personalized, non-repeatable performances. As stewards and stewardship
organizations, we cannot continue to simply complain about the
intractability of the problems or speak idealistically of fundamentally
impossible “solutions.”
...
If we are to successfully cope with the new “Age of Algorithms,” our
thinking about a good deal of the digital world must shift from
artifacts requiring mediation and curation, to experiences.
Specifically, it must focus on making pragmatic sense of an incredibly
vast number of unique, personalized performances (including interaction
with the participant) that can potentially be recorded or otherwise
documented, or at least do the best we can with this.
I agree that society is facing a crisis in its ability to remember the past. Cliff has provided a must-read overview of the context in which the crisis has developed, and some pointers to pragmatic if unsatisfactory ways to address it. What I would like to see is a even broader view, describing this crisis as one among many caused by the way increasing returns to scale are squeezing out the redundancy essential to a resilient civilization.
