Now, what can be done for one image can also be done for moving images, since movies are just a long series of still images in a row. Perceiving movies takes a lot more processing power, in part because there is the added dimension of time (do objects persist as the camera moves?). In a few years we’ll be able to routinely search video via AI. As we do, we’ll begin to explore the Gutenberg possibilities within moving images. “I consider the pixel data in images and video to be the dark matter of the Internet,” says Fei-Fei Li, director of the Stanford Artificial Intelligence Laboratory. “We are now starting to illuminate it.”
As moving images become easier to create, easier to store, easier to annotate, and easier to combine into complex narratives, they also become easier to be remanipulated by the audience. This gives images a liquidity similar to words. Fluid images flow rapidly onto new screens, ready to migrate into new media and seep into the old. Like alphabetic bits, they can be squeezed into links or stretched to fit search engines and databases. Flexible images invite the same satisfying participation in both creation and consumption that the world of text does.
In addition to findability, another ongoing revolution within media can be considered “rewindability.” In the oral age, when someone spoke, you needed to listen carefully, because once the words were uttered, they were gone. Before the advent of recording technology, there was no backing up, no scrolling back to hear what was missed.
The great historical shift from oral to written communications that occurred thousands of years ago gave the audience (readers) the possibility to scroll back to the beginning of a “speech,” by rereading it.
One of the revolutionary qualities of books is their ability to repeat themselves for the reader, at the reader’s request, as many times as wanted. In fact, to write a book that is reread is the highest praise for an author. And in many ways authors have exploited this characteristic of books by writing them to be reread. They may add plot points that gain meaning on second reading, hide irony that is only revealed on rereading, or pack it full of details that require close study and rereading to decipher. Vladimir Nabokov once claimed, “One cannot read a book: one can only reread it.” Nabokov’s novels often featured an unreliable narrator (for instance, Pale Fire and Ada, or Ardor), which strongly encouraged readers to review the tale from a later, more enlightened perspective. The best mysteries and thrillers tend to end with stealthy last-minute reversals that are brilliantly foreshadowed on second reading. The seven volumes of Harry Potter are packed with so many hidden clues that the stories need to be reread for maximum enjoyment.
Our screen-based media in the last century had much in common with books. Movies, like books, are narrative driven and linear. But unlike books, movies were rarely rewatched. Even the most popular blockbusters were released to theaters on a certain day, played in a local theater for a month, and then were rarely seen again, except on late-night television decades later. In the century before videotape, there was no replaying. Television was much the same. A show broadcasted on a schedule. You either watched it at the time or you never saw it. It was uncommon to watch a just released movie more than once, and only a few television episodes would reappear again as summer reruns. And even then, to watch it you needed to schedule your attention to be present on the day and time when that show was due to run.
Because of this “oral” characteristic of movies and television, shows were engineered with the assumption they would be seen only once. That reasonable assumption was made into a feature because it forced the movie’s narrative to convey as much as possible in the first impression. But it also diminished it because so much more could be crafted to deliver on second and third encounters.
First the VHS, then DVDs, later TiVos, and now streaming video make it trivially easy to scroll back screenworks. If you want to see something again, you do. Often. If you want to see only a snippet of a movie or television program, you do, at any time. This ability to rewind also applies to commercials, news, documentaries, clips—anything online, in fact. More than anything else, rewindability is what has turned commercials into a new art form. The ability to rewatch them has moved them out of the prison of ephemeral glimpses in the middle of ephemeral shows, to a library of shows that can be read and reread like books. And then shared with others, discussed, analyzed, and studied.
We are now witnessing the same inevitable rewindability of screen-based news. TV news was once an ephemeral stream of stuff that was never meant to be recorded or analyzed—merely inhaled. Now it is rewindable. When we scroll back news, we can compare its veracity, its motives, its assumptions. We can share it, fact-check it, and mix it. Because the crowd can rewind what was said earlier, this changes the posture of politicians, of pundits, of anyone making a claim.
The rewindability of film is what makes 120-hour movies such as Lost, or The Wire, or Battlestar Galactica possible, and enjoyable. They brim with too many details ingeniously molded into them to be apparent on initial viewing; scrolling back at any point is essential.
Music was transformed when it became recorded, rewindable. Live music was meant to be of the moment, and to vary from performance to performance. The ability to scroll back to the beginning and hear music again—that exact performance—changed music forever. Songs became shorter on average, and more melodic and repeatable.
Games now have scroll-back functions that allow replays, redos, or extra lives, a related concept. One can rewind the experience and try again, with slightly different variations, again and again, until one masters that level. On the newest racing games, one can rewind to any previous point by literally running the action backward. All major software packages have an undo button that lets you rewind. The best apps enable unlimited undos, so you can scroll back as far as you want. The most complex pieces of consumer software in existence, such as Photoshop or Illustrator, employ what is called nondestructive editing, which means you can rewind to any particular previous point you want at any time and restart from there, no matter how many changes you’ve made. The genius of Wikipedia is that it also employs nondestructive editing—all previous versions of an article are kept forever, so any reader can in fact rewind the changes back in time. This “redo” function encourages creativity.
Immersive environments and virtual realities in the future will inevitably be able to scroll back to earlier states. In fact, anything digital will have undo and rewindability as well as remixing.
Going forward, we are likely to get impatient with experiences that don’t have undo buttons, such as eating a meal. We can’t really replay the taste and smells of a meal. But if we could, that would certainly alter cuisine.
The perfect replication of media in terms of copies is well explored. But the perfect replication of media in terms of rewinding is less explored. As we begin to lifelog our daily activities, to capture our live streams, more of our lives will be scrollable. Typically I dip into my inbox or outbox several times a day to scroll back to some previous episode of my life. If we expect to scroll back, this will shift what we do the first time. The ability to scroll back easily, precisely, and deeply might change how we live in the future.
In our near future we’ll have the option to record as much of our conversations as we care to. It will cost nothing as long as we carry (or wear) a device, and it will be fairly easy to rewind. Some people will record everything as an aid to their memory. The social etiquette around recall will be in flux; private conservations are likely to be off-limits. But more and more of what happens in public will be recorded—and re-viewable—via phone cams, dashboard-mounted webcams on every car, and streetlight-mounted surveillance cams. Police will be required by law to record all activity from their wearables while they are on duty. Rewinding police logs will shift public opinion, just as often vindicating police as not. The everyday routines of politicians and celebrities will be subject to scrolling back from multiple viewpoints, creating a new culture where everyone’s past is recallable.