The augmented city: how technologists are transforming the Earth into theater

The Augmented Landscape is where augmented reality technology enhances how humans experience the landscape. “Landscape” is both the visible features of an area of land and the historical, cultural, and personal significance — its sense of place. Jonathan Ive, soon-to-be-former Apple Chief Design Officer, tells us: “the context of history and culture is as critical as understanding the opportunities that are actually afforded by developing technology.” Inspired by this advice, I’d like to provide an introduction to relevant history and culture applicable to this new emerging AR world.

1825, Divis Mountain, Belfast. One of the most technologically advanced surveys of the first half of the 19th century was the crown-sponsored mapping endeavor of the island of Ireland. The survey adopted innovative land surveying techniques with humanistic data collation. The mountain witnesses the first practical use of the “limelight” by engineer Thomas Drummond. From the thin places of Irish mountain tops, the device projects a dazzling bright white light over 66 miles enabling surveyors to triangulate with greater accuracy.

1837, Covent Garden Theatre, London. The invention is adapted to illuminate actors on stage. Adoption globally inspired the term when someone is “in the limelight.” Italian philosopher Giambattista Vico theorized that civilizations developed historically in a recurring cycle, “corsi e ricorsi.” History repeats today, with a difference. Technologies of surveying are supporting a global theater.

Invisible map, invisible stage

Immersive AR experiences will require some of the most sophisticated mapping ever created. It will be invisible to users. The individual, the story, the creation, the ordinary is elevated to take center stage. The “locus dramaticus,” or “privileged spot” in the words of theater theorist Andre Bazin. A carefully crafted topographical stage to save, share, and experience digital content. Laurence Olivier Award winner Peter Brook states: “I can take any empty space and call it a bare stage. A man walks across this empty space whilst someone else is watching him, and this is all that is needed for an act of theater to be engaged.”

Paralleling the contrarian Da Vinci who introduced dynamically changing theater stages, the Augmented Landscape encompasses a regularly refreshed stripped bare 3D map. Professionals and laypeople alike will continually map the land. Enabling others to stage their stories and art, using light to reveal the “genius loci” of a place. The new Apple [AR]T initiative, a collaboration with artists for location-specific AR experiences choreographed into the landscape of six cities, is one of many examples exploring this global stage. Would you like to sync to location?

Building the global stage

Want incredible immersive experiences in your city? Remaining technological hurdles include saving digital content to location, accessing a real-time 3D semantic world map, occlusion of digital content with the physical world, and multi-player. Centimeter positioning is required. However, Global Navigation Satellite Systems (GNSS) such as BeiDou, Galileo, and GPS fail to achieve this without the software and hardware to tap into geodetic infrastructure. Advancing capabilities of consumer cameras, leveraging dual raw GNSS data, 5G networks, and computer vision offer potential solutions, including triangulating position from landscape images snapped from a smartphone.

2019, Buckingham Palace, London. The palace is one of the locations augmented, or enabled, or activated, via Snap’s Lens Studio Landmarker enabling real-time AR immersive experiences. Studio has achieved over 400,000 AR lens and 15 billion plays. Snap currently has a US market catchment of 90% of 13-to 24-year-olds, a higher share than Facebook or Instagram. Global hate speech, disinformation, and abuse are potential consequences of this new mass AR medium if left unchecked. Digital Allemansrätten with embedded “Don’t Hate” and privacy governance measures offers a solution to design these out at the earliest stage to avoid a scarred landscape. Often humanity has not been so good to itself, and AR needs to develop knowing that.

Into the infinite limelight

The above mapping expedition influenced Enniskillen-educated Samuel Beckett, poet, playwright, and theater director innovator (as exemplified in his masterpiece play Waiting for Godot).

1973, Royal Court Theatre, London. Beckett collaborates with scenographer (theater designer) Jocelyn Herbert. They were concerned not only with the actor’s performance but also with visual and practical details of the stage design. The “mise-en-scène,” which defines the stage world for both the actors and the audience. Beckett’s minimalist stage composition defied convention — “There is a huge amount of work to achieve that nothingness, and to find the right table and chair, and objects for the actor that are both practical and poetic …  above all, we discovered light”. Adopting advances in lighting equipment control, precision, and brilliance became integral.

As part of the AR ecosystem, a multi-billion investment is underway into ‘light field’ technology, building on the work of Nobel Laureate Gabriel Lippman. The technology is poised to redefine an audience’s expectations of witnessing digital media. It is this kind of merging of arts and engineering that gifted Beckett the Nobel Prize in the year we first landed on the moon.

An augmented city

Theater cannot exist without architecture. In Shakespearean plays, the stage was relatively bare — the handling of the action and dialogue conceived as echoing through the architecture of the auditorium. Le Corbusier, the artist-architect who pushed architecture forward in the 20th century, embracing new technologies, proclaimed “the materials of city planning are: sky, space, trees, steel, and cement, in that order and that hierarchy.” With this direction, technologists are set to disrupt the status quo and join those who currently holds responsibility for the public realm: government, city planners, and architects. The inevitable disruption of place. The personalization of the landscape. A city of wonderwalls.

Los Angeles artist Koreen offers a preview. In an “Augmented City,” the building materials of choice are light, space, and sky, the brush is digital. A “digital marking” and syncing to the landscape. Abstract thought has shaped city design through the ages. The philosophy of Confucius architected ancient Chinese cities even through to the spatial order of the Forbidden City (故宫). Engineer Artist Vladimir Tomin offers a teaser trailer of the distance view of the Augmented City.

We remember Drummond, Le Corbusier, and Beckett as they understood their place in history. They combined the arts and technology.  They had “taste.” They were thinkers and doers. “If that fellow was dropped in the middle of the Sahara, he’d sit, be God, and make a map of it,” stated the father of James Joyce, who would himself say “I am really one of the greatest engineers, if not the greatest in the world.” If alive today, could these individuals be working in AR? Perhaps the next great engineers are working with AR today. Personal computing is wandering outside, becoming spatial computing, the web becoming the spatial web. Can there be astute experts here in this new territory? Indeed, leaders will need the right internal compass.

2019, 37.330855, -122.007757, Sir Ive notes, “If you genuinely have a concern for humanity, you will be preoccupied with trying to understand the implications, the consequences of creating something that hasn’t existed before.”

Christopher Mc Alorum is developing objective and impartial investigatory written work designed to combine insights from across the humanities and technology to inform and support all those developing augmented reality.