The metaverse needs aggressive regulation

Thirty years ago, while working at Air Force Research Laboratory, I built the first interactive augmented reality system, enabling users to reach out and engage a mixed world of real and virtual objects. I was so inspired by the reactions people had when they experienced those early prototypes, I founded one of the first VR companies in 1993, Immersion Corp, and later founded an early AR technology company, Outland Research. Yes, I’ve been an enthusiastic believer in the metaverse for a very long time.

I’ve also been a longtime critic of the field, issuing warnings about the negative impacts that AR and VR could have on society, especially when combined with the power of artificial intelligence. It’s not the technology I fear, but the fact that large corporations can use the infrastructure of the metaverse to monitor and manipulate the public at levels that make social media seem quaint. That’s because these platforms will not just track what you click on, but where you go, what you do, what you look at, even how long your gaze lingers. They will also monitor your facial expressions, vocal inflections, and vital signs (as captured by your smart-watch), all while intelligent algorithms predict changes in your emotional state.

This means the platform providers will not just know how you act, but how you react, profiling your responses at the deepest level. I know this degree of intrusion sounds like science fiction, but as someone who has spent decades creating technology in this space, I feel confident this will be our future unless we demand aggressive regulation.

But what should we regulate?

Most importantly, we need to restrict the level of monitoring allowed in the metaverse. The platform providers will have access to where we go, what we look at, who we talk to, even what we reach out and touch. They should not be allowed to store this data for more than the short periods of time required to mediate whatever experience is being simulated. That will reduce the degree to which they can characterize our behaviors over time. In addition, they should be required to inform the public as to what’s being tracked and how long it’s retained. For example, if they’re monitoring the direction of your gaze, you need to be overtly notified.

At the same time, there should be strict limits on what they’re allowed to track and for what purpose. For example, the public should fight hard against advertising algorithms that monitor your facial expressions, vocal inflections, posture, and vital signs (including your heart rate, respiration rate, pupil dilation, blood pressure, and even your galvanic skin response). Unless strictly regulated, these very personal physiological reactions will be used to fine-tune marketing messages in real time.

In addition, we need to regulate how third parties can manipulate us within the metaverse. That’s because with AR and VR, marketing will not be the overt pop-up ads of today, but subtle simulated additions to our world that look and feel real. If a third-party pays for a virtual product placement in your augmented surroundings, the platform should be required to inform you that it’s a targeted placement, not a serendipitous interaction.

This becomes even more important when third-parties engage us with simulated spokespeople (SimGens I call them). In the metaverse, you will be covertly targeted by people who look and act like any other user but are really AI-controlled agents that have been programmed to engage you in “promotional conversation.” With access to your facial expressions and vocal inflections, they will pitch you more skillfully than any used-car salesperson, adapting to your emotions in real-time. Even the way these simulated agents appear to you – their gender, hair color, eye color, clothing style – will be crafted by intelligent algorithms that skillfully predict what features are most likely to influence you personally.

It’s critical that we regulate this domain, requiring third parties to overtly inform us whenever we’re interacting with agenda-driven agents controlled by intelligent algorithms. This is especially important if those algorithms are also monitoring our reactions, for example assessing our posture, our breathing, and even our blood pressure, enabling conversational agents to skillfully adjust their messaging strategy in real time. This extreme level of interactive manipulation will happen unless it’s formally restricted.

Some argue against regulation, saying consumers can simply opt-out of the metaverse if they don’t want to submit to being tracked and profiled. I say that’s naïve, for these platforms will become so essential to how we access our world, opting-out will not be a choice. This is especially true of augmented reality, where critical information will be projected all around us throughout our daily activities. To portray how deeply integrated AR will be in our lives within ten years, I point you to Metaverse 2030, a short narrative I wrote that shows the magical possibilities of the AR metaverse along with its intrusive and overwhelming underbelly.

Ultimately, the metaverse will enable amazing applications that enrich our lives, expanding what it means to be human. At the same time, there are very real dangers we need to avoid. The best way to enable the magic while preventing the dangers is to aggressively regulate the space. And we need to do it now, before the problems become so deeply embedded in the infrastructure and business models they’re impossible to unwind.

The alternative is to end up living in a metaverse that looks and feels natural, while behind the scenes, powerful corporations are manipulating our experiences for the highest bidder without us even realizing it. That’s not the future we want, which means regulation is both necessary and urgent. Like it or not, the metaverse is coming soon.

Louis B. Rosenberg is a computer scientist, entrepreneur, and prolific inventor. Thirty years ago while working as a researcher at Stanford and Air Force Research Laboratory, Rosenberg developed the first functional augmented reality system. He then founded one of the early virtual reality companies (Immersion Corp) and one of the early augmented reality companies (Outland Research). He’s currently founder and CEO of swarm intelligence company Unanimous AI.