IEEE websites place cookies on your device to give you the best user experience. By using our websites, you agree to the placement of these cookies. To learn more, read our Privacy Policy.

The augmented reality contact lens reaches a major milestone

Mojo Vision CEO Drew Perkins wears one of the company’s augmented reality contact lenses.

Editor’s note: In March, I looked through Mojo Vision’s AR contact lens—but I didn’t put it in my eye. At that point, while non-working prototypes had been tested for wearability, nobody had worn the fully functional, battery-powered, wirelessly communicating, device. Today, Mojo announced that its augmented reality lens had gone on-eye—specifically, on the eye of Mojo Vision CEO Drew Perkins, on 23 June.

“I’ve worn it. It works....and it was the first ever on eye demonstration of a feature complete augmented reality smart contact lens,” reported Perkins in a blog post. “The final technical hurdle to wearing the lens was ensuring that the power and radio communications systems worked without wires. Cutting the cord [proved] that the lens and all major components are fully functional and reduce many of the technical challenges in building a smart contact lens.”

It’s an exciting milestone for Perkins and the Mojo Vision team, and we’ll continue monitoring their progress.

Early this year, IEEE Spectrum editor Tekla Perry tested Mojo Vision's AR contact lens by holding it very close to one eye and peeking through.Mojo Vision

In March,Mojo Vision unveiled its latest AR contact lens. Still a prototype, the device has clinical testing and further development ahead before it can apply for the U.S. Food and Drug Administration (FDA) approval needed to sell to consumers. But Mojo’s engineers are steadily ticking off engineering milestones.

Last week I got to literally peek through Mojo’s newest lens. Here’s what I saw, and what Steve Sinclair, Mojo senior vice president of product and marketing, had to say about the company’s progress so far and the challenges that remain.

First, the demo. I did not put the lens in my eye—this prototype is still in safety testing, and fitting a contact lens requires an eye exam. Instead, I held a lens very close to one eye and peeked through. I was able to move around freely, but because holding instead of wearing the device means that it cannot track eye movements, Mojo has temporarily incorporated a tiny crosshair into the user interface to help with alignment. The nature of the demo also meant that images I saw were flat; with a lens in both eyes, the images will appear in 3D.

• Medical-grade microbatteries on board along with a custom power management IC

• High-res 14,000 pixels per inch monochrome microLED display (up from about 8,000 pixels per inch)

• Custom 5-gigahertz radio using a proprietary low-latency communications protocol

• Eye tracking using accelerometer, gyroscope, and magnetometer

• Coming soon: Custom image sensors

I tried out several applications. To select them, I looked around the periphery of the real-world view in front of me, which caused icons to appear. By focusing on one (in this case, aligning the crosshairs), I selected it. The app I found to be the most fun to use wasn’t the most complex, but it did show off the sensors on board by tagging compass headings as I turned to face different directions. I also played with a travel tools app that included a high-resolution monochrome image of an incoming Uber driver, a biking app that called up heart rate and other information useful on a training ride, a teleprompter app that naturally scrolled up and down to move through the text, and a monochrome video stream. All these demos took place in the prototype lens, with me fully in control of them. Mojo’s team then moved to a VR simulation to demonstrate eye tracking features that won’t work unless the lens is sitting in your eye.

This isn’t the first time I’ve seen the Mojo lens in person. But since that last demo in 2020, the engineering team has moved from wireless power to batteries on board, increased the resolution of the display from 8,000 pixels per inch to 14,000, thinned commercial motion sensors and developed its own radio and power management, and created several apps. Image sensors—a feature of earlier demos that showed off edge detection in low light and other vision enhancements—have yet to be built into the current prototype, though Sinclair says that they are in the works, and that the company is continuing to test apps for people with low vision and still expects them to be among the earliest customers for the device.

Here’s what else Sinclair had to say about the technology as developed so far and the path ahead.

Let’s talk about the batteries.

Steve Sinclair: The battery is in the outer ring, embedded in the lens. We are partnering with a medical-grade battery company that makes implantable microbatteries for things like pacemakers, to design something safe to wear. Previously, we were using magnetic inductive coupling, and that seemed fine when you were holding it up to your eye. But the moment you put it on your eye and started moving your head and darting your eye and looking around, we were losing that connection to the wireless field. It wasn’t as reliable as we needed it to be. And so we made a decision about a year and a half ago to switch over to battery power.

I recall you always wanted to get batteries on board?

Sinclair: We thought so. But we sped that path up, because we came to the conclusion that the other path was just not a good way to go.

Why doesn’t this version have the image sensor?

Sinclair: We’ve decided to leave out the imager, the camera, for right now; it’s not critical to the use cases that we’re looking at first.

But you had been working to develop applications for people with low vision. What happened to that?

Sinclair: We’ve been using mainly low-vision capabilities built into smartphones right now to take pictures of things and bring them up to your eye for zooming in and out; we’ll add in the imager and test those capabilities out next. It’s just about there, but it wasn’t necessary to get to this milestone, so we decided to simplify things a little bit.

Given the amount of lens real estate being taken up by batteries and chips, was oxygenation an issue?

Sinclair: Absolutely. That was a core engineering point that needed to be solved before we could get to this lens. So this lens has channels built into it and a special design that allows air to get in and get through. The oxygen diffuses out over the surface of the cornea and the sclera.

The batteries and chips within the Mojo Lens are arranged to preserve peripheral vision.

Does the circuitry block vision at all?

Sinclair: See the cutout on the side? Imagine I’m wearing it. It’s going to be oriented like this [the cutout away from the nose] because I need the peripheral vision on the outside, not towards the nasal side. Ultimately, there’s even more we can do to push components further toward the edges of the lens to maximize light entering the pupil.

Now that you explain the arrangement of circuitry to protect peripheral vision, it seems obvious, but some of the most obvious things take figuring out.

Sinclair: It wasn’t immediately obvious; our original design had [the electronics] all the way around.

Sinclair: We’re already wearing lenses that don’t have any electronics in them, but they’re shaped the same way, so that we can test for comfort and for how long can we wear them and have the oxygenation still working to our satisfaction.

Next we start testing [the complete prototype] on-eye, and see just how well it works in different situations. We can’t say exactly when that happens. We hope it’s soon, but we’ve got to make sure it’s safe and everything’s working the way we expect it to work. That testing will start with Drew Perkins wearing it, our CEO, then probably Mike Wiemer, our CTO. And then it goes to folks like myself and others on the executive team, along with some of the key engineers that need to start evaluating things quickly.

We’ll be testing the software, the battery life, and the wireless connectivity and [its] speeds. There’s likely to be a spin of one, two, or three of the chips built into the lens as we discover things that don’t work right or could be optimized to work better together as a system.

What’s the timeline for commercialization?

Sinclair: Everything is predicated on eventually getting FDA certification. We don’t like to presume how long that’s going to take.

You’re going for the consumer market. Today, it seems, most AR glasses manufacturers have decided to focus on the enterprise market initially. Why the different path?

Sinclair: With AR glasses, there are definitely some awesome enterprise use cases. But these are not great for contact lenses. Say you’re an IT manager, and you’ve come up with an awesome contact lens application. Can you tell your workers to wear a contact lens? Not usually. But consumers can make that decision. The reason people wear contact lenses is very consumery: I want to look like myself. I want to use it when I’m working out or doing sports. I don’t like things on my face. I don’t like the fashion sense of glasses; it’s not me. We decided to lean into those reasons that people pick contact lenses.

Can you talk about pricing?

Sinclair: Like anything else, when we first bring it out, it’ll be a little expensive. Our goal when we’re running at volume is that it should come out somewhere close to a high-end smartphone. But factor in the fact that people are already spending US $500, $600, or $700 for eyewear today, subtract that out of the total price, and the adder on top of that is not huge.

This article appears in the June 2022 print issue as “My Peek Through Mojo Vision’s AR Contacts.”

Tekla S. Perry is a senior editor at IEEE Spectrum. Based in Palo Alto, Calif., she's been covering the people, companies, and technology that make Silicon Valley a special place for more than 40 years. An IEEE member, she holds a bachelor's degree in journalism from Michigan State University.

How fragile is this product and what is the life cycle of one. Will people really spend $500+ on something that can be easily lost or damaged.

What's the largest criminal enterprise in the world? Narcotics? Gambling? Human trafficking?

Nope. The biggest racket is the production and trade of counterfeit goods, which is expected to exceed US $1 trillion next year. You've probably suffered from it more than once yourself, purchasing on Amazon or eBay what you thought was a brand-name item only to discover that it was an inferior-quality counterfeit.

It's an all-too-common ploy, and legitimate manufacturing companies and distributors suffer mightily as a result of it. But the danger runs much deeper than getting ripped off when you were seeking a bargain. When purchasing pharmaceuticals, for example, you'd be putting your health in jeopardy if you didn't receive the bona fide medicine that was prescribed. Yet for much of the world, getting duped in this way when purchasing medicine is sadly the norm. Even people in developed nations are susceptible to being treated with fake or substandard medicines.

Tiny mechanical resonators produced the same way microchips are made (bottom) can serve to authenticate various goods. Being less than 1 micrometer across and transparent, these tags are essentially invisible.University of Florida

Counterfeit electronics are also a threat, because they can reduce the reliability of safety-critical systems and can make even ordinary consumer electronics dangerous. Cellphones and e-cigarettes, for example, have been known to blow up in the user's face because of the counterfeit batteries inside them.

It would be no exaggeration to liken the proliferation of counterfeit goods to an infection of the global economy system—a pandemic of a different sort, one that has grown 100 fold over the past two decades, according to the International AntiCounterfeiting Coalition. So it's no wonder that many people in industry have long been working on ways to battle this scourge.

The traditional strategy to thwart counterfeiters is to apply some sort of authentication marker to the genuine article. These efforts include the display of Universal Product Codes (UPC) and Quick Response (QR) patterns, and sometimes the inclusion of radio-frequency identification (RFID) tags. But UPC and QR codes must be apparent so that they are accessible for optical scanning. This makes them susceptible to removal, cloning, and reapplication to counterfeit products. RFID tags aren't as easy to clone, but they typically require relatively large antennas, which makes it hard to label an item imperceptibly with them. And depending on what they are used for, they can be too costly.

We've come up with a different solution, one based on radio-frequency (RF) nanoelectromechanical systems (NEMS). Like RFID tags, our RF NEMS devices don't have to be visible to be scanned. That, their tiny size, and the nature of their constituents, make these tags largely immune to physical tampering or cloning. And they cost just a few pennies each at most.

Unseen NEMS tags could become a powerful weapon in the global battle against counterfeit products, even counterfeit bills. Intrigued? Here's a description of the physical principles on which these devices are based and a brief overview of what would be involved in their production and operation.

You can think of an RF NEMS tag as a tiny sandwich. The slices of bread are two 50-nanometer-thick conductive layers of indium tin oxide, a material commonly used to make transparent electrodes, such as those for the touch screen on your phone. The filling is a 100-nm-thick piezoelectric film composed of a scandium-doped aluminum nitride, which is similarly transparent. With lithographic techniques similar to those used to fabricate integrated circuits, we etch a pattern in the sandwich that includes a ring in the middle suspended by four slender arms. That design leaves the circular surface free to vibrate.

The material making up the piezoelectric film is, of course, subject to the piezoelectric effect: When mechanically deformed, the material generates an electric voltage across it. More important here is that such materials also experience what is known as the converse piezoelectric effect—an applied voltage induces mechanical deformation. We take advantage of that phenomenon to induce oscillations in the flexible part of the tag.

To accomplish this, we use lithography to fabricate a coil on the perimeter of the tag. This coil is connected at one end to the top conductive layer and on the other end to the bottom conductive layer. Subjecting the tag to an oscillating magnetic field creates an oscillating voltage across the piezoelectric layer, as dictated by Faraday's law of electromagnetic induction. The resulting mechanical deformation of the piezo film in turn causes the flexible parts of the tag to vibrate.

This vibration will become most intense when the frequency of excitation matches the natural frequency of the tiny mechanical oscillator. This is simple resonance, the phenomenon that allows an opera singer's voice to shatter a wine glass when the right note is hit (and if the singer tries really, really hard). It's also what famously triggered the collapse of the Broughton suspension bridge near Manchester, England, in 1831, when 74 members of the 60th Rifle Corps marched across it with their footsteps landing in time with the natural mechanical resonance of the bridge. (After that incident, British soldiers were instructed to break step when they marched across bridges!) In our case, the relevant excitation is the oscillation of the magnetic field applied by a scanner, which induces the highest amplitude vibration when it matches the frequency of mechanical resonance of the flexible part of the tag.

These electron micrographs show some of the tags the authors have fabricated, which can take various forms. The preferred geometry (top) is a circular tag containing a piezoelectric ring suspended by four beams. It includes a coil (lighter shade), which connects with electrode layers on the top and bottom of the ring. Voltages induced in this coil by an external scanner set up mechanical oscillations in the ring.University of Florida

In truth, the situation is more complicated than this. The flexible portion of the tag doesn't have just one resonant frequency—it has many. It's like the membrane on a drum, which can oscillate in various ways. The left side might go up as the right side goes down, and vice versa. Or the middle might be rising as the perimeter shifts downward. Indeed, there are all sorts of ways that the membrane of a drum deforms when it is struck. And each of those oscillation patterns has its own resonant frequency.

We designed our nanometer-scale tags to vibrate like tiny drumheads, with many possible modes of oscillation. The tags are so tiny—just a few micrometers across—that their vibrations take place at radio frequencies in the range of 80 to 90 megahertz. At this scale, more than the geometry of the tag matters: the vagaries of manufacturing also come into play.

For example, the thickness of the sandwich, which is nominally around 200 nm, will vary slightly from place to place. The diameter or the circularity of the ring-shaped portion is also not going to be identical from sample to sample. These subtle manufacturing variations will affect the mechanical properties of the device, including its resonant frequencies.

In addition, at this scale the materials used to make the device are not perfectly homogeneous. In particular, in the piezoelectric layer there are intrinsic variations in the crystal structure. Because of the ample amount of scandium doping, conical clusters of cubic crystals form randomly within the matrix of hexagonal crystals that make up the aluminum nitride grains. The random positioning of those tiny cones creates significant differences in the resonances that arise in seemingly identical tags.

Random variations like these can give rise to troublesome defects in the manufacture of some microelectronic devices. Here, though, random variation is not a bug—it's a feature! It allows each tag that is fabricated to serve as a unique marker. That is, while the resonances exhibited by a tag are controlled in a general way by its geometry, the exact frequencies, amplitudes, and sharpness of each of its resonances are the result of random variations. That makes each of these items unique and prevents a tag from being cloned, counterfeited, or otherwise manufactured in a way that would reproduce all the properties of the resonances seen in the original.

An RF NEMS tag is an example of what security experts call a physical unclonable function. For discretely labeling something like a batch of medicine to document its provenance and prove its authenticity, it's just what the doctor ordered.

You might be wondering at this point how we can detect and characterize the unique characteristics of the oscillations taking place within these tiny tags. One way, in principle, would be to put the device under a vibrometer microscope and look at it move. While that's possible—and we've done it in the course of our laboratory studies—this strategy wouldn't be practical or effective in commercial applications.

But it turns out that measuring the resonances of these tags isn't at all difficult. That's because the electronic scanner that excites vibrations in the tag has to supply the energy that maintains those vibrations. And it's straightforward for the electronic scanner to determine the frequencies at which energy is being sapped in this way.

The authors directly measured the surface topography of a tag using a digital holographic microscope, which is able to scan reflective surfaces and precisely measure their heights (top). The authors also modeled various modes of oscillation of the flexible parts of such a tag (bottom). Each mode has a characteristic resonant frequency, which varies with both the geometry of the tag and its physical composition.University of Florida; Bottom: James Provost

The scanner we are using at the moment is just a standard piece of electronic test equipment called a network analyzer. (The word network here refers to the network of electrical components—resistors, and capacitors, and inductors—in the circuit being tested, not to a computer network like the Internet.) The sensor we attach to the network analyzer is just a tiny coil, which is positioned within a couple of millimeters of the tag.

With this gear, we can readily measure the unique resonances of an individual tag. We record that signature by measuring how much the various resonant-frequency peaks are offset from those of an ideal tag of the relevant geometry. We translate each of those frequency offsets into a binary number and string all those bits together to construct a digital signature unique to each tag. The scheme that we are currently using produces 31-bit-long identifiers, which means that more than 2 billion different binary signatures are possible—enough to uniquely tag just about any product you can think of that might need to be authenticated.

Relying on subtle physical properties of a tag to define its unique signature prevents cloning but it does raise a different concern: Those properties could change.

For example, in a humid environment, a tag might adsorb some moisture from the air, which would change the properties of its resonances. That possibility is easy enough to protect against by covering the tag with a thin protective layer, say of some transparent polymer, which can be done without interfering with the tag's vibrations.

But we also need to recognize that the frequencies of its resonances will vary as the tag changes temperature. We can get around that complication, though. Instead of characterizing a tag according to the absolute frequency of its oscillation modes, we instead measure the relationships between the frequencies of different resonances, which all shift in frequency by similar relative amounts when the temperature of the tag changes. This procedure ensures that the measured characteristics will translate to the same 31-bit number, whether the tag is hot or cold. We've tested this strategy over quite a large temperature range (from 0 to 200 °C.) and have found it to be quite robust.

A tag is characterized by the differences between its measured resonant frequencies (dips in red line) and the corresponding frequencies for an ideal tag (dips in black line). These differences are encoded as short binary strings, padded to a standard length, with one bit signifying whether the frequency offset of positive or negative (right). Concatenated, these strings provide a unique digital fingerprint for the tag (bottom)University of Florida

The RF network analyzer we're using as a scanner is a pricey piece of equipment, and the tiny coil sensor attached to it needs to be placed right up against the tag. While in some applications the location of the tag on the product could be standardized (say, for authenticating credit cards), in other situations the person scanning a product might have no idea where on the item the tag is positioned. So we are working now to create a smaller, cheaper scanning unit, one with a sensor that doesn't have to be positioned right on top of the tag.

We are also exploring the feasibility of modifying the resonances of a tag after it is fabricated. That possibility arises from a bit of serendipity in our research. You see, the material we chose for the piezoelectric layer in our tags is kind of unusual. Piezoelectric devices, like some of the filters in our cellphones, are commonly made from aluminum nitride. But the material we adopted includes large amounts of scandium dopant, which enhances its piezoelectric properties.

Unknown to us when we decided to use this more exotic formulation was a second quality it imparts: It makes the material into a ferroelectric, meaning that it can be electrically polarized by applying a voltage to it, and that polarization remains even after the applied voltage is removed. That's relevant to our application, because the polarization of the material influences its electrical and mechanical properties. Imparting a particular polarization pattern on a tag, which could be done after it is manufactured, would alter the frequencies of its resonances and their relative amplitudes. This approach offers a strategy by which low-volume manufacturers, or even end users, could “burn" a signature into these tags.

Our research on RF NEMS tags has been funded in part by Discover Financial Services, the company behind the popular Discover credit card. But the applications of the tiny tags we've been working on will surely be of interest to many other types of companies as well. Even governments might one day adopt nanomechanical tags to authenticate paper money.

Just how broadly useful these tags will be depends, of course, on how successful we are in engineering a handheld scanner—which might even be a simple add-on for a smartphone—and whether our surmise is correct that these tags can be customized after manufacture. But we are certainly excited to be exploring all these possibilities as we take our first tentative steps toward commercialization of a technology that might one day help to stymie the world's most widespread form of criminal activity.

This article appears in the June 2021 print issue as “The Hidden Authenticators."