Team creates 3D objects that change their appearance from different viewpoints

Picture a birthday card that flickers between images of a birthday cake and flowers as you turn the card and view it from different angles. No doubt you can think of other examples of such morphing images in, say, advertisements. Until now, however, the effect has been limited to flat surfaces.

Enter the work of MIT researchers who, for the first time, have developed a system for creating 3D objects that change their appearance when seen from different viewpoints. The same team also developed an editing tool available free online that can allow anyone to design and build such 3D objects, all through 3D printing.

“Our work opens up the idea of what a physical object can be,” says Yunyi Zhu, an MIT graduate student in the Department of Electrical Engineering and Computer Science (EECS). “It’s part of a larger vision for making dynamic objects that change their appearance, color pattern, and shape. We address appearance, which is one dimension of the concept of reprogrammable objects,” says Zhu, who is a co-first author of a paper on the work presented in October at the 2021 Association for Computing Machinery’s Symposium on User Interface Software and Technology.

Stefanie Mueller is the leader of the work, an associate professor in EECS and the Department of Mechanical Engineering, and an author of the paper. Mueller is also affiliated with MIT’s Computer Science and Artificial Intelligence Laboratory.

Other co-first authors of the paper are Jiani Zeng (an MIT graduate student who has since graduated) and Honghao Deng (a Harvard graduate student who has since graduated). In addition to Zhu, Zeng, Deng, and Mueller, other authors are Michael Wessely, a postdoctoral associate in EECS, and Axel Kilian, a visiting assistant professor in the Department of Architecture.

“This exciting research radically shifts what the future of product design [will] look like. As an interaction designer, I was stimulated to imagine how in the future everyday physical objects could have thousands of different appearances (colors and patterns) depending on the viewing angles. I can think of many application, from mesmerizing fashion designs to aesthetic sculptures that convey different poetic narratives depending on the viewpoints,” says Ken Nakagaki, a postdoctoral associate at the MIT Media Lab who is an incoming assistant professor at the University of Chicago. Nakagaki was not involved in the work.

How it Works

The changing images on a flat card—and now 3D objects—are thanks to many tiny lenses printed across the card or object. About six of the lenses created by the MIT team could fit across the surface of a dime. Known as lenticular lenses, each lens covers a pattern of tiny colored image spots.

“Because of the magnifying effect of the lens, it displays the color from only one of the colored image spots, which is only a small portion of the entire area under the lens,” the authors write. Which magnified spot the viewer sees depends on the viewer’s viewpoint and the “different incident angles of the light hitting the lens” from that viewpoint. Multiple lenses together “form a lenticular display and collectively show an image that varies dependent on viewpoint.”

The MIT team’s editing tool takes as input the 3D model, or object, that is desired as well as the images that a designer wants to morph from one into another. It then calculates the lens placement and color pattern across the object’s surface that will lead to the desired effect. The designer can then preview the resulting object before sending the “directions” to a 3D printer, where the object, color patterns, and lenses are all printed in one pass.

Many Applications

The team went on to create four examples of 3D objects with images that change based on the viewer’s perspective. The idea was to demonstrate the ability to create objects with different geometries, different image complexities, and varying numbers of morphing images. The demonstration objects, each of which are printed with thousands of lenses, include:

—a kettlebell for exercise that can indicate to the user whether they are holding the device properly. If the device is too low, an arrow pointing up appears; if too high, an arrow pointing down takes its place. The correct position is indicated with a check mark.

—A bedside lamp shade that displays different greetings depending on whether a person is sitting up in bed (Good Day) or lying down (Good Night).

—A case for earpods that flickers between several colored stripes depending on how the user handles it.

—A shoe printed with a motivational message that only the user can see (the message is invisible to a bypasser or someone standing nearby).

The Sweet Spot

Zhu says that the major challenge involved in the work was determining the “sweet spot” for the size of each lens. “The smaller the lens, the better the resolution,” she says. “But smaller lenses also have more defects, so the fabrication quality will be worse.” Zhu and colleagues ultimately found that a lens that’s three millimeters in diameter “fits most needs.”

The team also tested other parameters, including the orientation of a lens (for example, printed facing upwards or sideways) and various post-processing techniques for the lenses. Lenses facing upwards and treated with painting varnish led to the best results.

It took about two years to move the project from concept to the creation of demonstration objects. The best part of the work for Zhu was “seeing that the process actually works and holding an object in my hand. That was extremely satisfying.”