Am I inside a video game? Introducing Pokemon Go for grown-ups

Lumin uses AR, debuts at DIA; Could be coming to a museum near you in the future

(Photo provided by the DIA)

DETROIT – What if, rather than just looking at a mummy, you could take an X-ray machine with you and see the skeleton underneath?

Or, rather than examining a museum artifact about the size of a cork -- what if you could hold up a device and unroll the “cork,” and be able to check out the details up close?

Recommended Videos



Well, now you can.

With Lumin, a brand new mobile tour at the Detroit Institute of Arts, visitors are able to explore things that aren’t really there. Yes, you read that correctly. It's kind of like being inside a video game.

Here's the idea behind it: When you’re taking in new information, have you ever noticed how it helps when you engage with the experience directly? Along the same lines, it’s why experts say you should take notes in class, rather than just listen to your teacher. It’s always more effective when you become an active participant.

When you engage in such a way, you automatically focus your full attention on the experience in front of you. So, what if the concept went beyond just taking notes in class?

That brings us back to Lumin, which uses augmented reality to let visitors explore different areas of the museum, and "see" digital objects that look so detailed and real, you'll be pinching yourself.

Could this be the future for museums, or modernize the experience? It’s certainly possible.

Lumin at the DIA debuts to the world Jan. 25, but members of the media were offered a sneak peek several weeks in advance of the public premiere.

Lumin has been compared to the location-based reality game Pokemon Go. Here's how the program will work at the DIA: You’ll be able to check out a smartphone from the front desk, specifically a Lenovo Phab 2 Pro, which is an Android. As long as you’re over the age of 18 and can provide a photo ID, you’re eligible to borrow one of the phones. Someone at the desk will show you how to use the device, but it’s really easy. Currently at the DIA, 40 are available. Each phone is hooked to a GoPro-type holder, with a handle attached for easier carrying.

You’ll receive the smartphone with the Lumin tour enabled and ready to go. You’ll start by looking at a basic Google map, showing six tour sites (although, the DIA said more will be added soon). When I visited Jan. 12, six tour stops were available.

Each stop is numbered. Just tap which one you’d like to see first -- you don’t have to go in order, and can pick your own pace, as well -- and the mobile tour will route you to the exhibit of your choice from wherever you’re standing. You follow these little blue dots that pop up on your screen, like a GPS. Your device lets you know once you’ve arrived. Easy enough.

You then look to the left side of your screen and tap a button to view augmented reality. In case you’ve never experimented with augmented reality (and I hadn’t), it’s described as the blending of virtual reality and real life.

Developers have created images within applications that blend in with contents of the real world. With augmented reality, otherwise known as AR, you’re able to interact with virtual items -- and you’re able to easily distinguish between the two. It's an interesting balance. You can definitely tell what's real and what's not, but the objects you'll see in AR are incredibly life-like and detailed.

So, now you know.

Once you’re in AR mode, your mobile tour will walk you through some prompts. At my first stop, I played with a matching-type game where I had to focus in on the details of an exhibit. Then, in front of a section of the wall from the Ishtar Gate, I used Lumin to walk through a digital reconstruction of the gates of ancient Babylon. This is where things got really mind-blowing, and it dawned on me just how much the mobile tour, and the DIA, could change the entire museum experience as we know it.

You’re using the smartphone as your “eyes” to examine the art. You’re engaging with the DIA’s collection, but rather than being on a physical tour, where pointing out some facts and figures will resonate with some visitors but maybe not others, you’re doing it at your own pace.

If you want to poke around the Ishtar Gate -- and you WILL want to -- you can spend as long as you’d like (within reason, of course!) And when you’re done, you just hit the X in the corner of your screen. It’s very simple to navigate from piece to piece.

“When you’re viewing (the Ishtar Gate) and the mummy, you’re surrounded by virtual objects,” said David Lerman, founder and CEO of GuidiGO, a key partner in creating the project and making it a reality. “So we worked with video game developers. It feels like you’re in a game.”

That’s a perfect way to describe it.

“You’re in the other side of the screen, like a video game,” Lerman said. “But instead of looking at the screen, you’re inside the screen.”

The tour really impressed me: How much I could see, do and experience. As I walked around the Ishtar Gate, I got all the way up to the front to check out all the different views. And then Richard Scott, the director of IT for the DIA, reminded me to look up. That’s what you have to keep in mind: AR goes beyond what you can see right in front of you. You can look all around: up, down, to the front and to the back. When I panned up with the phone, I saw a gorgeous blue-tiled ceiling. Depending on which direction I walked, I saw several different vantage points and perspectives.

I will say, I kept wanting to zoom in with my fingers, kind of like what I’m used to doing on my iPhone. But for the most part, it didn't work. With AR, it makes more sense to approach the objects, treating the technology like a magnifying glass. It’ll definitely be your instinct to zoom in on objects like you would on your phone, but that won’t jive for many of the exhibits. It's just a different way of thinking.

As I mentioned a little earlier, members of the media got to experiment with the technology in advance of the public launch. In addition to Scott, I walked around with DIA interpretive planner Megan DiRienzo.

I’ll share some other notes from tour stops that really impressed me.

When we looked at a beige limestone sculpture featured on the mobile tour, I was able to tap my screen to show the original vibrant colors that adorned the Assyrian palace thousands of years ago. Sometimes it’s easy to forget that there ARE original colors, you know? Also, the DIA consulted with experts, so the colors were chosen based on science.

In the Egyptian gallery, that's where I held my device up to a 2,000-year-old mummy, to reveal an X-ray view of the skeleton inside. As DiRienzo pointed out, the museum features a small photo sitting inside the case next to the mummy, showing the skeleton. But what are the chances you’re going to crouch down and take a look? Having a handheld device with which to circle and examine the mummy really brings it all to life.

You can also tap touchpoints of the skeleton to learn more. It's surreal: You're interacting with digital objects.

And that’s where AR proves how perfect it is for this project: The AR aims to overlay videos, photographs, sounds or touch-activated animations on your screen. Those provide contextual information on the art -- such as how an object was initially used, its original location, or details not normally seen by the public.

We also used AR for extra perspective on the DIA’s kilga.

Are you familiar with the term? (I wasn’t, either).

A kilga is almost like a water jar and a stand used many, MANY years ago. Of all the exhibits the DIA has to offer, it’s safe to say the kilga doesn’t draw as many crowds as some other parts of the museum. 

So, showcasing the kilga on the mobile tour takes visitors to an area they might not otherwise see.

Also, the AR view of the kilga really shows its function. When you compare the two views side by side, you can picture what the kilga was used for and how it worked.

The exhibit all comes full circle when you add together the view in front of your eyes and the AR view, providing extra context and visuals.

And finally, if you’re local to the Detroit area, you should definitely go look at the tree stump on the mobile tour. It was made by an African artist. Look at this photo below for an idea of what you see with and without AR.

With AR, you can peer inside the stump and really take a look at all the intricate details that went into the piece. I’ll admit, having DiRienzo as one of my tour guides really paid off at this exhibit.

“Being able to carve something from the inside is (the artist's) signature,” DiRienzo said. “The bird reference on top is a king. (The piece is) just packed with so much information and meaning. This is beautiful.”

DiRienzo’s passion for the art showed in the way she used her hands to describe each piece, or the way her eyes lit up when she’d get on a roll, talking about a particular stop on the tour.

But that’s the cool thing about Lumin. To view the art with DiRienzo provides a real insider’s perspective, don’t get me wrong. But still, to have the mobile tour at your fingertips, you’re still able to learn and see so much more than you would normally take in during any other museum trip. It’s almost like carrying an expert around with you. As much as I learned from DiRienzo, I learned from Lumin (well, maybe almost!)

The museum’s curators helped decide which pieces to feature.

The DIA wants to move past the six exhibits, but museum officials also want to be selective, and pick and choose what they’ll feature in a thoughtful way. Based on how much content there is to work with, the sky is the limit. Still, DiRienzo and Scott stressed how much they’d like to strike a balance, and they don’t want to overdo it.

The tour took about six months to put together, Scott said. Right now, developers are experimenting with adding sound.

The technology behind Lumin is fascinating. The mobile tour was developed by the DIA in partnership with Google and GuidiGO.

Lumin uses Google’s Tango technology so that visitors can engage with the DIA's renowned collection.

It all started in late 2015 when Scott heard about the Tango technology, and he kept it on his radar. In early 2016, GuidiGO said the company could partner with Google for Tango. The DIA confirmed it was interested in the program and development took off from there.

It was perfect in a way, because Lumin represents what the DIA is all about: “It’s supportive,” DiRienzo said. “It’s about accessible interpretations. You don’t have to be an expert. That’s our whole thing.”

The goal is to connect people with art.

Added Scott, “(And) the animators with GuidiGO (were) incredible.”

The DIA is the first art museum in the world to integrate this 3-D mapping and smartphone augmented reality technology into a public mobile tour. The product manager with Google spoke with excitement about the collaboration, as well.

“The Tango tools enable the curators to tell a story within the museum (and) show the visitors new things about the artifacts that mesh with their experience,” said Justin Quimby, a senior product manager who worked on the project with the DIA and GuidiGO. “The content is important. Tools alone are not sufficient.”

And for anyone who hasn’t had a chance to visit the DIA, just know: As for content, the museum has plenty.

Quimby said he loved his experience at the DIA when he visited, and he really enjoyed working on the project with all the different key players. Lerman echoed the same sentiment.

Quimby described Tango as the hardware and the software behind the program. It’s what enables the smartphone to know where it is inside the museum, and it helps anchor the digital objects. For example, when you consider the 3D model of the mummy, which has the X-ray and CT scan information overlaid on top, Tango makes it so that the digital object occupies the same space as the physical exhibit.

Tango is a new capability for a smartphone. Quimby broke it down like this: 15 or 20 years ago, if you were asked, “does your cellphone have GPS and a really good camera?” You’d probably say no. It just wasn’t as common to have those nicer features on your phone -- features that today, we consider day-to-day, basic tools. But if you got a new phone tomorrow and you didn’t have solid GPS or a good camera, someone would probably question why you got the phone in the first place. With Tango, this marks the very beginning of smartphone augmented reality.

When the final tour came together, everything made sense in a seamless way, just as the developers had hoped.

“(Like), of course there’s this overlay on top of the mummy,” Quimby said.

And at that point, it’s almost like the app has gone away, and the experience comes to the forefront.

“It’s thrilling for me to see,” Quimby said. “And now, the subject matter experts can tell their story in a new way.”

Lerman also described working with Google and the DIA as a “real pleasure.”

“They understood the technology fast,” he said of the staff in Detroit. “(The DIA) was particularly ambitious."

When I asked the DIA about the possibility of field trips, or if students would be able to use the devices -- remember, at this time, you have to be 18 or older -- museum officials said they’re first going to host a “teacher’s night” type of event, where instructors can go on the tour and provide feedback as to how these devices might be used best as an educational tool.

As for the name of the project, it’s derived from the Latin word for light (lumen), and refers to the moment of illumination — the spark and magic — that takes place when people have an enlightening experience with a work of art.

So from here, the DIA will serve as the fearless leader for other museums that might want to follow suit. The museum is testing the Lumin prototype and will ask for user feedback, to learn if the tours are working as planned and if adjustments need to be made.

As I indicated earlier, I did get to go on the tour with two experts -- lucky me! -- but I also mentioned, I think it’s safe to say anyone could figure out the tour, even people who aren’t very tech-savvy.

With Scott and DiRienzo, I was able to gain insight into how the project came about, but even without them, there’s probably nothing groundbreaking on the mobile tour that I would have missed out on. I feel confident when I say I could have poked around or maneuvered the device just fine on my own. Sure, I’m 30, I own a smartphone and write for local news websites, but I’m not particularly tech-inclined or over-the-top knowledgeable. I think you'll fall into a bit of a rhythm on the tour, directing yourself to each piece and figuring out what to look for as you follow the prompts on your screen. I’ll go out on a limb and say yes, you can bring your grandma!

In summary: Sometimes it’s hard to tell when you're visiting a museum -- unless you’re on a guided tour -- what each piece really symbolizes or means, without having someone there to provide context or keep you focused. But with Lumin, you can explore the work on a deeper level. This is a way to REALLY interact with your surroundings. And it’s engaging. You can get lost in it without leaving the building. If you’re ever at a museum or a spot that offers Lumin or something like it, give it a shot. You won’t be disappointed.