When museums think about remote immersive experiences, Virtual Reality (VR) is usually the first technology that comes to mind. While we have created VR experiences in the past – such as an entire exhibition for the Carnegie Museum of Natural History – VR did not seem to be the best fit in this instance. The education team’s brief included the following stipulations:

• Immersive life-size experience
• No application to download
• Usable instantly by anyone on any mobile device
• Compatible with computers and tablets
• Photorealistic rendering

Three years ago we would have replied: “Sorry, this is not possible.” But Mobile AR has changed the game and is now able to tick all these boxes.

When iOS and Android adopted .USDZ and .GLB formats in 2018, it became possible for more than one billion devices to display an AR experience directly inside a mobile browser.

• .GLB is a 3D file format that describes an object according to its surface, with meshes, materials and textures. .GLB is the bundled version of the .GLTF format which was established by the Khronos Group, and is supported by Google, Microsoft, Facebook, and Adobe.
• .USDZ is another 3D file format developed by Pixar and Apple. It’s the only AR file format compatible with Apple devices.

Let’s put it this way, .USDZ and .GLB files are to AR what .PDF files are to documents. They display 3D objects, in AR in your mobile browser (Chrome or Safari), in 3D in your computer browser (similar to Sketchfab).

If you would like to see for yourself what mobile AR looks like, try the following:

• On your mobile device, google one of these words: Tiger, Apollo 11, Neil Armstrong, Chauvet cave, or Tyrannosaurus.
• Tap the “view in 3D” icon, next to the relevant thumbnail.

More AR search examples are available here.

10 exhibits in 3 weeks to be AR ready

The NHMU Education Department requested this AR functionality for the beginning of the school year. Museum staff had already scanned their exhibits in photogrammetry, including a selection of paleontological items, bird skulls, and artifacts of Native American culture.

The GuidiGO AR team used GuidiGO Studio and AR Composer platform to quickly optimize the mesh and textures for Android and iOS devices and make sure the quality of the 3D models was consistent especially in low light environments.

We are thrilled with the result. Check it out for yourself by displaying the baby mammoth skeleton in your room! We are even more excited to hear about the impact of our work after receiving this kind note from Tracey Collins, Interim Director of Education at NHMU:

NHMU was thrilled to partner with GuidiGO to deliver such innovative science education and immersive learning experiences to 25,000 students in Utah this school year. We have received great feedback so far and are looking for new ways, Museum-wide, to incorporate AR functionality into all of our virtual opportunities.

To learn more about GuidiGO AR, visit www.guidigo.com/AR

Other example by GuidiGO – Tap on the image below on your mobile to discover this famous masterpiece in AR, in real size:

This AR experience is part of “Launchpad: Apollo 11 Promises Kept“, a new 10,000 sq. ft. exhibition paying tribute to the past, present and future of space exploration. The exhibition combines the Museum’s significant collection of Apollo-era artifacts with cutting-edge interactive experiences to create an immersive and exciting cosmic journey.

Photo by Fort Worth Museum of Science and History

What is unique about AR Composer is that digital objects keep their original positions, creating persistent AR experiences shareable across several devices.

To position digital artefacts accurately, AR Composer comes with an easy-to-use 6-DOF (Degrees Of Freedom) Editor. 3D objects can be moved from left to right, up and down, forwards and backwards, including three rotation options. With this unique feature, objects can be placed accurately in any space, in no time.

Discover how to try AR Composer at the end of this post.

AR Composer also comes with 2 modes: Creator and Visitor mode. In Creator mode, the museum team can create as many persistent scenes as they want, placing and anchoring their objects wherever they like in the physical space. When this process is finished, the museum simply has to lock the scenes in Visitor mode to make them available on any device running AR Composer. Visitors can then use the devices provided by the venue or their personal device to enjoy the content.

Another perk of AR Composer is that there is no limit to the number of 3D assets that can be uploaded, which gives digital and education teams at museums the opportunity to design scenes with the creativity of a theatrical production. Objects can be animated, interactive and can display data to showcase digital storytelling or contextual information.

There are a number of creative ways to use the app. Examples include:

• Sharing works that are missing from a collection, whether they are in another museum, being restored, or on loan.
• Rebuilding archeological monuments – for example, making an entire temple appear where only an ancient column remains.
• Comparing works to previous versions, or to the artist’s preliminary sketches.
• Taking complex mechanisms apart and then putting them back together to see how they work. Examples include machines, the human body, the solar system, etc.
• Making digital characters move around in front of the visitor.
• Recreating the decor and atmosphere of an entire room in real size, by superimposing its former or future state on top of its current reality.
• Adding flesh and skin to the skeletons of extinct species, e.g. dinosaurs.
• Creating digital collaborative artwork.
• Creating a portal to access a virtual world.
• Creating a digital character guide to provide specialized tours for children, teens or other visitor groups.

AR Composer saves museums from having to build their own customized application and CMS (Content Management System) from scratch. That way, the majority of the budget can be allocated to creating content and designing compelling visitor experiences.

Try AR Composer now! We are offering AR Composer to museums and heritage sites in private Beta, with several demo assets included. If you are interested, please contact us.

With the addition of the full AR app, MUSE is offering visitors an experience that harmonizes perfectly with their innovative spirit – in total fluidity and transparency.

A Fascinating Content

The app has what it takes to please even the most directionally-challenged of us. The days of losing your way in exhibitions are over: with Go!Muse visitors can see their precise location on each of the six floors of the museum, and are guided in Augmented Reality toward the object of their interest.

As for the educational content, the app includes over one hundred AR hotspots that can be activated by the visitor to unlock multimedia content (text, videos and more) that provides additional context for the various attractions.

Last but not least, the app makes it possible to view realistic virtual portrayals of ten dinosaurs and a whale by adding life-size animated bodies to their skeletons in Augmented Reality, right in the middle of the museum.

An Emotional Experience

On the day of the launch, over one thousand visitors came with their families to check out the new app. It was the ideal opportunity to confirm that the app would be as magical as we had hoped. It was touching to see children, parents and grandparents equally surprised and enthralled by the illusion of a partially-digital reality. No matter their age, every visitor had the happy face of a child thrilled by an amusement park or magic show.

A Synergy of Multiple Talents

The obvious joy of the visitors made me proud of the work our team had accomplished. This project took no less than six months to complete and incorporated the work of at least a dozen individuals: an illustrator, a designer, a 3D animator, a game developer, front and back-end developers, a project lead and a program manager. The synergy of all these talents makes it clear that our field has changed. Cultural mediation is no longer about making a connection between historical and scientific facts, or simply coming up with a captivating narrative. It is also about creating an unforgettable emotional experience for every type of visitor.

To meet this objective, we collaborated with MUSE’s digital department. Fabio Manucci, the paleontologist of the museum, worked closely with Davide Bonadona, an illustrator celebrated for his work on dinosaurs, to create a scientifically accurate portrayal of each specimen.

From a scientific point of view, there is always a risk when you attempt to bring to life creatures that have been extinct for millions of years. But museums also have a responsibility to provide accurate representations of dinosaurs, in contrast to the fantastical portrayals in cinema and video games.

We then made it possible for these virtual animals to come to life inside the museum. Each 3D model was superimposed on its actual skeleton to within a centimeter of precision using GuidiGO AR Composer.

Seeing these dinosaurs in the flesh – and not just as bones – is impressive enough. But watching them move before your eyes makes the experience even more stunning. Since the skeletons themselves obviously remain static, we had to find a way to suspend disbelief when the animals are in motion. To do so, we added a Virtual Reality mode that hides the skeleton when the dinosaur begins to move.

A Large-Scale Experience

With the launch of Go!Muse, MUSE has become the first museum of Science in the world to offer an Augmented Reality experience at this scale, with ten animated dinosaurs and a whale as part of their permanent collection, and 80 AR-compatible tablets available to visitors.

MUSE official video trailer

While the app is designed for a wide range of users of all ages, it is sure to capture the imagination of young visitors most of all, due to their interest in novelties and interactivity.

Since they’ve been immersed in the digital world from birth, the younger generations are already used to learning and discovering outside of the traditional learning environments of libraries and museums. Touchscreens are already part of their daily lives. Virtual Reality headsets are soon to follow, and any museum planning for the future should be aware of that fact. It is essential for cultural institutions to pursue immersive, innovative technologies if they want to form lasting ties with the adults of tomorrow.

Experiences like this one will help young ones to discover the magic of museums and the joy of interacting with works of art and science along with other visitors.

Singapore’s oldest museum now allows its visitors to experience the building through an enhanced architectural tour. It tells the story of the building’s long history and its transformation from a Natural History Museum to the most significant historical museum of Singapore.

Visitors are given a Tango-compatible Lenovo Phab 2 Pro device, which will transport them back in time and enable them to see the architectural evolution of the building, how it looked like over a century ago and onwards, and the artefacts that used to be displayed in it. The digital tour, led by volunteer guides, evokes the museum’s imagery in a way that helps visitors obtain deeper understanding of its contents.

There are currently six tour stops that incorporate a range of AR experiences, such as:

The Building’s Evolution

The evolution of a 130-year-old building is bound to be an interesting one. Combining Tango technology’s ability to both augment and recreate reality, visitors are able to see the replica of the museum’s transformation throughout the decades.

The Museum’s Rotunda

The old rotunda used to display a natural history collection, as well as a marble bust of Sir Stamford Raffles, the founder of Singapore. Through the lens of their device, visitors can walk around a life-size model of the room, surrounded with the original artefacts that once graced it, and see the clay tiles instead of the current marble flooring.

The Indian Fin Whale

This iconic 42-feet-long skeleton used to be displayed in the museum in the early 1900s, until it was returned to Malaysia in the 1970s. Thanks to Tango technology, visitors can once again see its fossil under the glass passage on the second floor, where it once stood. However, that’s not all – they can also witness the reimagining of how it looked and sounded like while it was still alive.

Lumin content was created by the DIA’s interpretive team leveraging GuidiGO’s new augmented reality platform, AR Composer, built specifically for Tango. This allows the user to interact with real size 3D animations, information and display directions in the visitor’s field of view.

Lumin connects visitors with artworks in a global context beyond the DIA,” said Salvador Salort-Pons, DIA director. “Augmented reality allows the user to see the unseen, imagine art in its original setting and understand how objects were used and experienced in people’s everyday lives. It is an exciting way to incorporate the latest technology into the visitor experience.

Visitors will hold a Lenovo Phab 2 Pro, the first Tango enabled phone, to explore different works of art. The initial phase of the project includes seven tour stops with a variety of AR experiences, including:

A Mummy, from Ancient Egypt

Mummies give us insights into ancient burial rituals. With the app’s AR overlay, you can peer beneath the sarcophagus and even the mummies’ bandages to explore an X-ray-like view of the skeleton.

The Ishtar Gate, from Ancient Babylonia

The Ishtar Gate once stood six stories high. Now, with the power of Tango’s motion tracking technology, you can explore the gate at scale, walk under its arch, and see where the Detroit Institute of Art’s 3×4-foot mosaic piece fits into this architectural wonder.

A limestone Relief, from Mesopotamia

The limestone relief was originally painted with vibrant colors, which have long since faded. By looking through your Tango enable device, you can tap on the relief to see what it would have looked like when it was first created thousands of years ago.

A Cylinder Seal, from Mesopotamia

Small round cylinder seals were once used to make impressions on clay for items like jewelry and signatures for administrative purposes. Now, instead of just seeing the seals in a case, you can roll the seals in AR to see the written characters and figurative scenes that they create.

WANT TO SEE LUMIN IN ACTION? CLICK HERE TO WATCH THE VIDEOS.

This is just the beginning of how you’ll be able to use Tango in museums to see more, hear more and learn more – in other words, to experience museums around the world in a whole new way.

“Lumin” in the Press:

• Hyperallergic: More Screens, More Knowledge: Testing the Detroit Institute of Arts’ New Augmented Reality App >>
• Fortune: Google Wants To Give Museum Visitors A Pokémon Go-Like Experience >>
• Venture Beat: Google opens Tango augmented reality platform to museums, starting with Detroit Institute of Arts >>
• The News Herald: Augmented reality adds dimension to art at DIA >>
• The Economist: Futurism: museums and technology >>
• Geek: Detroit Institute of Arts Enriches Museum Experience with Google Tango >>
• Tom’s Hardware: Google’s Project Tango Heads To The Detroit Institute Of Arts >>
• CNET: Google’s project Tango AR lets museum visitors delve deeper >>
• Tech Crunch: A Detroit art museum is leveraging Google Tango for an AR history lesson >>

The RMN-Grand Palais won first place in the “Virtual and Augmented Reality/Immersion” category for their exhibition entitled “Sites Eternels“ (Eternal Sites). We are particularly pleased with this news as GuidiGO created the Tango app used for this exhibition in collaboration with the Google Cultural Institute.

Using 3D images from the startup Iconem, the app we developed allowed visitors to explore a reproduction of the ancient city of Palmyra in augmented reality on Tango tablets, and to witness the destruction of its arch piece by piece, along with a simulation of its reconstruction.

This world premiere of the augmented reality experience with Google Tango was offered for free to the Grand Palais visitors, highlighting the latest technological advancements and applying them to the preservation and dissemination of our cultural heritage.

GuidiGO AR Composer opens up incredible new opportunities for museums and cultural sites, as they can now design their own AR experiences in just a few simple steps, and make them available on site or for remote use by schools and other educational institutions.

Unlike virtual reality, Augmented Reality is not designed to immerse you in an entirely different environment” says GuidiGO CEO David Lerman, “AR integrates virtual elements into the real environment, and allows you to interact with them as you move around. You can physically feel the presence of AR objects, particularly when they appear in real size. The applications for Augmented Reality are actually quite different from virtual reality, but no less fascinating and engaging for users. Just look at the excitement around Pokémon GO!

This is a great opportunity to reach an even wider audience, including the digital generation, who are looking for new experiences. AR Composer is also an extraordinary educational tool capable of engaging different types of visitors as they explore permanent collections or temporary exhibitions.

GuidiGO AR Composer: How does it work?

GuidiGO AR Composer makes it possible to create experiences in augmented reality using virtual components. The Tango tablet works as a viewfinder to position virtual objects on the ground or anywhere else, with no need for markers or signals of any kind. The settings and location of the virtual objects are saved as soon as they are placed. They are then visible to visitors just like any real object.

A virtual 3D object can be static, animated, or interactive. It can be a character, an entire scene, a sound, or a visual hotspot, depending on the experience created by the museum.

GuidiGO AR Composer makes full use of Tango capabilities, including motion tracking, which enables virtual objects to behave as if they were real. For example, virtual objects increase in size as you approach them, and you can walk around them to get a closer look from any angle. It is also possible to interact with them, as you can see in this video where a dinosaur roars when the visitor approaches.

Ready-to-Go AR Plugins

Just like GuidiGO Studio, AR Composer is extremely user-friendly and doesn’t require special skills or training. Using presets or custom plugins, museums and cultural sites can quickly create a variety of experiences in which their visitors will see the real and virtual blend together seamlessly. For example, museums will be able to:

• Geolocate and guide visitors using a virtual path that appears at their feet:
  
• Leave comments about a place, or work of art, using AR hotspots:
  
• Create a Pokémon GO-style treasure hunt, inviting visitors to explore and collect virtual objects they find in the galleries:
  
• Share works that are missing from the collection, whether they are on reserve, being restored, or on loan… or simply too fragile or too large to be displayed in public,
• Place a work in its original context – for example, making an entire temple appear where only an ancient column remains,
• Compare a work to its previous version, or to the artist’s preliminary sketches,
• Take complex mechanisms apart and put them back together again to see how things work, whether it’s a machine, a process, the human body, the solar system, etc.
• Make virtual characters move around in front of the visitor,
• Recreate the décor and atmosphere of an entire room by superimposing its former or future state on top of its current reality:
  

And any other applications imaginable…

First Tango Smartphones Available

The Phab2 Pro, the first smartphone to feature Tango technology, is available at the Lenovo shop.

This new type of smartphone is the first step in a major technological breakthrough. Tango makes it finally possible to create AR experiences without using markers or any kind of infrastructure. In other words, there’s finally a mobile device that sees like we do.

I caught myself wondering what the magic was about parks that made children never want to leave…and why they don’t tend to feel the same way when we take them to the museum.

The answer might seem obvious: at the park, children can play. But what is the psychology behind why they often prefer a fun excursion to the park over a visit to the museum? In my opinion, it all comes down to four key components:

Interaction, Emotion, Accomplishment, Emulation

These are the components that come into play when a child seeks the thrill of going down a slide, or pushes to come in first, or tries to climb higher than they have before. These also come into play when a child encounters a new environment, builds something with friends, or challenges them. All of these activities present the opportunity for children to feel joy and pride, and receive praise from their parents.

Is it possible to activate these motivators at the museum?

What can we do to make a visit to the museum just as exciting for our children as an afternoon at the park? A museum isn’t a playground; it doesn’t provide quite the same freedoms that a park or garden does. However, some museums do offer engaging and thrilling experiences for children – experiences that address the psychological motivators mentioned above.

But things can get messy when you decide to visit a museum with your family on a whim. As a parent, I’ve often been disappointed by the lack of “on demand” solutions for capturing a child’s attention and helping them to have fun in cultural environments. While science museums tend to be very imaginative when it comes to engaging young visitors, the same can’t be said for all museums. In most cases, it’s best to plan your visit in advance to make sure appropriate activities are available for children. Otherwise, you may be left to your own devices on subjects you may not necessarily be familiar with.

Digital devices to the rescue?

This is where digital tools on tablets and mobile phones come in handy. They provide an “on demand” solution with great possibilities, they offer content that can be adapted to many audiences depending on age level, skills and interests.

Digital tools provide quick solutions that are likely to activate all four motivators we’ve discussed above — turning museum visits into an experience your child will want to do again and again, and giving parents tools they’ll be excited to talk about with their friends. Here are a few examples:

INTERACTIONS

This is, of course, where the digital tablet comes in. A tablet should never be the focal point of the experience, but rather, a starting point. It should foster conversation between you and your child and facilitate interaction with the works of art and other artifacts on display. This is why digital games that combine storytelling and puzzles are the perfect tool for facilitating that interaction — they engage children and make them active participants in the cultural experience.

EMOTIONS

Digital tools can intensify the emotions children feel when they see a work of art or visit a heritage site. First of all, when they listen to a story, they are transported into the world of the imagination. In order for this to happen, you need a passionate narrator who not only tells the story, but brings the characters to life.

Digital tools offer infinite possibilities that are both fun and exciting for your child. For example, with augmented reality, your child could meet a historical character, dress a friend in a medieval costume, or pet a dinosaur… and of course, take photos to capture all the fun! Your child could also have a memorable cultural experience in virtual reality, which has been shown to have a particular emotional impact (see our article on VR here).

In short, digital tools bring you the best of new immersive technologies that can transport your child straight into another time and place.

ACCOMPLISHMENT

A digital visit can also foster a sense of accomplishment in several different ways. First, they provide rewards when the children earn points, collect objects, or look for virtual clues. Games also encourage them to keep progressing, for example, when they make it to the next step in a game because they were able to complete a challenge.

Gaining knowledge also contributes to that sense of accomplishment. When children learn something from a story, a quiz, or funny anecdote, they feel like they’re the only one who knows this special information, and they feel proud of what they know.

Another way to activate this motivator is to encourage creativity by asking the child to solve a complicated puzzle or complete an artistic challenge.

EMULATION

Games should be as varied as possible. That way they present a variety of challenges based on listening and observation (for example: searching for clues in the décor or in a work of art), questions that require careful reflection, or fun tasks (like taking a selfie posing like a character in a painting).
Games should also allow the child to accumulate points and/or objects to stimulate a healthy sense of competition. Children can then share their scores with friends.

These four motivators also have a biological basis

This empirical approach also has a more scientific explanation. Each of the motivators mentioned above also release different hormones that contribute to feelings of joy and happiness: dopamine, serotonin, oxytocin and endorphins.

• Dopamine produces the feeling of satisfaction we get when we earn a reward, meet an objective, or experience a good surprise.
• Serotonin triggers the feeling of self-esteem and self-confidence. It is released when a child feels proud.
• Oxytocin is released during social interactions; it produces feelings of attachment, connection, and sharing with others.
• Endorphins produce feelings of well-being, related to laughter.

Besides their emotional virtues, these hormones also facilitate memorization. Dopamine, for example, strengthens episodic memory, which allows us to relive scenes from the past.

When to start?

Today’s children are immersed in an environment that is both digital and physical. They see technology as a means to foster interpersonal relations and connect with both the reality and fiction of others’ lives.

When digital tools are used intelligently, they help museums and heritage sites to create strong ties with young visitors, allowing them to discover and enjoy the many treasures these sites have to offer.

Whether digital tools should be used in museums and heritage sites is no longer a question. The question for museums to answer is “How can we adapt digital tools to the particular needs of each museum and heritage site?”

In practice, these solutions require an infrastructure that is often more cumbersome and more expensive than expected, and the results are sometimes disappointing (see our article on Tech Trends). Taking into account the limits of such technologies, GuidiGO decided to explore a different path by making our app compatible with Google Project Tango. This technology offers an incomparable advantage: it works without any infrastructure. Yes, you read that correctly: no infrastructure at all! No need for an internet connection, WiFi, sensors, or GPS. Museums have nothing to install and no equipment to maintain!

David Lerman, GuidiGO’s CEO, explains the magic behind it: “The tablet is loaded with sensors that detect motion, volume, and depth. A 3D map of the interior of the museum is created in advance by scanning the walls, partitions, installations, etc. The tablet later recognizes these elements and makes it possible for the visitor to remain continuously localized, with no need to connect to any signal.”

But that’s not all, not only do visitors know their exact position, they can also opt for a 3D view with an AR path superimposed on the floor…

… or choose a dual view with the 2D map:

Would you like to see a live demo? Take a look at our app running in this video shot at the beautiful Museu Nacional d’Art de Catalunya, during the recent MWC 2016 (Mobile World Congress) in Barcelona. The GuidiGO app was presented to the press in the presence of teams from Google Project Tango, and Lenovo – the manufacturer of the tablet.

Besides being a game changer for indoor positioning, Project Tango’s ability to recognize the user’s environment opens the door to numerous AR applications. At the Museu Nacional d’Art de Catalunya, we were excited to present the interpretation of a painting in Augmented Reality. Above a 33-foot-long canvas (“The Battle of Tetuan” by Mariano Fortuny), AR 3D markers appear in the air, indicating that additional content is available to explain a detail in the piece.

 
And this is only the beginning… Stay tuned!

Learn more:

• Lenovo: “How Project Tango Will Change the Way You Use Your Phone”
• Tom’s Guide: “How Google’s Project Tango Won Me Over with AR and Art”
• Digital Trends: “Digital Trends Top Tech of MWC 2016 award winners” (Read: Lenovo’s Project Tango)
• Digital Trends: “Touring a museum with Project Tango is an empowering glimpse into the future”
• Slash Gear: “How Lenovo and Project Tango will reinvent your smartphone”
• CNET: “Project Tango could come to a museum near you”
• BGR: “Lenovo and Google are about to blow the lid off digital indoor mapping”
• The Verge: “Google just showed me the future of indoor navigation” (watch GuidiGO app at work in the video!)
• PC Mag: “A Night at the Museum With Project Tango”