Project Brief

A collaboration project with the Davis Museum at Wellesley College to create a HoloLens application, called HoloMuse, which allows a user to explore artifacts and curate their own exhibit using 3D scanned archeological artifacts. We designed HoloMuse to facilitate learning and engagement with museum collections without taking away from the experience of viewing an original artifact within the context of a classroom or exhibit. HoloMuse can be used inside the gallery or in the classroom. It enables users to pick up, rotate, scale, and alter a hologram of an original archeological artifact using inair gestures. Users can also curate their own exhibit or customize an existing one by selecting artifacts from a virtual gallery and placing them within the physical world so that they are viewable only using the device. We intend to study the impact of HoloMuse on learning and engagement with college-level art history and archeology students.

Problem Statement

Artifacts are restricted to certain spaces, usually museums, which makes it hard to study them in the classroom setting. There have been attempts to bring artifacts into the classroom setting through online websites such as SketchFab and using model replicas. However, there is still a lack of ability to actually bring the artifacts into the classroom and learn about them.

This project aims to solve this by bringing artifacts into the classroom setting with Augmented Reality, by using the Microsoft HoloLens. We envisioned an application that allows a user to select a 3d model representation of the object from an inventory. Following this selection, the user can use air gestures and voice commands to manipulate the object. Through the application, the user can scale, rotate, and move the object in the space around them. They can also remove the outside paint layer of the object so that they can inspect the object's wear pattern, request to see more information, reset the size of the object, and delete it.

Figure 1. The first iteration of the application: A user opening a virtual gallery to select an object to place in the space and a user customizing an exhibit by placing an artifact in the physical world (a museum)

This project brought forth new problems for User Experience design since there is no pre-existing research on how users interact in holographic spaces. This made designing the interface a new challenge and an exciting opportunity to test with users and gather qualitative feedback at many stages of the development project.

Figure 2. Sketches of the application after the first iteration

Figure 3. Screenshots of the second iteration of the application

Evaluation

Once in initial development, we tested the application with multiple pilot studies. We gathered qualitative data from these studies, which was translated into design decisions, and used to improve the application. The main feedback we got from users was that the application did not provide enough information at all times (internal locus of control) and relied on users remembering what they had to do. Furthermore, due to the technology being so novel, this resulted in users being very overwhelmed, constantly confused, and unable to complete tasks. To fix this, we changed the interface so that it reflected more information about the objects on the inventory and a side menu so that the user could directly manipulate the objects by navigating each object's menu, rather than remembering the specific voice commands.

Figure 4. Demoing the HoloMuse application at Wellesley College and TEI 2017

Once the interface had reached its final iteration, we tested it against two other technologies that can bring artifacts into the classroom setting (online 3D modeling manipulation software and 3D prints of archeological objects). This study was an in-lab study with 60 participants in total, 20 in each condition. Each participant was asked both objective questions about the artifact they explored and subjective questions about their experience. The objective questions were then evaluated by an expert in archeological studies and the subjective questions were analyzed using NASA-TLX. The results of this study will be published in the TEI 2018 paper.

We are currently extending the project to also augment the museum experience in the form of a guided museum tour. A user will be able to use the HoloMuse application within the museum to have a "guided" tour and learn more about objects by viewing information through different mediums (audio, video, text).

Publications/Conferences

C. Pollalis, E. Minor, Lauren Westendorf, W. Fahnbulleh, I. Virgilio, A. Kun, O. Shaer. Evaluating Learning with Tangible and Virtual Representations of Archeological Artifacts, Paper, Proc. TEI 2018 Tangible Embeded Interfaces. (28% acceptance rate)

C. Pollalis, W. Fahbulleh, J. Tynes, O. Shaer. HoloMuse: Enhancing engagement with Archeological Artifacts through Gesture-Based Interaction with Holograms, Proc. TEI 2017 Tangible Embeded Interfaces. (45% acceptance rate)