The Cleveland Museum of Art and ArtLens Exhibition
The Cleveland Museum of Art (CMA), located in Cleveland, Ohio, USA, is renowned for the quality and breadth of its collection, which includes more than 61,000 objects and spans 6,000 years of achievement in the arts. As an international leader in innovative, iterative digital initiatives, CMA opened ArtLens Exhibition, in September 2017. ArtLens Exhibition represents one component of ARTLENS Gallery which also includes ArtLens Studio, ArtLens App, ArtLens Wall, and the Digital Beacon. Using gesture-sensing, eye-tracking, and facial-recognition technology, ArtLens Exhibition aims to provide visitors with the toolsets to understand how to look at art in any context.
The latest iteration of ArtLens Exhibition opened in June 2019 with 21 new masterworks of art from across the CMA’s permanent collection. An exciting new feature of this installation is the incorporation of a 3D model viewer into the existing projection interactives. For the first time, we used photogrammetry to create 15 3D models that visitors can manipulate and zoom using gesture-based Kinect technology. The CMA’s Digital Innovation team led this new initiative and the 3D models were created by CMA’s Chief Photographer, Howard Agriesti. Visitors can explore artworks in CMA’s collection like never before—turning them digitally around to see the tops, bottoms, and insides. As an inherently iterative space, ArtLens Exhibition seeks to push the boundaries of innovation, providing visitors with the tools to look closer at art. We pursued this feature because feedback from visitors and CMA staff indicated this was a desired experience of the interactives. After visitors engage with the 3D model, they can immediately play the artwork’s next assigned game to learn more and dive deeper into the object.
After researching several different approaches to incorporating our 3D models into the games, we decided to integrate the Sketchfab platform into our existing interactives. The decision was largely based on the sophistication of the interface and level of control for modifying and improving the models.
Artwork Selection Process and Photogrammetry
As the collaborative Digital Innovation team worked together with curators to select the artwork for the reinstallation, it was our goal that all 3D artworks would be photographed using photogrammetry. Photogrammetry is the process of using specialized cameras to capture a “point cloud” representation and texture map of a 3-dimensional object and recombine them as a 3D model. This concept was new to the institution and represents another way in which CMA continues to advance what is possible in the museum field. The previous iteration of ArtLens Exhibition used 360 photography for the “attract loops” in the projection interactives. The attract loop is the initial animation that plays with the intention of drawing the visitor in to interact with the game. Photogrammetry represents the next level of innovation. All 3D models were produced in our museum’s photo studio and we began the process over a year before ArtLens Exhibition reopened in June 2019.
Creating the Interactive
Our team partnered with Potion Design, who led the UX/UI development for incorporating the Sketchfab viewer into our existing gesture-sensing interactives. CMA spent a significant amount of time fine-tuning the visitor experience so that users could interact with the 3D models in a rewarding way. An important part of successfully interacting with the models includes advising visitors to use only one hand—all other existing gesture-sensing games use only one hand. To communicate this, we created an “onboarding” sequence that directs users to drag an icon from one point to another across the object. Users can then grab the object anywhere to rotate or use the zoom icon to enlarge it. When the timed 3D viewer experience is complete, the object’s next game begins seamlessly.
Within ArtLens Exhibition, there are six gesture-based projection interactives featuring an attract loop of digital projections of artworks from the gallery. After selecting an artwork from the attract loop, the Sketchfab viewer is loaded into a webview in the application, and the artwork appears to zoom in and rotate. This is actually achieved by zooming in the entire web view itself and then rotating the object within Sketchfab, providing the seamless zoom and rotate introduction. Once the object is scaled to size for interaction, some simple controls appear. These controls are simply Cinder UI elements that are rendered over the web view. The user can then manipulate and control by grabbing, dragging, and releasing gestures which are captured by the Kinect camera. The grab, drag, and release gestures are translated by our app into Sketchfab API calls to rotate, tilt, or zoom the object. The object can rotate a full 360°, but we decided to limit the tilting or vertical rotation of the object to 80°. In the context of the interaction, tilting the object upside-down and then rotating it was not useful and too disorienting.
The ArtLens Exhibition games are applications coded in C++ using the Open Source creative coding library Cinder. The web view uses the Chromium Embedded Framework (CEF) and loads a custom html page containing the Sketchfab viewer and API code. The app is configurable to load the Sketchfab models of the artwork all at once or one at a time, i.e., “on demand”. Through testing, we found that loading the models one at a time delivered better performance and frame rate.
Generation of Artwork 360s
Since ArtLens Exhibition reopened, we have witnessed visitors of all ages interacting with our new 3D models. We are very grateful to have partnered with Thomas Flynn and Dale Utt, who assisted with the lighting and editing of our 3D models in Sketchfab—their expertise and enthusiasm contributed significantly to the success of this project. The next iteration of ArtLens Exhibition will open in the next 18-24 months, but in the meantime, our team will continue to produce more 3D models from our collection as time permits. Finally, look for links to our Sketchfab models on our Collection Online later this summer.