Forging a 3D Paleoart Pipeline with Sketchfab

Our work in 3D

The Natural History Museums of Los Angeles County (NHMLAC) include the Natural History Museum, the William S. Hart Museum, and what is probably the most famous paleontological dig site in the world: La Brea Tar Pits. We operate under the collective vision to inspire wonder, discovery, and responsibility for our natural and cultural worlds. Founded in 1913, NHMLAC has a long history of using paleoart to promote this vision. A growing menagerie of life-sized Ice Age animal sculptures circled the tar pits decades before a museum was built there. Today, the sculpture of a mammoth sinking into asphalt in front of La Brea Tar Pits Museum is considered one of the most iconic landmarks in Los Angeles.

Like many other museums and cultural institutions, we have been experimenting with 3D models as a way to advance our research, education, and outreach. Given our location in Los Angeles, NHMLAC is particularly well suited to be a leader in emerging technologies using 3D models like augmented reality (AR) and virtual reality (VR), collectively known as XR. L.A. is the epicenter of a burgeoning experiential economy that includes everything from VR arcades and immersive projection exhibits to high tech theme parks like Disneyland. “Silicon Beach” is home to companies such as Snap Inc., a Museum partner and industry giant in the XR space. And we are also located right across the street from the University of Southern California (USC), a leading XR research institution. With collaborators from USC, we have LiDAR scanned our historic dioramas and rigorously tested whether visitors learn more through typical museum labels or immersive AR exhibits.

Recently, we have been constructing a pipeline to develop and utilize 3D paleoart models. This is work that combines our deep history of traditional paleoart, ongoing initiatives to digitize our most important fossil collections, and our research on emerging tech like AR. Sketchfab is a crucial component of this pipeline that helps us streamline the creation of 3D models as well as maximize their impact across our mission. What we find most valuable about Sketchfab is its simplicity and interoperability among different 3D use cases.

I often tell people that 3D specimens (whether virtual or physical) need to be experienced in three dimensions. But how can we experience the same 3D specimen when we are working remotely during a pandemic? Even if we were all onsite in the same room, most museum professionals lack the expertise or software to natively view and manipulate virtual 3D objects. Below, we use two examples of recent paleoart projects at NHMLAC to illustrate how Sketchfab has helped us overcome these barriers: L.A. Underwater, a physical exhibition on L.A.’s many marine fossils that opened in April 2022 and Tar AR, an ongoing research collaboration with USC to understand how museum patrons learn with AR.

Fleshing out extinct animals

All our paleoart begins with a list of goals and objectives created by our Exhibition Team. What should this work teach to our patrons; should it make them feel awe or fear; what format will it be in, a movie, an AR lens, a printed image? Answering these questions up front helps us better define scope, final file formats, and aesthetics.

Once these broader objectives are determined, the paleoartist always works closely with a paleontologist to examine real fossil material and learn about the biology of an extinct organism. How would it have behaved? What was its role in the ancient environment? This research, combined with inferences from modern species, leads to a series of orthogonal 2D sketches, refined by the paleontologist, that will serve as the basis for an initial 3D model.

If we don’t already have a nice 3D scan of a fossil in our Sketchfab collection, we can quickly build a rough photogrammetry model of the fossil for the artist to sculpt over, ensuring biological accuracy. While these models aren’t intended for public consumption, they are very useful to the paleoartist as well as the team working behind the scenes to build the exhibition. Using the AR feature in the Sketchfab app, we could experiment with different placements and orientations of this ammonoid fossil in our physical gallery without ever having to touch the actual fossil, which is incredibly heavy and irreplaceable.

Depending on the primary use for paleoart models, we approach sculpting differently. For Tar AR, we needed to show many different animated models in AR simultaneously without overtaxing the processors on consumer grade smartphones so we commissioned Polyperfect to help us build up low poly models from the simplest shapes possible. For L.A. Underwater, the models would be in a pre-rendered cinematic wall projection so we started with high detail and later retopologized and reduced size as needed. Our paleoartist for L.A. Underwater, Cullen Townsend, does his modeling in ZBrush, roughing out the overall body shape before slowly adding sharper features and details. A sculpt can usually begin in the 10k polygon range, and typically ends up in the 10 to 15 million range. Additional features like teeth, ears, and eyes are created as separate pieces that can be accessed by the animator later. Although these raw files are too complex to view easily online, Cullen exports optimized models that can be hosted on Sketchfab for internal review.

It’s hard to review a 3D specimen in two dimensions so we use Sketchfab to host 3D files natively for our internal review process. With minimal to no 3D training, a paleontologist can easily rotate and view the model from all angles to sign off on its accuracy. Besides ease of use, the interoperability of Sketchfab also shines through here. With just one Sketchfab link, we can review the model on a desktop computer, in AR on a physical desktop, or in VR. For Tar AR, we wanted to ensure that the low poly models felt real in AR headsets. Without having to create a new Unity scene or import any files, I could just open a web browser on my desktop and immediately view all our models lifesize with our HTC Vive. Gazing up at the mammoths was truly impressive. The American lion purring next to me was downright scary.

Once the final model is approved by the paleontologist, and checked for topology errors, it is sent to an animator for UV unwrapping. Textures and color can be applied in Maya, Substance Painter, or Photoshop after UV unwrap is complete. Kain Suwannaphin, our animator on L.A. Underwater, used Maya to retopologize the high poly models, baking the detail into normal maps that wrap around much simpler meshes.

Animation (typically done in Maya) is another area where Sketchfab excels for review purposes. Loaded into Sketchfab as .fbx files, paleontologists can easily review each animation, frame by frame, from every possible angle.

The model inspector feature of Sketchfab is also a great onboarding tool for museum staff unfamiliar with the admittedly unintuitive construction of 3D models. Clicking through the various layers of the model, I can explain how it’s ok that the spines on a fish’s fin aren’t sculpted directly into the mesh because they’ll be convincingly drawn on later in the texture.

In-house graphic artists can also easily reposition and relight models within Sketchfab to produce marketing materials and other 2D assets.

The final step before models are released to the public is the most important. Each work of paleoart is a scientific hypothesis about an extinct organism. These models should be documented and submitted to peer review just like any other scientific research our paleontologists produce. Paleoart has a lengthy staying power and can influence downstream reconstructions of extinct organisms in popular media for decades. Sketchfab is again a valuable tool here. We recently published a scientific paper on the Tar AR models, documenting the scientific, aesthetic, and technical decisions that went into their creation. We demanded that the figures for this paper be embedded Sketchfab animated 3D models, not static, 2D screen grabs. Other researchers need to be able to manipulate both the models and their animations to accurately review, critique, and build off our work to produce even better paleoart in the future.

Putting paleoart out into the world

With a few simple clicks, we can take our working Sketchfab model and easily share it in a number of different ways.

• A model that students can manipulate and examine on desktop computers or tablets.
• An embedded illustration in scientific and popular articles.
• Automatic, app-free AR through QR codes.
• An infrastructure for hosting free downloads in multiple formats.

The future of 3D

While Sketchfab is an incredible resource, the paleoart pipeline isn’t perfect. Although Sketchfab can convert between a respectable number of formats, artifacts do still surface when complex models are converted automatically to glTF/USDZ for app-free AR. I also find the process for accurately setting the real world scale of models in Sketchfab cumbersome. As scientists, most of the models we upload have exact dimensions that should be maintained. Fellow researchers have no way to gauge the scale of a fossil they are examining on Sketchfab. It would be much easier if we could provide measurements in cm between two landmarks on a model rather than having to scroll back and forth, eyeballing how big it looks next to a 1 m square.

Using embedded models as scientific figures is also still problematic. Scientific journals remain for the most part, static 2D text documents. And despite its drawbacks, printed paper is a remarkably resilient file format. In fields like paleontology, modern researchers still read and cite articles and figures made over a hundred years ago. How can we ensure that researchers one hundred years from now can still experience our embedded 3D models? Perhaps Sketchfab can work with academic societies and existing data repositories like Morphosource and Zenodo to develop “model of record” standards so that 3D assets can be archived indefinitely while still remaining easy to view and manipulate in 3D for the layperson.

The world of 3D is really just beginning. Many people cite 2016’s Pokémon Go as a turning point for the adoption of AR but I actually think a more important moment occurred later in 2019 when Google started returning AR objects through its standard search. Google Tyrannosaurs rex on your phone and you’re immediately invited to view a T. rex right there in your living room. As more companies adopt this kind of app-free AR, users will start to expect that any sneakers they see in an online store can be seamlessly dragged off their screens and onto their coffee tables for examination in 3D. In time, I suspect that 3D will become just as easy to create, manipulate, and share as jpegs. By using Sketchfab to develop and share their virtual assets, museums and cultural institutions can help prepare themselves for this incredible future.